Large Language Models as Optimizers (OPRO)
Product* ⏫ 10/2023: [Eureka: Human-Level Reward Design via Coding Large Language Models (Eureka)](https://arxiv.org/abs/2310.12931)
Capabilities6 decomposed
llm-based gradient-free optimization via in-context learning
Medium confidenceUses large language models as black-box optimizers by prompting them with optimization trajectories (previous solutions and their scores) to generate improved candidate solutions iteratively. The LLM learns optimization patterns from in-context examples without explicit gradient computation, treating the optimization problem as a sequence prediction task where better solutions are generated by conditioning on historical performance data.
Treats optimization as an in-context learning problem where the LLM infers optimization dynamics from trajectory history rather than using explicit gradient signals or learned surrogate models. The key architectural insight is that LLMs can act as meta-optimizers by recognizing patterns in (solution, score) pairs and generating better candidates without domain-specific training.
Outperforms traditional Bayesian optimization and evolutionary algorithms on discrete/non-differentiable problems by leveraging LLM's semantic understanding of solution space structure, while requiring no gradient computation or surrogate model training.
trajectory-conditioned solution generation with scoring feedback
Medium confidenceImplements an iterative loop where the LLM receives a formatted history of (solution, evaluation_score) pairs and generates a new candidate solution. The prompt structure encodes the optimization trajectory as in-context examples, allowing the LLM to learn implicit patterns about which solution characteristics correlate with higher scores. After evaluation, the new solution and its score are appended to the trajectory for the next iteration.
Encodes the full optimization history as in-context examples rather than using a learned surrogate model or explicit reward function. The LLM implicitly learns to recognize patterns in the trajectory (e.g., 'solutions with property X scored higher') and applies those patterns to generate the next candidate, enabling adaptation without explicit model updates.
Simpler and faster to implement than Bayesian optimization or neural surrogate models, while capturing richer semantic patterns than random search or grid search by leveraging the LLM's pre-trained understanding of solution quality.
prompt optimization via iterative refinement and scoring
Medium confidenceApplies the OPRO framework specifically to optimize natural language prompts by treating prompt text as the solution space and downstream task performance (e.g., accuracy on a benchmark) as the evaluation metric. The LLM generates improved prompt variations by analyzing which previous prompts achieved higher scores, learning to modify instruction phrasing, examples, and constraints to maximize task performance. This enables automated prompt engineering without manual trial-and-error.
Treats prompts as first-class optimization variables, using the LLM itself to generate improved prompts by analyzing which previous prompts achieved higher downstream task performance. This creates a self-improving loop where the LLM learns to write better instructions for itself or other models, without requiring gradient computation or labeled training data.
Faster and cheaper than manual prompt engineering or grid search, while more interpretable and controllable than black-box hyperparameter optimization, because the LLM generates human-readable prompts that practitioners can understand and further refine.
hyperparameter optimization via llm-guided search
Medium confidenceApplies OPRO to optimize hyperparameters (learning rates, batch sizes, regularization coefficients, etc.) by representing hyperparameter configurations as text and iteratively generating improved configurations based on their validation performance. The LLM learns implicit relationships between hyperparameter values and model performance from the trajectory history, generating candidates that balance exploration (trying new values) and exploitation (refining promising regions).
Uses the LLM's semantic understanding of numerical relationships to generate hyperparameter configurations that are more likely to improve performance, rather than random sampling or grid search. The LLM learns implicit patterns like 'smaller learning rates help with larger models' or 'higher dropout rates reduce overfitting' from the trajectory, enabling more intelligent exploration.
More interpretable than Bayesian optimization (generates human-readable configurations) and faster than random/grid search, while requiring no surrogate model training or gradient computation. However, slower than specialized AutoML tools like Optuna or Hyperband that use learned surrogates.
reward function discovery via code generation (eureka extension)
Medium confidenceExtends OPRO to automatically design reward functions for reinforcement learning by prompting an LLM to generate Python code that computes rewards based on environment observations. The LLM iteratively refines reward functions by analyzing which previous reward functions led to better task performance (e.g., higher episode returns), learning to write code that captures task-relevant objectives without manual reward engineering. This enables automated reward design for complex control tasks.
Generates reward functions as executable Python code rather than treating them as hyperparameters or learned models. The LLM learns to write code that captures task-relevant objectives by analyzing which reward functions led to better RL agent performance, enabling discovery of novel reward structures that humans might not manually design.
Eliminates manual reward engineering bottleneck in RL, enabling faster iteration and discovery of non-obvious reward structures. More flexible than inverse RL (which requires demonstrations) and more interpretable than learned reward models, though computationally expensive due to RL training cost per iteration.
multi-step reasoning trajectory generation for complex optimization
Medium confidenceExtends OPRO to handle complex optimization problems by prompting the LLM to generate multi-step reasoning or decomposed solutions rather than single-shot candidates. The LLM learns to break down optimization problems into subproblems, generate intermediate solutions, and compose them into final candidates. This enables optimization of problems with hierarchical or compositional structure, where the LLM's reasoning process itself becomes part of the optimization trajectory.
Treats the LLM's reasoning process as part of the optimization trajectory, allowing the optimizer to learn not just what solutions are good, but how to reason about generating good solutions. This enables optimization of problems where the reasoning path is as important as the final answer.
More interpretable and flexible than black-box optimization for complex problems, while leveraging LLM's reasoning capabilities to handle problems that require planning or constraint satisfaction. Slower than single-shot generation but enables optimization of problems that single-shot approaches cannot solve.
Capabilities are decomposed by AI analysis. Each maps to specific user intents and improves with match feedback.
Related Artifactssharing capabilities
Artifacts that share capabilities with Large Language Models as Optimizers (OPRO), ranked by overlap. Discovered automatically through the match graph.
Agents
Library/framework for building language agents
Mathematical discoveries from program search with large language models (FunSearch)
### Audio Processing <a name="2023ap"></a>
Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization (Retroformer)
### Other Papers <a name="2023op"></a>
Build a Large Language Model (From Scratch)
A guide to building your own working LLM, by Sebastian Raschka.
hello-agents
📚 《从零开始构建智能体》——从零开始的智能体原理与实践教程
Tutory
AI-driven tutor and teaching assistant for personalized...
Best For
- ✓Researchers optimizing prompt templates or hyperparameters for LLM tasks
- ✓Teams solving discrete optimization problems where gradient-based methods are infeasible
- ✓Practitioners needing few-shot optimization without training custom models
- ✓AutoML and neural architecture search applications
- ✓Prompt engineers optimizing instruction templates for downstream tasks
- ✓Hyperparameter tuning for machine learning models
- ✓Discrete optimization problems (e.g., combinatorial search, code generation)
- ✓Few-shot learning scenarios with limited evaluation budget
Known Limitations
- ⚠Optimization quality depends heavily on LLM's ability to recognize patterns in the trajectory history — may plateau on complex multimodal landscapes
- ⚠Each optimization step requires a full LLM forward pass, making it computationally expensive compared to gradient-based methods for large-scale problems
- ⚠No theoretical convergence guarantees; performance is empirical and problem-dependent
- ⚠Requires sufficient evaluation budget to build meaningful in-context examples; performs poorly with <5-10 prior solutions
- ⚠LLM may generate solutions that are syntactically valid but semantically nonsensical for the target domain
- ⚠Trajectory length is bounded by LLM context window; long optimization histories may be truncated or summarized, losing fine-grained signal
Requirements
Input / Output
UnfragileRank
UnfragileRank is computed from adoption signals, documentation quality, ecosystem connectivity, match graph feedback, and freshness. No artifact can pay for a higher rank.
About
* ⏫ 10/2023: [Eureka: Human-Level Reward Design via Coding Large Language Models (Eureka)](https://arxiv.org/abs/2310.12931)
Categories
Alternatives to Large Language Models as Optimizers (OPRO)
Are you the builder of Large Language Models as Optimizers (OPRO)?
Claim this artifact to get a verified badge, access match analytics, see which intents users search for, and manage your listing.
Get the weekly brief
New tools, rising stars, and what's actually worth your time. No spam.
Data Sources
Looking for something else?
Search →