Program Space Search With Llm Guided Exploration

via “graph search-based planning with hierarchical exploration”

Agent S: an open agentic framework that uses computers like a human

Unique: Implements classical graph search planning combined with LMM-based heuristics for node evaluation, enabling systematic exploration of action sequences with backtracking capabilities rather than greedy single-step decision making

vs others: Provides more systematic exploration than greedy agents through graph search, though at higher computational cost; enables recovery from dead-end paths through backtracking

via “llm-guided hierarchical task planning with dynamic subtask generation”

LLM-powered lifelong learning agent in Minecraft

Unique: Uses in-context LLM prompting with world state and skill library as context to generate task hierarchies on-the-fly, rather than relying on pre-trained planners or symbolic planning languages. Integrates execution feedback into the prompt loop to enable dynamic replanning without retraining.

vs others: More flexible than symbolic planners (PDDL, HTN) because it leverages LLM reasoning to handle open-ended, under-specified goals; more adaptive than single-policy RL agents because it replans based on execution feedback and skill availability.

via “multi-step interactive environment navigation”

* ⭐ 11/2022: [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (BLOOM)](https://arxiv.org/abs/2211.05100)

Unique: Treats environment interaction as a reasoning problem where the LLM generates actions based on observations and reasoning, rather than using reinforcement learning or imitation learning. The LLM learns the task structure from few-shot examples and generalizes to new environments without explicit training.

vs others: Achieves 34% absolute improvement over imitation and RL baselines on ALFWorld and 10% on WebShop by leveraging the LLM's reasoning capability to generalize from few examples, rather than requiring large amounts of demonstration data or reward signals.

via “program-space search with llm-guided exploration”

### Audio Processing <a name="2023ap"></a>

Unique: Uses LLM as a learned heuristic within a structured search loop rather than as a one-shot generator, combining neural guidance with deterministic evaluation to explore discrete program spaces. Implements iterative refinement where the LLM learns from failed attempts through in-context examples, enabling discovery of solutions outside typical training data distributions.

vs others: Outperforms pure LLM code generation by grounding proposals in executable feedback, and outperforms traditional program synthesis by leveraging learned heuristics to prune the search space intelligently rather than relying on exhaustive enumeration or hand-crafted rules.