Abstract
Autonomous, goal-driven agents powered by LLMs have recently emerged aspromising tools for solving challenging problems without the need fortask-specific finetuned models that can be expensive to procure. Currently, thedesign and implementation of such agents is ad hoc, as the wide variety oftasks that LLM-based agents may be applied to naturally means there can be noone-size-fits-all approach to agent design. In this work we aim to alleviatethe difficulty of designing and implementing new agents by proposing aminimalistic generation framework that simplifies the process of buildingagents. The framework we introduce allows the user to define desired agentbehaviors in a high-level, declarative specification that is then used toconstruct a decoding monitor which guarantees the LLM will produce an outputexhibiting the desired behavior. Our declarative approach, in which thebehavior is described without concern for how it should be implemented orenforced, enables rapid design, implementation, and experimentation withdifferent LLM-based agents. We demonstrate how the proposed framework can beused to implement recent LLM-based agents (e.g., ReACT), and show how theflexibility of our approach can be leveraged to define a new agent with morecomplex behavior, the Plan-Act-Summarize-Solve (PASS) agent. Lastly, wedemonstrate that our method outperforms other agents on multiple popularreasoning-centric question-answering benchmarks.