Abstract
Large Language Model (LLM)-based agents have recently shown impressivecapabilities in complex reasoning and tool use via multi-step interactions withtheir environments. While these agents have the potential to tackle complicatedtasks, their problem-solving process, i.e., agents' interaction trajectoryleading to task completion, remains underexploited. These trajectories containrich feedback that can navigate agents toward the right directions for solvingproblems correctly. Although prevailing approaches, such as Monte Carlo TreeSearch (MCTS), can effectively balance exploration and exploitation, theyignore the interdependence among various trajectories and lack the diversity ofsearch spaces, which leads to redundant reasoning and suboptimal outcomes. Toaddress these challenges, we propose SE-Agent, a Self-Evolution framework thatenables Agents to optimize their reasoning processes iteratively. Our approachrevisits and enhances former pilot trajectories through three key operations:revision, recombination, and refinement. This evolutionary mechanism enablestwo critical advantages: (1) it expands the search space beyond local optima byintelligently exploring diverse solution paths guided by previous trajectories,and (2) it leverages cross-trajectory inspiration to efficiently enhanceperformance while mitigating the impact of suboptimal reasoning paths. Throughthese mechanisms, SE-Agent achieves continuous self-evolution thatincrementally improves reasoning quality. We evaluate SE-Agent on SWE-benchVerified to resolve real-world GitHub issues. Experimental results across fivestrong LLMs show that integrating SE-Agent delivers up to 55% relativeimprovement, achieving state-of-the-art performance among all open-sourceagents on SWE-bench Verified. Our code and demonstration materials are publiclyavailable at https://github.com/JARVIS-Xs/SE-Agent.