Abstract
Large Language Models (LLMs) based agents excel at diverse tasks, yet theysuffer from brittle procedural memory that is manually engineered or entangledin static parameters. In this work, we investigate strategies to endow agentswith a learnable, updatable, and lifelong procedural memory. We propose Mempthat distills past agent trajectories into both fine-grained, step-by-stepinstructions and higher-level, script-like abstractions, and explore the impactof different strategies for Build, Retrieval, and Update of procedural memory.Coupled with a dynamic regimen that continuously updates, corrects, anddeprecates its contents, this repository evolves in lockstep with newexperience. Empirical evaluation on TravelPlanner and ALFWorld shows that asthe memory repository is refined, agents achieve steadily higher success ratesand greater efficiency on analogous tasks. Moreover, procedural memory builtfrom a stronger model retains its value: migrating the procedural memory to aweaker model yields substantial performance gains.