Abstract
Text-based games provide valuable environments for language-based autonomousagents. However, planning-then-learning paradigms, such as those combiningMonte Carlo Tree Search (MCTS) and reinforcement learning (RL), are notablytime-consuming due to extensive iterations. Additionally, these algorithmsperform uncertainty-driven exploration but lack language understanding andreasoning abilities. In this paper, we introduce the Monte Carlo planning withDynamic Memory-guided Large language model (MC-DML) algorithm. MC-DML leveragesthe language understanding and reasoning capabilities of Large Language Models(LLMs) alongside the exploratory advantages of tree search algorithms.Specifically, we enhance LLMs with in-trial and cross-trial memory mechanisms,enabling them to learn from past experiences and dynamically adjust actionevaluations during planning. We conduct experiments on a series of text-basedgames from the Jericho benchmark. Our results demonstrate that the MC-DMLalgorithm significantly enhances performance across various games at theinitial planning phase, outperforming strong contemporary methods that requiremultiple iterations. This demonstrates the effectiveness of our algorithm,paving the way for more efficient language-grounded planning in complexenvironments.