Abstract
Large Language Models (LLMs) have shown promise as decision-makers in dynamicsettings, but their stateless nature necessitates creating a natural languagerepresentation of history. We present a unifying framework for systematicallyconstructing natural language "state" representations for prompting LLM agentsin repeated multi-agent games. Previous work on games with LLM agents has takenan ad hoc approach to encoding game history, which not only obscures the impactof state representation on agents' behavior, but also limits comparabilitybetween studies. Our framework addresses these gaps by characterizing methodsof state representation along three axes: action informativeness (i.e., theextent to which the state representation captures actions played); rewardinformativeness (i.e., the extent to which the state representation describesrewards obtained); and prompting style (or natural language compression, i.e.,the extent to which the full text history is summarized). We apply this framework to a dynamic selfish routing game, chosen because itadmits a simple equilibrium both in theory and in human subject experiments\cite{rapoport_choice_2009}. Despite the game's relative simplicity, we findthat there are key dependencies of LLM agent behavior on the natural languagestate representation. In particular, we observe that representations whichprovide agents with (1) summarized, rather than complete, natural languagerepresentations of past history; (2) information about regrets, rather than rawpayoffs; and (3) limited information about others' actions lead to behaviorthat more closely matches game theoretic equilibrium predictions, and with morestable game play by the agents. By contrast, other representations can exhibiteither large deviations from equilibrium, higher variation in dynamic game playover time, or both.