Abstract
Metacognition--the awareness and regulation of one's cognitive processes--iscentral to human adaptability in unknown situations. In contrast, currentautonomous agents often struggle in novel environments due to their limitedcapacity for adaptation. We hypothesize that metacognition is a criticalmissing ingredient in adaptive autonomous systems, equipping them with thecognitive flexibility needed to tackle unfamiliar challenges. Given the broadscope of metacognitive abilities, we focus on two key aspects: competenceawareness and strategy selection for novel tasks. To this end, we propose theMetacognition for Unknown Situations and Environments (MUSE) framework, whichintegrates metacognitive processes--specifically self-awareness andself-regulation--into autonomous agents. We present two initial implementationsof MUSE: one based on world modeling and another leveraging large languagemodels (LLMs), both instantiating the metacognitive cycle. Our systemcontinuously learns to assess its competence on a given task and uses thisself-awareness to guide iterative cycles of strategy selection. MUSE agentsshow significant improvements in self-awareness and self-regulation, enablingthem to solve novel, out-of-distribution tasks more effectively compared toDreamer-v3-based reinforcement learning and purely prompt-based LLM agentapproaches. This work highlights the promise of approaches inspired bycognitive and neural systems in enabling autonomous systems to adapt to newenvironments, overcoming the limitations of current methods that rely heavilyon extensive training data.