Abstract
Iterative human engagement is a common and effective means of leveraging theadvanced language processing power of large language models (LLMs). Usingwell-structured prompts in a conversational manner, human users can effectivelyinfluence an LLM to develop more thoughtful and accurate responses. Motivatedby this insight, we propose the Iteration of Thought (IoT) framework forenhancing LLM responses by generating "thought"-provoking prompts vis a vis aninput query and the current iteration of an LLM's response. Unlike static orsemi-static approaches, e.g. Chain of Thought (CoT) or Tree of Thoughts (ToT),IoT adapts its reasoning path dynamically, based on evolving context, andwithout generating alternate explorative thoughts which are ultimatelydiscarded. The three components of the IoT framework are (1) an Inner DialogueAgent (IDA) responsible for generating instructive, context-specific prompts;(2) an LLM Agent (LLMA) that processes these prompts to refine its responses;and (3) an iterative prompting loop that implements a conversation between theformer two components. We introduce two variants of our framework: AutonomousIteration of Thought (AIoT), where an LLM decides when to stop iterating, andGuided Iteration of Thought (GIoT), which always forces a fixed numberiterations. We investigate the performance of IoT across various datasets,spanning complex reasoning tasks from the GPQA dataset, explorativeproblem-solving in Game of 24, puzzle solving in Mini Crosswords, and multi-hopquestion answering from the HotpotQA dataset. Our results show that IoTrepresents a viable paradigm for autonomous response refinement in LLMs,showcasing significant improvements over CoT and thereby enabling more adaptiveand efficient reasoning systems that minimize human intervention.