Abstract
Human reasoning is flexible, adaptive, and grounded in priorexperience-qualities that large language models (LLMs) still struggle toemulate. Existing methods either explore diverse reasoning paths at inferencetime or search for optimal workflows through expensive operations, but bothfall short in leveraging multiple reusable strategies in a structured,efficient manner. We propose Guideline Forest, a framework that enhances LLMsreasoning by inducing structured reasoning strategies-called guidelines-fromverified examples and executing them via step-wise aggregation. Unliketest-time search or single-path distillation, our method draws on verifiedreasoning experiences by inducing reusable guidelines and expanding each intodiverse variants. Much like human reasoning, these variants reflect alternativethought patterns, are executed in parallel, refined via self-correction, andaggregated step by step-enabling the model to adaptively resolve uncertaintyand synthesize robust solutions.We evaluate Guideline Forest on fourbenchmarks-GSM8K, MATH-500, MBPP, and HumanEval-spanning mathematical andprogrammatic reasoning. Guideline Forest consistently outperforms strongbaselines, including CoT, ReAct, ToT, FoT, and AFlow. Ablation studies furtherhighlight the effectiveness of multi-path reasoning and stepwise aggregation,underscoring the Guideline Forest's adaptability and generalization potential.