Abstract
When leveraging language models for reasoning tasks, generating explicitchain-of-thought (CoT) steps often proves essential for achieving high accuracyin final outputs. In this paper, we investigate if models can be taught tointernalize these CoT steps. To this end, we propose a simple yet effectivemethod for internalizing CoT steps: starting with a model trained for explicitCoT reasoning, we gradually remove the intermediate steps and finetune themodel. This process allows the model to internalize the intermediate reasoningsteps, thus simplifying the reasoning process while maintaining highperformance. Our approach enables a GPT-2 Small model to solve 9-by-9multiplication with up to 99% accuracy, whereas standard training cannot solvebeyond 4-by-4 multiplication. Furthermore, our method proves effective onlarger language models, such as Mistral 7B, achieving over 50% accuracy onGSM8K without producing any intermediate steps.