Abstract
Although large language models (LLMs) have achieved remarkable performanceacross various tasks, they remain prone to errors. A key challenge is enablingthem to self-correct. While prior research has relied on external tools orlarge proprietary models, this work explores self-correction in small languagemodels (SLMs) through iterative fine-tuning using solely self-generated data.We introduce the Self-Taught Self-Correction (STaSC) algorithm, whichincorporates multiple algorithmic design choices. Experimental results on aquestion-answering task demonstrate that STaSC effectively learnsself-correction, leading to significant performance improvements. Our analysisfurther provides insights into the mechanisms of self-correction and the impactof different design choices on learning dynamics and overall performance. Tosupport future research, we release our user-friendly codebase and lightweightmodels.