Abstract
Large language models (LLMs) have demonstrated strong performance intranslating natural language questions into SQL queries (Text-to-SQL). Incontrast, small language models (SLMs) ranging from 0.5B to 1.5B parameterscurrently underperform on Text-to-SQL tasks due to their limited logicalreasoning capabilities. However, SLMs offer inherent advantages in inferencespeed and suitability for edge deployment. To explore their potential inText-to-SQL applications, we leverage recent advancements in post-trainingtechniques. Specifically, we used the open-source SynSQL-2.5M dataset toconstruct two derived datasets: SynSQL-Think-916K for SQL generation andSynSQL-Merge-Think-310K for SQL merge revision. We then applied supervisedfine-tuning and reinforcement learning-based post-training to the SLM, followedby inference using a corrective self-consistency approach. Experimental resultsvalidate the effectiveness and generalizability of our method, SLM-SQL. On theBIRD development set, the five evaluated models achieved an average improvementof 31.4 points. Notably, the 0.5B model reached 56.87\% execution accuracy(EX), while the 1.5B model achieved 67.08\% EX. We will release our dataset,model, and code to github: https://github.com/CycloneBoy/slm_sql.