Abstract
Recent advances such as self-consistency and test-time reinforcement learning(TTRL) improve the reliability of large language models (LLMs) withoutadditional supervision, yet their underlying mechanisms and statisticalguarantees remain poorly understood. We present a unified framework forcertifiable inference in LLMs, showing that majority voting provides astatistical certificate of self-consistency: under mild assumptions, theaggregated answer coincides with the mode of the model's terminal distributionwith high probability. We derive finite-sample and anytime-valid concentrationbounds that quantify this confidence, and introduce the Martingale MajorityCertificate (MMC), a sequential stopping rule that adaptively determines whensufficient samples have been drawn. We further prove that label-freepost-training methods such as TTRL implicitly sharpen the answer distributionby exponentially tilting it toward its mode, thereby reducing the number ofsamples required for certification. Building on this insight, we propose newpost-training objectives that explicitly optimise this trade-off betweensharpness and bias. Together, these results explain and connect two centraltest-time scaling strategies, self-consistency and TTRL, within a singlestatistical framework for label-free, certifiable reliability in reasoningLLMs.