Probabilistic Soundness Guarantees in LLM Reasoning Chains

  • 2025-07-17 09:40:56
  • Weiqiu You, Anton Xue, Shreya Havaldar, Delip Rao, Helen Jin, Chris Callison-Burch, Eric Wong
  • 0

Abstract

In reasoning chains generated by large language models (LLMs), initial errorsoften propagate and undermine the reliability of the final conclusion. CurrentLLM-based error detection methods often fail to detect propagated errorsbecause they do not properly account for how earlier errors might corruptjudgments of downstream reasoning. To better detect such propagated errors, weintroduce Autoregressive Reasoning Entailment Stability (ARES), a novelprobabilistic framework that prevents error propagation by judging each claimbased only on previously-assessed sound premises. This inductive method yieldsa nuanced score for each step and provides certified statistical guarantees ofits soundness, rather than a brittle binary label. ARES achievesstate-of-the-art performance across four benchmarks (72.1% Macro-F1, +8.2points) and demonstrates superior robustness on very long synthetic reasoningchains, where it excels at detecting propagated errors (90.3% F1, +27.6points).

 

Quick Read (beta)

loading the full paper ...