Abstract
LLMs can perform multi-step reasoning through Chain-of-Thought (CoT), butthey cannot reliably verify their own logic. Even when they reach correctanswers, the underlying reasoning may be flawed, undermining trust inhigh-stakes scenarios. To mitigate this issue, we introduce VeriCoT, aneuro-symbolic method that extracts and verifies formal logical arguments fromCoT reasoning. VeriCoT formalizes each CoT reasoning step into first-orderlogic and identifies premises that ground the argument in source context,commonsense knowledge, or prior reasoning steps. The symbolic representationenables automated solvers to verify logical validity while the NL premisesallow humans and systems to identify ungrounded or fallacious reasoning steps.Experiments on the ProofWriter, LegalBench, and BioASQ datasets show VeriCoTeffectively identifies flawed reasoning, and serves as a strong predictor offinal answer correctness. We also leverage VeriCoT's verification signal for(1) inference-time self-reflection, (2) supervised fine-tuning (SFT) onVeriCoT-distilled datasets and (3) preference fine-tuning (PFT) with directpreference optimization (DPO) using verification-based pairwise rewards,further improving reasoning validity and accuracy.