Large language models (LLMs) are increasingly trusted with mathematical, scientific and engineering work, yet their mistakes are often confidently written and hard to spot. A single wrong step tends to propagate, making the model waste inference tokens following a false premise. Existing safeguards either check only the final answer and restart from scratch, or rely on a verifier LLM that competes for GPU capacity and slows down response time. We propose online, deterministic verification: many kinds of reasoning steps can be validated by code, so a step can be verified in real time, as it is written, without any model call or GPU use. Whenever a check fails, we point out the mistake and let the model continue from the same spot. We demonstrate this approach on symbolic mathematics, where derivations already contain explicit expressions that a computer algebra system can check in milliseconds. Tested on the ASyMoB benchmark with three open-weights models, our online validation improves accuracy by 1.4-5.6 percentage points, corresponding to a 1.8-14.9% net reduction in incorrect answers, while increasing token usage by only 6-7%. Because the verifier is independent of the model, the approach complements progress in LLMs rather than competing with it: every new model can benefit immediately, and the benefit grows as deterministic checkers extend to more kinds of reasoning steps.
Figures & tables
Figure 1: Real-time error detection and correction during LLM reasoning. Mathematical claims are extracted and checked asynchronously on CPU while the LLM continues generating on GPU. Passed checks leave generation uninterrupted. When a check fails, the continuation after the offending step is discarded, corrective feedback is appended, and generation resumes. Verification requires no additional model call or GPU computation.
Figure 2: Live claim extraction, verification, and online steering. (A) A plain generation proceeds uninterrupted and produces an incorrect answer. (B) During checked generation, a completed mathematical claim is extracted from the stream and dispatched for verification while the model continues decoding. A failed check interrupts generation; the continuation after the offending step is discarded, and corrective feedback is appended before generation resumes. (C) The three stages of the method, corresponding to the numbered annotations in panel B. Extraction and verification require no additional model calls.
plain
checked
Qwen3-30B-A3B-Instruct-2507
answers correct, %
62.4±0.8
68.0±0.5(×1.09)
questions correct all 5
796
958(×1.20)
tokens per answer
3,320
3,520(×1.06)
tokens discarded at rollbacks
—
2.1±0.5
checker CPU per check, ms
—
24.6
Table 1: Accuracy and inference cost with online verification. Three open-weight models were evaluated on 2,700 easy-to-intermediate ASyMOB problems, with five repeats per problem under plain and checked generation. Online checking improves accuracy across all three models with a 6–7% increase in mean tokens per answer. “Questions correct all 5” counts problems answered correctly in every repeat; “answers intervened on” reports the percentage of answers receiving at least one correction. Parenthetical multipliers give the checked-to-plain ratio. Verification runs entirely on CPU, with little generated text discarded at rollbacks.
Figure 3: Accuracy improves where the checker fires. (A) Checked-minus-plain accuracy differences, in percentage points, for flagged (orange) and unflagged (gray) answers. Accuracy increases by 6.0–10.2 points among flagged answers, while changes among unflagged answers are consistent with zero. Error bars show 95% confidence intervals over five seeds. (B) Fractions of flagged and unflagged answers under each condition; annotations report the flagged percentage. Flags are recorded online during checked generation and identified by replaying the same detection rule offline on completed plain answers.
Figure 4: Accuracy gains by baseline reliability. Problems are grouped by the number of correct plain-model answers across four repeats. Performance is evaluated on the remaining repeat and averaged over all five choices of the held-out repeat (leave-one-repeat-out evaluation). Top: Held-out accuracy for plain (blue) and checked (orange) generation. Annotations show accuracy gains in percentage points. Error bars indicate one bootstrap standard deviation across 2,000 resamples of questions, with the full grouping and evaluation procedure recomputed for each resample. Bottom: Each group’s share of all correct plain-model answers (gray) and of the total net gain in correct answers under checking (green). Each color sums to 100% within a model, up to rounding; negative contributions indicate a net loss in the corresponding group.
Large language models (LLMs) are increasingly used as graders, verifiers, and process auditors, but most mathematical evaluations still emphasize final-answer accuracy. This can obscure whether a model can verify a non-canonical but valid solution trace. We introduce a controlled linear-equation benchmark for evaluating LLMs in the evaluator role. Each instance asks the model to judge final-answer correctness, step-level trace correctness, and the first incorrect step. Our evaluation of state-of-the-art open LLMs reveals a significant robustness gap: models that accurately evaluate canonical solutions often fail when presented with perturbed but logically equivalent variants. Across GPT-OSS 20B, Qwen3-14B, and Phi-4-Reasoning, base models perform well on canonical traces but degrade substantially on perturbed traces, especially for error localization. On valid perturbed traces, base-model false-rejection rates reach 75.6-85.3%, showing strong sensitivity to canonical solution form. Supervised fine-tuning, distillation, and test-time compute improve robustness in some settings, but gains are model dependent and can trade off against canonical performance. The results show that reliable process-level verification remains challenging, and evaluator robustness should be measured separately from solver accuracy, even in a simple algebraic domain with exact ground truth.
Fateme Mazdarani, Carlos Toxtli
School of Computing Clemson University Clemson, SC, USA
Recent breakthroughs in large language models (LLMs) have led to notable successes in complex reasoning tasks, such as mathematical problem solving. A common strategy for improving performance is parallel thinking, in which multiple reasoning traces are generated and the final prediction is made using aggregation schemes like majority voting or best-of-N decoding. However, two key challenges persist. First, multi-sample decoding incurs substantial inference latency, especially for long-form outputs. Second, effective mechanisms for reliably assessing the correctness of individual reasoning traces are still limited. To address these challenges, we introduce One-Token Verification (OTV), a computational method that estimates reasoning correctness in a single forward pass during generation. OTV is activated by a learnable token and integrated into the LLM via low-rank adaptation to probe internal reasoning signals through the key-value cache, supporting token-level correctness estimation at any stage of generation without disrupting primary reasoning. Experiments on mathematical reasoning benchmarks demonstrate that OTV consistently surpasses existing verifiers. Additionally, OTV reduces token usage by up to 90% through correctness-guided early termination, prioritizing shorter, more reliable solutions.
Zhan Zhuang, Xiequn Wang, Zebin Chen +4
Southern University of Science and Technology, Shenzhen, China · City University of Hong Kong, Hong Kong SAR · Zhejiang University, Hangzhou, China
While large language models have made significant progress in mathematical reasoning, they remain unreliable at judging the correctness of their own solutions. Existing approaches that equip models with self-verification typically treat solution generation and verification as two separate tasks, leading to substantially increased training time. In this paper, we propose FABSVer, which fuses these two tasks into a single generation pass, dramatically reducing training overhead while jointly optimizing both capabilities. We further identify a convergence bottleneck both theoretically and empirically: as training progresses, the reward reaches a plateau because the policy is constrained by a fixed reference model. To overcome this, we introduce Dynamic Reference Model Update (DRMU), which raises the reward ceiling and enables sustained reward growth. Extensive experiments on math benchmarks demonstrate that FABSVer achieves superior self-verification and reasoning performance across three model scales, while requiring only 51%--71% of the training time of existing methods. Analysis further reveals distinct learning phases in how models acquire self-verification, and that the gap between verify and answer rewards shrinks noticeably as model size increases.