One-Token Verification for Reasoning Correctness Estimation

Zhan Zhuang; Xiequn Wang; Zebin Chen; Feiyang Ye; Ying Wei; Kede Ma; Yu Zhang

cs.LGMar 1, 2026

One-Token Verification for Reasoning Correctness Estimation

Authors: Zhan Zhuang, Xiequn Wang, Zebin Chen, Feiyang Ye, Ying Wei, Kede Ma, Yu Zhang

Organizations: Southern University of Science and Technology, Shenzhen, China · City University of Hong Kong, Hong Kong SAR · Zhejiang University, Hangzhou, China

Abstract

Recent breakthroughs in large language models (LLMs) have led to notable successes in complex reasoning tasks, such as mathematical problem solving. A common strategy for improving performance is parallel thinking, in which multiple reasoning traces are generated and the final prediction is made using aggregation schemes like majority voting or best-of- $N$ decoding. However, two key challenges persist. First, multi-sample decoding incurs substantial inference latency, especially for long-form outputs. Second, effective mechanisms for reliably assessing the correctness of individual reasoning traces are still limited. To address these challenges, we introduce One-Token Verification (OTV), a computational method that estimates reasoning correctness in a single forward pass during generation. OTV is activated by a learnable token and integrated into the LLM via low-rank adaptation to probe internal reasoning signals through the key-value cache, supporting token-level correctness estimation at any stage of generation without disrupting primary reasoning. Experiments on mathematical reasoning benchmarks demonstrate that OTV consistently surpasses existing verifiers. Additionally, OTV reduces token usage by up to $90\%$ through correctness-guided early termination, prioritizing shorter, more reliable solutions.

Figures & tables

Figure 1: Conceptual illustration of the proposed OTV. By reusing the KV cache and activating a LoRA-based verifier via a special token [ToT] , OTV reliably estimates the correctness of reasoning traces in a single forward pass.

		Qwen3-4B		Qwen3-8B		DAPO-Qwen-32B
		AIME24	AIME25	AIME24	AIME25	AIME24	AIME25
	Pass@128	91.46	83.13	69.58	56.98	83.75	68.75
Internal	Pass@1	60.29	46.67	26.29	19.32	51.77	36.42
	Maj@128	75.42 $\pm$ 1.61	66.46 $\pm$ 1.16	44.22 $\pm$ 2.11	28.44 $\pm$ 2.06	66.72 $\pm$ 0.93	41.77 $\pm$ 1.76
	DeepConf	77.76 $\pm$ 2.43	66.77 $\pm$ 1.18	45.73 $\pm$ 1.50	31.87 $\pm$ 2.35	66.77 $\pm$ 1.17	44.79 $\pm$ 1.65
	GenRM	79.11 $\pm$ 2.06	66.72 $\pm$ 1.82	44.38 $\pm$ 2.27	32.66 $\pm$ 2.30	66.72 $\pm$ 0.59	42.55 $\pm$ 1.66

Table 1: Weighted majority-voting accuracy (%) on AIME. For process reward models, we use the final token score as the confidence for each reasoning trace. Following Fu et al. (2025) , we use method-specific aggregation windows: confidence is computed as the mean score over the last tokens of each trace (

100

tokens for OTV;

2,048

tokens for DeepConf). We then discard the bottom

50\%

of traces by confidence before performing weighted majority voting. Each run samples

128

reasoning traces, and all results are averaged over

64

runs. We also report Pass@

128

as an oracle upper bound for parallel thinking. Within each column, the best result is highlighted in bold, and the second best is underlined.

	Best-of- $N$		Drop@ $10$		Stop@ $600$		Halve@ $300$
	AIME24	AIME25	AIME24	AIME25	AIME24	AIME25	AIME24	AIME25
Qwen3-4B
DeepConf	64.95 (9664)	43.07 (9322)	62.86 (7044)	40.78 (7260)	59.90 (7150)	40.89 (7084)	61.98 (6742)	43.54 (6318)
VersaPRM-8B	54.48 (6560)	43.28 (6132)	60.52 (2589)	43.44 (6334)	63.75 (3270)	46.77 (6113)	59.06 (2981)	37.24 (6438)
Math-Shepherd-7B	73.59 (5820)	66.51 (5824)	61.61 (5679)	46.25 (6116)	57.45 (5989)	47.45 (6218)	54.64 (5722)	45.36 (6118)
Qwen2.5-PRM800K-7B	69.90 (4891)	45.10 (6202)	66.77 (2196)	42.86 (6445)	60.31 (3434)	49.53 (6409)	62.24 (2919)	45.21 (6619)

Table 2: Accuracy (%) and average output length (in parentheses) on AIME at

N=128

under Best-of-

N

and three compute-efficient variants ( i.e ., Drop@

10

, Stop@

600

, and Halve@

300

). All results are averaged over

64

runs.

Figure 2: Confidence dynamics on three representative AIME24 problems ( i.e ., #3, #9, and #22). For each predictor, we plot the mean confidence trajectory over

32

sampled reasoning traces, shown separately for traces that end with correct (red) and incorrect (green) final answers. Shaded bands around each mean curve denote the inter-quantile range across traces, summarizing cross-trace variability.

Figure 3: Effect of verifier capacity ( i.e ., LoRA rank) on training dynamics and downstream voting accuracy. Left: verifier training loss over optimization steps for the “probe” baseline, which trains only the regression head (no LoRA; no KV cache) and for OTV with varying LoRA ranks. Middle/Right: Weighted majority-voting accuracy on AIME, as a function of the number of sampled traces. All results are averaged over $64$ runs.

	Maj@ $128$		Best-of- $N$		Halve@ $300$
	AIME24	AIME25	AIME24	AIME25	AIME24	AIME25
Constant label	81.41 $\pm$ 1.65	68.49 $\pm$ 2.04	72.76 (4,340)	54.27 (4,362)	66.41 (3,951)	49.27 (4,503)
Sigmoid ramp	80.05 $\pm$ 1.91	66.51 $\pm$ 0.92	77.40 (4,617)	56.41 (4,426)	68.44 (4,210)	46.30 (4,294)
Noise-perturbed ramp	81.87 $\pm$ 1.65	67.92 $\pm$ 1.61	72.60 (5,226)	54.90 (5,152)	71.30 (4,134)	48.33 (5,736)
Stepwise ramp	82.40 $\pm$ 1.50	68.80 $\pm$ 1.60	77.29 (5,351)	55.36 (5,452)	70.63 (4,231)	37.60 (7,095)
Linear ramp (Default)	83.33 $\pm$ 1.57	69.32 $\pm$ 1.46	73.44 (5,447)	53.91 (5,416)	67.03 (4,132)	49.02 (3,170)

Table 3: Ablation of token-level pseudo-confidence labeling heuristics for training OTV.

	$\max$				$\mathrm{mean}$				$\min$
	$0\%$	$25\%$	$50\%$	$75\%$	$0\%$	$25\%$	$50\%$	$75\%$	$0\%$	$25\%$	$50\%$	$75\%$
All tokens	75.78	76.82	80.42	80.36	75.94	79.53	82.19	80.00	80.10	80.83	81.25	79.17
Last $10\%$ tokens	76.77	80.21	82.76	81.46	82.19	83.02	83.28	82.19	83.07	83.07	83.12	82.29
Last $1,600$ tokens	75.78	78.65	81.25	82.14	80.99	82.71	83.23	82.24	83.23	83.23	82.97	79.95
Last $400$ tokens	77.50	80.36	82.66	82.86	81.93	82.81	83.33	83.23	82.86	83.23	83.28	83.75
Last $100$ tokens	78.39	80.21	82.34	81.82	82.08	82.97	83.33	82.76	82.86	82.60	82.76	83.96

Table 4: Effect of trace-level confidence aggregation and filtering on weighted majority voting. We convert OTV’s token-level confidence estimates into a single trace-level score by aggregating over the last few tokens, discard the lowest-confidence

\rho

fraction of traces ( i.e .,

0\%

25\%

50\%

, and

75\%

), and evaluate three aggregation operators.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Evaluation on GSM8K using Qwen3-4B-Base .

	$N$	1	4	8	16	32	64
LLaMA-7B	Pass@ $k$	61.87 $\pm$ 0.74	75.01 $\pm$ 0.79	81.07 $\pm$ 0.49	85.36 $\pm$ 0.48	88.68 $\pm$ 0.35	91.27 $\pm$ 0.38
	Maj@ $k$	61.87 $\pm$ 0.74	67.29 $\pm$ 0.88	69.57 $\pm$ 0.43	70.53 $\pm$ 0.44	71.47 $\pm$ 0.30	71.79 $\pm$ 0.30
	DeepConf	61.87 $\pm$ 0.74	68.64 $\pm$ 0.80	70.63 $\pm$ 0.60	71.76 $\pm$ 0.58	72.06 $\pm$ 0.20	72.32 $\pm$ 0.21
	OTV ( $r=2$ )	61.87 $\pm$ 0.74	71.13 $\pm$ 0.74	73.50 $\pm$ 0.47	74.55 $\pm$ 0.52	74.80 $\pm$ 0.37	75.16 $\pm$ 0.22
	OTV ( $r=16$ )	61.87 $\pm$ 0.74	70.72 $\pm$ 0.81	72.82 $\pm$ 0.35	73.80 $\pm$ 0.32	74.37 $\pm$ 0.28	74.83 $\pm$ 0.29
Mistral-7B	Pass@ $k$	66.02 $\pm$ 1.18	83.16 $\pm$ 0.65	88.46 $\pm$ 0.50	91.58 $\pm$ 0.44	93.99 $\pm$ 0.28	95.87 $\pm$ 0.24

Appendix

Table 5: Weighted majority-voting accuracy on GSM8K using MetaMath-LLaMA-7B and MetaMath-Mistral-7B as backbone models. Results are reported as mean

\pm

standard deviation over

64

runs.

Semantics-preserving modifications
Type	Text
Ground-truth	It takes $2/2=1$ bolt of white fiber\nSo the total amount of fabric is $2+1=3$ bolts of fabric\n#### 3
Original	To solve this problem, we need to determine the total number of bolts required for the robe, which includes both blue and white fiber.\n\nStep 1: Identify the amount of blue fiber needed.\nThe robe takes 2 bolts of blue fiber.\n\nStep 2: Calculate the amount of white fiber needed.\nThe robe takes half as much white fiber as blue fiber. Since it takes 2 bolts of blue fiber, it will take 2 / 2 = 1 bolt of white fiber.\n\nStep 3: Add the amounts of blue and white fiber to find the total number of bolts.\nTotal bolts = Blue fiber + White fiber\nTotal bolts = 2 + 1\nTotal bolts = 3.
Shortening	To find total bolts for the robe (blue + white fiber):\n\nStep 1: Blue fiber needed is 2 bolts.\n\nStep 2: White fiber is half blue, so 2/2=1 bolt.\n\nStep 3: Total = 2+1=3.
Lengthening	To solve the problem of determining the total number of bolts required for the robe, we must consider both the blue fiber and white fiber that are necessary for its construction.\n\nStep 1: First, we need to identify the specific amount of blue fiber that is needed to make the robe. After checking the relevant specifications or requirements for the robe, it is determined that the robe takes 2 bolts of blue fiber.\n\nStep 2: Next, we need to calculate the amount of white fiber that is needed. According to the given information, the robe takes half as much white fiber as blue fiber. Since we already established that it takes 2 bolts of blue fiber, we can find the amount of white fiber by taking half of that number. To do this, we perform the calculation: 2 bolts (blue fiber) divided by 2, which equals 1 bolt. Therefore, the robe requires 1 bolt of white fiber.\n\nStep 3: Finally, to find the total number of bolts required for the robe, we need to add together the amounts of blue fiber and white fiber that we have determined. This means we take the 2 bolts of blue fiber and add the 1 bolt of white fiber to it. Performing this addition gives us: 2 + 1 = 3.
Removing newline token	To solve this problem, we need to determine the total number of bolts required for the robe, which includes both blue and white fiber. Step 1: Identify the amount of blue fiber needed. The robe takes 2 bolts of blue fiber. Step 2: Calculate the amount of white fiber needed. The robe takes half as much white fiber as blue fiber. Since it takes 2 bolts of blue fiber, it will take 2 / 2 = 1 bolt of white fiber. Step 3: Add the amounts of blue and white fiber to find the total number of bolts. Total bolts = Blue fiber + White fiber. Total bolts = 2 + 1. Total bolts = 3. Answer: 3.

Appendix

Table 6: Representative perturbed solution variants used in the sensitivity analysis. Each row applies a controlled edit to the same base solution to evaluate how the verifier’s confidence responds to semantics-preserving vs. logic-breaking modifications.

Panel of Figure 5 — Figure 5: Trace-level confidence trajectories across problems in (a) AIME24 and (b) AIME25.

Reasoning Verification

Reasoning verification checks whether intermediate steps and final answers are logically valid and supported by evidence.

See all 57 papers

Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoning
Feb 20, 2026Lexiang Tang, Weihao Gao, Bingchen Zhao +4Efficient Language Model Reasoning LLM Reliability
Peking University, Beijing, China · University of Edinburgh, Edinburgh, UK.
Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens
Feb 13, 2026Wei-Lin Chen, Liqian Peng, Tian Tan +5LLM Inference Efficiency Inference-Time Scaling
University of Virginia · Google
COMPASS: Finding Where Reasoning Lives in Language Models
Oct 5, 2026Pratyay Dutta, Kowshik Thopalli, Vivek NarayanaswamyEfficient Language Model Reasoning Activation Steering
University of California, Riverside, CA 92521. Work performed during a summer internship at Lawrence Livermore National Laboratory · Lawrence Livermore National Laboratory, Livermore, CA 94550.

Feb 20, 2026J/K move · Enter open · S save

Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoning

Lexiang Tang, Weihao Gao, Bingchen Zhao, Lu Ma+3

Peking University, Beijing, China · University of Edinburgh, Edinburgh, UK.

Efficient Language Model Reasoning LLM Reliability Contrastive Decoding Numerical Reasoning in Language Models

Open paper AskPDF arXiv

Recent work on test-time scaling for large language model (LLM) reasoning typically assumes that allocating more inference-time computation uniformly improves correctness. However, prior studies show that reasoning uncertainty is highly localized: a small subset of low-confidence tokens disproportionately contributes to reasoning errors and unnecessary output expansion. Motivated by this observation, we propose Thinking by Subtraction, a confidence-driven contrastive decoding approach that improves reasoning reliability through targeted token-level intervention. Our method, Confidence-Driven Contrastive Decoding, detects low-confidence tokens during decoding and intervenes selectively at these positions. It constructs a contrastive reference by replacing high-confidence tokens with minimal placeholders, and refines predictions by subtracting this reference distribution at low-confidence locations. Experiments show that CCD significantly improves accuracy across mathematical reasoning benchmarks while substantially reducing output length, with minimal KV-cache overhead. As a training-free method, CCD enhances reasoning reliability through targeted low-confidence intervention without computational redundancy. Our code will be made available at: https://github.com/bolo-web/CCD.

Abstract

Reasoning Verification

Explore similar work

Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoning

Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens

COMPASS: Finding Where Reasoning Lives in Language Models