Verifier-Guided Self-Consistency: Test-Time Reasoning with Imperfect Generators and Verifiers

Ekaterina Fadeeva; Xueyan Li; Mrinmaya Sachan; Maxim Panov

cs.LGNov 4, 2026

Verifier-Guided Self-Consistency: Test-Time Reasoning with Imperfect Generators and Verifiers

Authors: Ekaterina Fadeeva, Xueyan Li, Mrinmaya Sachan, Maxim Panov

Organizations: ETH Zürich · Max Planck Institute for Intelligent Systems · MBZUAI

Abstract

Large language models can solve many multi-step reasoning problems, but their reasoning remains brittle: plausible intermediate steps often lead to incorrect answers. Improving reasoning at test time requires balancing two imperfect signals: the generator, which provides diversity, and the verifier, which guides correctness. Existing approaches largely favor one side of this tradeoff. Verifier-weighted voting relies on trajectories sampled from the generator and cannot steer generation toward correct ones. By contrast, beam-style verifier-guided search can steer generation, but may over-concentrate on narrow trajectory sets and amplify verifier errors. We propose Verifier-Guided Self-Consistency (VGSC), a stochastic decoding framework that combines step-level correctness and future-outcome estimates to guide reasoning during generation. VGSC repeatedly proposes candidate next steps from the base language model and uses both signals in an acceptance/resampling procedure, while maintaining multiple trajectories for final answer-level self-consistency. We provide a probabilistic interpretation of VGSC as approximate sampling from a verifier-tilted trajectory distribution. Across multiple reasoning benchmarks, generator models, and verifier families, VGSC achieves higher average accuracy than representative baselines.

Figures & tables

Figure 1: Illustration of inference-time reasoning with imperfect generators and verifiers. Left : Weighted Self-Consistency samples complete trajectories from

P

and applies verifier weighting after generation, so it cannot recover from poor samples. Middle : Beam Search uses verifier scores during decoding, but deterministic selection can concentrate search in a narrow high-

V

region; with a noisy verifier, chains may converge to the same false-positive region. Right : VGSC samples candidate next steps from

P

and uses bounded acceptance-resampling with

V

and

Q

, balancing verifier guidance and trajectory diversity.

Figure 2: Simplified MATH example with Llama 3.2 3B Instruct and a PRM verifier. The future-outcome score $Q$ favors prefixes likely to admit a correct continuation, steering VGSC toward the correct answer.

Dataset	Domain	# Samples	Mean # Steps	Answer format
MATH	Math	500	8.4	numeric
OlympiadBench	Math	675	4.6	numeric
ScienceQA (SciQA)	Science reasoning	500	3.0	multiple-choice
CommonsenseQA (CsQA)	Commonsense reasoning	500	9.9	multiple-choice
Knights&Knaves (K&K)	Logic	100	10.5	multiple-choice

Table 1: Summary statistics of the evaluation datasets. Mean number of reasoning steps is computed for Qwen2.5 Math 1.5B Instruct using our rule-based step splitting procedure.

Method	MATH	Olympiad Bench	SciQA	CsQA	K&K	Average Accuracy	Average Tokens	Estimated TFLOPs
No-verifier baselines
Single sample	70.6 $\pm$ 0.4	33.6 $\pm$ 0.4	56.0 $\pm$ 0.4	35.1 $\pm$ 0.7	44.2 $\pm$ 3.0	47.9 $\pm$ 0.6	587	2.1
SC $N=5$	75.2 $\pm$ 0.6	35.4 $\pm$ 0.3	59.0 $\pm$ 0.7	40.9 $\pm$ 1.2	54.6 $\pm$ 1.4	53.0 $\pm$ 0.4	2,928	9.1
SC $N=25$	76.5 $\pm$ 0.5	38.7 $\pm$ 0.3	63.4 $\pm$ 0.5	46.6 $\pm$ 0.3	56.0 $\pm$ 1.0	56.2 $\pm$ 0.3	14,778	44.6
Post-hoc verifier aggregation
WSC $N=5$	78.0 $\pm$ 0.2	38.6 $\pm$ 0.3	61.5 $\pm$ 0.6	43.2 $\pm$ 0.5	51.4 $\pm$ 1.6	54.5 $\pm$ 0.4	2,964	52.0

Table 2: Accuracy

\uparrow

across datasets and average generated tokens per sample for Qwen2.5 Math 1.5B Instruct generator with Qwen2.5 Math PRM 7B verifier. Results are averaged over

5

seeds. TFLOPs are estimated using Eq. ( 39 );. The best result in each column is bold and the second-best result is underlined .

Figure 3: Accuracy–compute tradeoff on MATH, OlympiadBench, and SciQA with Qwen2.5-Math-1.5B and PRM-7B verifiers. For search methods, the budget increases as

N=C=k

, jointly scaling cross-trajectory exploration and within-step candidate generation; SC and WSC use

k^{2}

trajectories. The horizontal axis reports inference compute in TFLOPs per example.

Figure 4: Robustness to corrupted verifier scores. We perturb verifier outputs by adding Gaussian noise in logit space with standard deviation

\sigma

and then mapping back to probabilities.

Method	Guidance	$\alpha$	Avg. acc. $\uparrow$
BS	$V$	—	46.0
BS	$V\widehat{Q}$	—	53.6
BS+SC	$V$	—	57.0
BS+SC	$V\widehat{Q}$	—	60.8
VGSC	$V$ only	5	61.0
VGSC	$\widehat{Q}$ only	5	57.5

Table 3: Effect of verifier components for Qwen2.5 Math 1.5B Instruct with Qwen2.5 Math PRM 7B. Accuracy is averaged across five benchmarks. Bold entries identify the best VGSC variant within each

\alpha

group.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Post-hoc verifier aggregation
Method	Verifier Aggregation	MATH	Olympiad Bench	SciQA	CsQA	K&K	Average Accuracy
WSC	product	80.5	43.6	67.0	49.8	56.2	59.4
WSC	min	80.1	43.2	67.5	49.6	65.3	61.1
WSC	mean	77.5	39.3	64.0	48.0	55.7	56.9
DLE	product	78.6	43.8	67.5	52.8	60.4	60.6
DLE	min	79.2	42.7	67.2	52.4	58.0	59.9

Appendix

Table 4: Accuracy

\uparrow

across datasets for Qwen2.5 Math 1.5B Instruct generator with Qwen2.5 Math PRM 7B verifier. Results are averaged over

5

random seeds. The best result in each column is bold and the second-best result is underlined . Product aggregation has overall the best performance.

Figure 5: VGSC sensitivity to the verifier-trust exponent

\alpha

for

100

sample subsets of 3 datasets.

Figure 6: Allocation of search compute under a fixed budget. We fix

N\cdot C=36

and vary how the budget is split between the number of reasoning chains

N

and candidates

C

Dataset	MATH	Olympiad Bench	SciQA	CsQA	K&K	Average
$\alpha=1$	10.76%	43.66%	57.15%	63.54%	56.23%	46.27%
$\alpha=5$	16.88%	73.97%	72.00%	78.26%	68.01%	61.82%
$\alpha=10$	20.45%	80.79%	76.34%	82.59%	73.21%	66.68%
$\alpha=20$	26.12%	85.49%	80.27%	85.59%	77.48%	70.99%

Appendix

Table 5: Fraction of VGSC decoding steps that reach fallback for Qwen2.5 Math 1.5B Instruct + PRM.

No-verifier baselines
Method	MATH			OlympiadBench
Method	Accuracy $\uparrow$	Pass@N $\uparrow$	Diversity $\uparrow$	Accuracy $\uparrow$	Pass@N $\uparrow$	Diversity $\uparrow$
SC	76	80	0.487	40	46	0.551
Post-hoc verifier aggregation
WSC	78	80	0.487	45	46	0.551
Verifier-guided search baselines

Appendix

Table 6: Accuracy, average pass@N and whole-chain diversity statistics on random 100-example subsets of MATH and OlympiadBench using Qwen2.5 Math 1.5B Instruct with the Qwen2.5 Math PRM 7B verifier.

Benchmark(s)	Task format	Prompt template	Expected final answer
MATH500, OlympiadBench	Open-ended	Please reason step by step, and put your final answer within \boxed{} .	\boxed{<answer>}
ScienceQA, CommonsenseQA	Multiple choice	Solve the problem carefully. Show your reasoning in short numbered steps. Then on the last line, output only the positive integer of the answer choice in exactly this format: \boxed{(1\|2\|3\|…)} .	\boxed{k} , $k\in\{1,2,3,\dots\}$
Knights&Knaves	Multiple choice (binary)	Solve the problem carefully. Show your reasoning in short numbered steps. Then on the last line, output only the positive integer of the answer choice in exactly this format: \boxed{(1\|2)} .	\boxed{k} , $k\in\{1,2\}$

Appendix

Table 7: Prompt templates used for each benchmark group. The semantic content of the prompts is kept fixed across generator families; only the model-specific chat-template packaging is changed when needed.

Dataset	Example input question	Gold answer
MATH500	Convert the point $(0,3)$ in rectangular coordinates to polar coordinates. Enter your answer in the form $(r,\theta)$ , where $r>0$ and $0\leq\theta<2\pi$ .	$\left(3,\frac{\pi}{2}\right)$
OlympiadBench	Xenia and Sergey play the following game. Xenia thinks of a positive integer $N$ not exceeding 5000. Then she fixes 20 distinct positive integers $a_{1},a_{2},\ldots,a_{20}$ such that, for each $k=1,2,\ldots,20$ , the numbers $N$ and $a_{k}$ are congruent modulo $k$ . By a move, Sergey tells Xenia a set $S$ of positive integers not exceeding 20, and she tells him back the set $\{a_{k}:k\in S\}$ without spelling out which number corresponds to which index. How many moves does Sergey need to determine for sure the number Xenia thought of?	$2$
ScienceQA	Which figure of speech is used in this text? Sing, O goddess, the anger of Achilles son of Peleus, that brought countless ills upon the Achaeans. — Homer, The Iliad Options: (1) chiasmus (2) apostrophe	2
CommonsenseQA	A revolving door is convenient for two direction travel, but it also serves as a security measure at a what? Choices: (1) bank (2) library (3) department store (4) mall (5) new york	1
Knights&Knaves	A very special island is inhabited only by knights and knaves. Knights always tell the truth, and knaves always lie. You meet 2 inhabitants: Zoey, and Oliver. Zoey remarked, “Oliver is not a knight”. Oliver stated, “Oliver is a knight if and only if Zoey is a knave”. So who is Zoey? Choices: (1) knight (2) knave	2

Appendix

Table 8: Examples from the five evaluation datasets using the exact dataset paths and splits from our experiments. For multiple-choice datasets, we show the full prompted question including answer choices, since the model is instructed to return the boxed index of the selected option.

Setting	Value
Minimum tokens per step	10
Maximum tokens per step	2048
Maximum reasoning steps per trajectory	50
Total maximum tokens	4096
Temperature	1.0
Top- $p$ , top- $k$	1.0, 1000

Appendix

Table 9: Shared decoding settings used in experiments.

Method	Inference budget
SC	$N\in\{5,25\}$ reasoning chains
WSC	$N\in\{5,25\}$ reasoning chains
DLE	$N=25$ reasoning chains
TSMC	$N=25$ particles
BS	$N=5$ chains, $C=5$ candidate expansions
BS+SC	$N=5$ chains, $C=5$ candidate expansions

Appendix

Table 10: Method-specific inference budgets used in the main experiments.

Parameter	PRM verifier	ReProbe verifier
Architecture details	LoRA fine-tuning of Qwen2.5 Math PRM 7B; last $4$ layers only; rank $128$ ; alpha $256$ ; dropout $0.05$ ; attentions and MLP projections	Last hidden layer from LLM as features; ReProbe uses a 2-layer transformer head with dim. 768, 8 heads, and dropout $0.1$
Trainable parameters	46.1M	15m - 19m depending on base LLM
Maximum length	2048	2048
Batch size	32 (1 per device, 32 accumulation steps)	16
Learning rate	$10^{-4}$	$10^{-4}$
Epochs	5	6

Appendix

Table 11: Hyperparameters for the two verifier families. For PRM, the step verifier is frozen and the future verifier uses last-layer QLoRA; ReProbe variants are trained separately for each supervision target.

Generator +	Qwen2.5 Math 1.5B Instruct +			Llama 3.2 3B Instruct +			Qwen2.5 Math 1.5B Instruct +
Verifier	Qwen2.5 Math PRM 7B			Qwen2.5 Math PRM 7B			ReProbe
Method	Tokens	Params	TFLOPs	Tokens	Params	TFLOPs	Tokens	Params	TFLOPs
Single sample	587	1.5B	2.1	754	3.0B	5.1	638	1.5B	2.2
SC ( $N=5$ )	2,928	1.5B	9.1	3,058	3.0B	18.9	3,206	1.5B	9.9
SC ( $N=25$ )	14,778	1.5B	44.6	15,288	3.0B	92.3	16,021	1.5B	48.4
WSC ( $N=5$ )	2,964	8.5B	52.0	3,029	10.0B	62.5	3,243	1.519B	10.1

Appendix

Table 12: Estimated inference compute. FLOPs are calculated using equation ( 39 ). PRM VGSC counts a shared frozen prefix and two last-layer branches (equal-layer estimate); ReProbe VGSC counts two

19

M probes.

Method	MATH	Olympiad Bench	SciQA	CsQA	K&K	Average Accuracy	Average Tokens	Estimated TFLOPs
No-verifier baselines
Single sample	36.2 $\pm$ 0.5	11.2 $\pm$ 0.3	51.7 $\pm$ 0.6	42.3 $\pm$ 0.3	29.0 $\pm$ 1.9	34.1 $\pm$ 0.4	754	5.1
SC $N=5$	38.6 $\pm$ 0.9	10.0 $\pm$ 0.2	69.5 $\pm$ 0.7	59.8 $\pm$ 0.2	36.6 $\pm$ 2.0	42.9 $\pm$ 0.5	3,058	18.9
SC $N=25$	50.9 $\pm$ 0.1	16.4 $\pm$ 0.3	82.7 $\pm$ 0.5	71.4 $\pm$ 0.2	49.4 $\pm$ 1.7	54.2 $\pm$ 0.4	15,288	92.3
Post-hoc verifier aggregation
WSC $N=5$	46.4 $\pm$ 0.5	13.7 $\pm$ 0.3	68.5 $\pm$ 0.7	61.1 $\pm$ 0.6	34.8 $\pm$ 1.3	44.9 $\pm$ 0.3	3,029	62.5

Appendix

Table 13: Accuracy

\uparrow

across datasets and average generated tokens per sample for Llama 3.2 3B Instruct generator with Qwen2.5 Math PRM 7B verifier. Results are averaged over

5

random seeds. The best result in each column is bold and the second-best result is underlined .

Method	MATH	Olympiad Bench	SciQA	CsQA	K&K	Average Accuracy	Average Tokens	Estimated TFLOPs
No-verifier baselines
Single sample	67.2 $\pm$ 0.5	29.6 $\pm$ 0.3	68.9 $\pm$ 0.5	68.1 $\pm$ 0.7	60.0 $\pm$ 1.6	58.8 $\pm$ 0.4	845	7.5
SC $N=5$	74.7 $\pm$ 0.4	36.7 $\pm$ 0.2	75.1 $\pm$ 0.7	70.4 $\pm$ 0.4	66.8 $\pm$ 1.0	64.7 $\pm$ 0.3	4,315	35.3
SC $N=25$	78.6 $\pm$ 0.4	41.6 $\pm$ 0.2	74.3 $\pm$ 0.3	72.9 $\pm$ 0.3	67.6 $\pm$ 1.1	67.0 $\pm$ 0.2	21,581	173.4
Post-hoc verifier aggregation
WSC $N=5$	76.0 $\pm$ 0.4	37.9 $\pm$ 0.4	79.3 $\pm$ 0.4	72.8 $\pm$ 0.3	68.4 $\pm$ 1.0	66.9 $\pm$ 0.3	4,308	96.9

Appendix

Table 14: Accuracy

\uparrow

across datasets and average generated tokens per sample for Gemma3 4B Instruct generator with Qwen2.5 Math PRM 7B verifier. Results are averaged over

5

random seeds. The best result in each column is bold and the second-best result is underlined .

Method	MATH	Olympiad Bench	SciQA	CsQA	K&K	Average Accuracy	Average Tokens
No-verifier baselines
Single sample	66.8 $\pm$ 0.4	29.9 $\pm$ 0.4	46.2 $\pm$ 0.4	29.8 $\pm$ 0.4	38.0 $\pm$ 1.0	42.1 $\pm$ 0.2	638
SC ( $N=5$ )	70.0 $\pm$ 0.4	34.2 $\pm$ 0.4	57.0 $\pm$ 0.4	39.8 $\pm$ 0.4	53.0 $\pm$ 1.0	50.8 $\pm$ 0.2	3,206
SC ( $N=25$ )	73.6 $\pm$ 0.4	38.5 $\pm$ 0.4	63.8 $\pm$ 0.4	47.8 $\pm$ 0.4	54.0 $\pm$ 1.0	55.5 $\pm$ 0.2	16,021
Post-hoc verifier aggregation
WSC ( $N=5$ )	72.7 $\pm$ 0.4	35.3 $\pm$ 0.4	56.8 $\pm$ 0.4	38.0 $\pm$ 0.4	51.2 $\pm$ 1.0	50.8 $\pm$ 0.2	3,243

Appendix

Table 15: Accuracy

\uparrow

across datasets and average generated tokens per sample for Qwen2.5 Math 1.5B Instruct generator with ReProbe verifier. Results are averaged over

5

random seeds. The best result in each column is bold and the second-best result is underlined .

Language Model Decoding

Language model decoding studies how models choose and generate outputs, balancing quality, reliability, diversity, and inference cost.

See all 174 papers

Can Language Models Learn to Reject Their Own Bad Reasoning Steps?
Oct 5, 2026Siheng Xiong, Xiaoze Liu, Yiqiao Jin +2LLM Self-Correction Language Model Decoding
Georgia Institute of Technology · Purdue University
Self-Trained Verification for Training- and Test-Time Self-Improvement
May 28, 2026Chen Henry Wu, Aditi RaghunathanLLM Self-Refinement Language Model Self-Improvement
Carnegie Mellon University
Denoising Iterative Self-Correction: Structured Verification Loops for Reliable LLM Reasoning
Jun 19, 2026Shen Yin, David Ken, Joel StremmelLLM Self-Correction LLM Self-Refinement
Thomson Reuters Labs

Oct 5, 2026J/K move · Enter open · S save

Can Language Models Learn to Reject Their Own Bad Reasoning Steps?

Siheng Xiong, Xiaoze Liu, Yiqiao Jin, Xiaoqian Wang+1

Georgia Institute of Technology · Purdue University

LLM Self-Correction Language Model Decoding Efficient Language Model Reasoning LLM Reasoning Reasoning Verification

Open paper AskPDF arXiv

Verifier-guided decoding can prevent harmful reasoning steps from contaminating subsequent generation, but typically relies on an external learned verifier. We ask whether a language model can instead reject its own bad reasoning steps. We define a prefix's recoverability as the probability that the frozen generator can complete it correctly. Diagnostics show that adjacent recoverability changes are often difficult to resolve with practical Monte Carlo budgets, while same-prefix candidates exhibit a sparse low-recoverability tail. We introduce Self-Step Rejection (SSR), which trains a lightweight LoRA acceptance gate on the generator backbone while keeping the base model frozen. SSR uses confidence-qualified first-passage supervision: steps before the first resolved crossing of a root-relative recoverability barrier are accepted, the crossing step is rejected, and unresolved steps and suffixes are excluded. Training combines pointwise classification, same-prefix pairwise learning, and group-relative policy refinement using final-answer correctness. At inference, SSR accepts candidates or resamples from the unchanged prefix under rejection budgets, without an external learned verifier. Across three reasoning models and five mathematical reasoning benchmarks, SSR improves macro-average accuracy over single-pass decoding by 5.4--10.1 points using 1.21--1.40x as many generated tokens, and achieves the highest macro-average accuracy among evaluated step-level methods. Full-solution scaling methods require 4.47--8.27x the single-pass token cost for comparable performance.

Abstract

Language Model Decoding

Explore similar work

Can Language Models Learn to Reject Their Own Bad Reasoning Steps?

Self-Trained Verification for Training- and Test-Time Self-Improvement

Denoising Iterative Self-Correction: Structured Verification Loops for Reliable LLM Reasoning