Confidence-Calibrated Inference Expansion for Evaluator-Guided Test-Time Reasoning

Weida Liang; Kenji Kawaguchi

cs.AIOct 14, 2026

Confidence-Calibrated Inference Expansion for Evaluator-Guided Test-Time Reasoning

Authors: Weida Liang, Kenji Kawaguchi

Organizations: School of Computing, National University of Singapore

Abstract

Evaluator-guided reasoning presents a local control problem at every step: trust the current answer, return a fallback, or spend more inference. We formulate this choice as a compute-aware three-action controller, scoring terminal actions by predicted answer success and expansion by a rollout-based continuation-success surrogate, with compute cost priced explicitly. A local action-gap analysis characterizes when score errors can change a decision, while an expected regret bound accounts for errors along the realized expansion path. Guided by this analysis, we learn and calibrate action-specific confidence estimates, with local uncertainty penalties for ambiguous decisions. On held-out width expansion and revision tasks, the controller improves observed utility over calibration-selected fixed policies while reducing normalized online compute by about 30% and 16%, respectively. Across four Qwen3-8B seeds, compute is lower than fitted Q in both settings, while utility differences are small and mixed. The results frame evaluator-guided reasoning as resource allocation under uncertainty, rather than a uniformly stronger reasoner.

Figures & tables

Method	MATH Acc.	AIME25 Acc.	Mean Utility	Compute
Trust at $N=4$	0.742	0.469	0.601	0.240
Best fixed: majority, $N=32$	0.821	0.625	0.707	0.781
Max width: verifier, $N=48$	0.810	0.642	0.706	1.000
Majority vote, $N=48$	0.824	0.646	0.715	1.000
Margin heuristic	0.807	0.594	0.688	0.634
Uncalibrated confidence	0.819	0.608	0.701	0.611

Table 1: Scenario I on MATH and AIME25. Best fixed width is majority vote at

N=32

, selected on calibration data. Naive maximum width is verifier selection at

N=48

and sets normalized compute to 1.0.

Width policy	Mean utility	Compute
Trust at $N=4$	0.601	0.240
Verifier trust at $N=8$	0.652	0.389
Verifier trust at $N=16$	0.687	0.579
Verifier trust at $N=24$	0.694	0.721
Verifier trust at $N=32$	0.693	0.781
Verifier trust at $N=48$	0.706	1.000

Table 2: Held-out fixed-width sweep at

\lambda=0.02

. Utility is averaged across MATH and AIME25; compute is normalized to verifier trust at

N=48

Method	Success	Utility	Compute
First-draft trust	0.603	0.598	0.240
Best fixed-depth revision	0.681	0.668	0.632
Naive max-round revision	0.688	0.668	1.000
Rollback heuristic	0.668	0.657	0.557
Uncalibrated confidence	0.667	0.655	0.606
Calibrated confidence	0.680	0.668	0.579

Table 3: Scenario II on General Reasoning at

\lambda=0.02

. Fixed-depth baselines are selected on calibration data under the same objective as the adaptive controller.

Scenario	Target coverage	Utility	Compute	Expand coverage
Width	70%	0.713	0.584	0.714
Width	80%	0.715	0.566	0.803
Width	90%	0.715	0.547	0.882
Width	95%	0.716	0.522	0.934
Revision	70%	0.672	0.570	0.706
Revision	80%	0.671	0.549	0.792

Table 4: Sensitivity to nominal coverage target at

\lambda=0.02

. Expand coverage is measured on held-out states.

Scenario	Paired $\Delta U$ 95% CI	Paired $\Delta C$ 95% CI	Seed direction
Width	$-0.001$ [ $-0.006,0.004$ ]	$-0.037$ [ $-0.055,-0.020$ ]	$U$ 2/4, $C$ 4/4
Revision	$+0.002$ [ $-0.003,0.007$ ]	$-0.027$ [ $-0.040,-0.014$ ]	$U$ 3/4, $C$ 4/4

Table 5: Four-seed paired comparison with fitted Q on a fixed Qwen3-8B problem split. All methods are refit and recalibrated within each generation seed. Paired

\Delta U

and

\Delta C

are reported as Student-

t

95% intervals with three degrees of freedom.

Controller	Width $U/C$	Revision $U/C$
Shared weights, per-action calibration	0.710 / 0.595	0.662 / 0.585
Separate heads, per-action calibration	0.714 / 0.586	0.668 / 0.579
Separate heads + local uncertainty	0.715 / 0.547	0.673 / 0.529

Table 6: Controller-component ablation at

\lambda=0.02

. Cells report utility/compute.

Figure 1: Counterfactual width action shares on a common state pool (left) and revision expansion rates by round (right). The revision panel distinguishes full-pool rates from on-policy conditional hazards.

Figure 2: Retuned utility differences for compute prices

\lambda\in\{0,0.01,0.02\}

, relative to calibration-selected fixed policies.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

(a) Verifier-guided width expansion
Backbone	MATH	AIME25	Utility	Compute
Qwen3-4B	0.759	0.500	0.617	0.644
Qwen3-8B	0.829	0.615	0.711	0.563
DeepSeek-R1-Distill-Qwen-14B	0.884	0.771	0.819	0.434
Aggregate	0.824	0.628	0.715	0.547
(b) Judge-guided revision expansion

Appendix

Table 7: Per-backbone results on the held-out evaluation split. Aggregate rows are macro-averages across backbones.

Seed	Width $\Delta U$	Width $\Delta C$	Revision $\Delta U$	Revision $\Delta C$
1	$-0.005$	$-0.051$	$-0.002$	$-0.037$
2	$-0.002$	$-0.039$	$+0.001$	$-0.029$
3	$+0.001$	$-0.033$	$+0.004$	$-0.024$
4	$+0.002$	$-0.025$	$+0.005$	$-0.018$

Appendix

Table 8: Seed-level differences versus fitted Q across four independent generation seeds.

Predictive weights	Calibration	Uncertainty	Width $U/C$	Revision $U/C$
Single shared head	Global	None	0.706 / 0.603	0.657 / 0.594
Single shared head	Per action	None	0.710 / 0.595	0.662 / 0.585
Separate heads	Global	None	0.711 / 0.591	0.663 / 0.582
Separate heads	Per action	None	0.714 / 0.586	0.668 / 0.579
Separate heads	Per action	Local	0.715 / 0.547	0.673 / 0.529

Appendix

Table 9: Predictive-weight, calibration, and uncertainty ablation. Cells report utility and compute for width and revision.

Action set	Width utility	Revision utility
Trust, fallback, expand	0.715	0.673
Trust, expand	0.709	0.663

Appendix

Table 10: Utility with and without the fallback action at

\lambda=0.02

Controller	Width $U/C$	Revision $U/C$
One-shot budget-conditioned value	0.711 / 0.602	0.665 / 0.581
Receding-horizon acquisition value	0.716 / 0.565	0.671 / 0.548
Confidence + uncertainty	0.715 / 0.547	0.673 / 0.529

Appendix

Table 11: Budget-conditioned value baselines at

\lambda=0.02

. Each cell reports utility/compute; both terminal-answer options are evaluated at each candidate budget.

Scenario	Expand target	Utility	Compute
Width	Oracle continuation	0.715	0.547
Width	Policy induced	0.714	0.543
Width	One step	0.709	0.538
Revision	Oracle continuation	0.673	0.529
Revision	Policy induced	0.669	0.533
Revision	One step	0.665	0.522

Appendix

Table 12: Expand-target sensitivity. Utility is computed as

U=S-0.02C

from unrounded success and compute.

Scenario	Degradation	Utility	Compute	Expand ECE	Small-gap mismatch
Width	10% flips	0.709	0.552	0.057	0.286
Width	20% flips	0.700	0.561	0.074	0.319
Width	30% flips	0.689	0.573	0.093	0.354
Width	10% early / 30% late	0.695	0.567	0.087	0.341
Revision	10% flips	0.664	0.535	0.068	0.281
Revision	20% flips	0.654	0.546	0.086	0.316

Appendix

Table 13: Joint fitting and calibration under controlled target corruption.

(a) Width action share
Controller	Trust	Fallback	Expand
Raw confidence	0.513	0.079	0.408
Calibrated confidence	0.560	0.084	0.356
Confidence + uncertainty	0.581	0.086	0.333

Appendix

Table 14: Action-allocation diagnostics. Width shares use a common held-out state pool. Revision rates distinguish the full state pool from reached on-policy states.

Round	Success	Utility	Compute	Next fix	Next corruption
0	0.603	0.598	0.240	0.170	0.031
1	0.651	0.642	0.442	0.149	0.034
2	0.681	0.668	0.632	0.101	0.058
3	0.673	0.657	0.809	0.095	0.025
4	0.688	0.668	1.000	–	–

Appendix

Table 15: Revision outcomes by round. Each row reports full-cohort success, utility, and compute at that depth, followed by the next round’s fix and corruption rates at

\lambda=0.02

. Fix and corruption rates are computed over the full evaluation cohort, conditional on answer correctness at the preceding round. Dashes indicate that no further revision is available at the maximum depth.

Scenario	Comparator	Fitting cost, tokens	Saving per query	Break-even queries
Width	Best fixed width	$3\times 10^{7}$	2,150	$1.4\times 10^{4}$
Width	Naive max width	$3\times 10^{7}$	4,320	$7\times 10^{3}$
Revision	Fixed $r=2$	$1\times 10^{7}$	500	$2\times 10^{4}$
Revision	Naive max round	$1\times 10^{7}$	3,980	$3\times 10^{3}$

Appendix

Table 16: Approximate offline break-even query counts using measured fitting cost and online savings.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 371 papers

Expanding LLM Reasoning
Oct 4, 2026Rian Atri, Evan LuoLLM Inference Efficiency Inference-Time Scaling
Keiji AI · University of California, Berkeley
AERA: Adaptive Evidence Residual Allocation for Efficient Test-Time Reasoning
Aug 28, 2026Ziming Wang, Ivor Tsang, Hangwei QianAdaptive Inference Efficient Language Model Reasoning
National University of Singapore, Singapore · Agency for Science, Technology and Research (A*STAR), Centre for Frontier AI Research (CFAR), Singapore · Nanyang Technological University, Singapore
CALIBER: Calibrating Confidence Before and After Reasoning in Language Models
Jun 23, 2026Conor Finlay, Joshua Kurien, Saurabh Dash +2Confidence Estimation in Language Models Language Model Calibration
1Cohere Labs

Oct 4, 2026J/K move · Enter open · S save

Expanding LLM Reasoning

Rian Atri, Evan Luo

Keiji AI · University of California, Berkeley

LLM Inference Efficiency Inference-Time Scaling Efficient Language Model Reasoning Inference-Time Compute Allocation LLM Reasoning

Open paper AskPDF arXiv

Extra inference compute is usually spent on sampling more reasoning chains. We study where inside an existing chain an additional continuation should begin. We define expansion utility, the change in correctness from restarting a chain at a stored step, and measure it at every eligible step for nine models on six benchmarks (41 model and benchmark cells). Restart position matters: steps selected on one set of continuations beat uniform placement when scored on disjoint ones, in held-out audits on 5, 16, and 38 cells (+4.25 points [+2.51, +6.63] in a fresh five-cell audit). A fixed rule that restarts from the last eligible steps, always-last, is a strong baseline: our learned router beats uniform placement but shows no detected gain over it, and on DeepSeek-R1-Distill-Qwen-14B/MATH-500 always-last exceeds the exact self-consistency frontier at matched aggregate generated output by +0.052 [+0.008, +0.098], using 0.774x the aggregate generated output of four-sample self-consistency. Cross-fitted oracle selection still finds held-out headroom beyond declared positional classes, a target for future selectors. Finally, breaking step-label ties by earliest index flips the sign of a pointwise selector's gain over uniform placement in every seed of a five-seed diagnostic with four rollouts per step; randomized ties remove the bias.

Abstract

Efficient Language Model Reasoning

Explore similar work

Expanding LLM Reasoning

AERA: Adaptive Evidence Residual Allocation for Efficient Test-Time Reasoning

CALIBER: Calibrating Confidence Before and After Reasoning in Language Models