On the Limits of Test-Time Compute for Self-Evaluation

Anirudh Buvanesh; Khaled Rouissi; Sujin Yun; Johan Obando-Ceron; Morgane M Moss; Samuel Lavoie; Michael Noukhovitch; Laurent Charlin

cs.LGOct 30, 2026

On the Limits of Test-Time Compute for Self-Evaluation

Authors: Anirudh Buvanesh, Khaled Rouissi, Sujin Yun, Johan Obando-Ceron, Morgane M Moss, Samuel Lavoie, Michael Noukhovitch, Laurent Charlin,

Organizations: Mila · Université de Montréal · Nous Research · Google ·

Abstract

Self-verification and aggregation have emerged as popular approaches for bridging the gap between pass@1 and pass@N. However, increasing the number of candidates does not consistently narrow the gap between final-answer accuracy and pass@N. To understand this limitation, we use ranking error to measure how reliably models distinguish their correct generations from incorrect ones. We show that ranking error admits a bias-variance decomposition, with both components shaped by algorithmic design choices and inference compute. Across our experiments, additional compute reduces variance, while bias initially decreases before approaching a plateau. Guided by this analysis, we introduce Scorpio, a simple and general method that jointly scores randomly grouped candidates. Across math, code, and puzzle benchmarks, Scorpio improves verification, tree search, and aggregation. For verification, Scorpio improves accuracy over the strongest baseline by up to 7 percentage points while matching or exceeding that baseline's peak accuracy with 17-42% lower inference cost. For tree search, using Scorpio to score partial solutions and select the final answer improves accuracy by up to 10 percentage points over prior methods. Finally, Scorpio improves the scaling of aggregation-based methods by 11 percentage points on coding tasks.

Figures & tables

Figure 1: Sampling more candidates improves the chance of a correct answer, but selecting it does not scale well with

N

. (a) The gap between

\mathrm{pass@}1

and

\mathrm{pass@}N

exists even for frontier models on agentic tasks. 2 2 2 Accuracies from the Terminal-Bench 4 leaderboard . (b) As

N

increases, the gap between

\mathrm{pass@N}

and the selection accuracy of V 1 ( Singh et al., 2026 ) , a recent self-verification method for scaling test-time compute, widens. (c) A similar trend holds for Recursive Self-Aggregation (RSA) ( Venkatraman et al., 2025 ) . For (b) and (c), we run each method until accuracy plateaus; refer Sec. E.1 for further details.

Figure 2: Scorpio scales better with inference compute than prior verification methods across math (HMMT), puzzle (RG Games), and code benchmarks (LCBv6). We compare Scorpio ’s scoring with V1 ( Singh et al., 2026 ) , a recent test-time scaling algorithm that scores candidates through pairwise comparisons between similar candidates, and pointwise scoring, where LLMs evaluate candidates in isolation. Scorpio jointly scores candidates in groups of four. Inference compute is measured as the total dollar cost of all input and output tokens, and is scaled by repeating candidate evaluations. Results are averaged over 20 seeds, with shaded regions indicating

95\%

confidence intervals.

Figure 3: Decomposition of ranking error into bias and variance. Results for GPT-OSS-20B on HMMT (math) and LCBv6 (code). As inference compute increases, variance generally decreases, while bias decreases before approaching a plateau (see Sec. C.3 for why bias can decrease with inference compute). Despite its simpler design, Scorpio reduces variance faster than pointwise scoring and V1’s ( Singh et al., 2026 ) pairwise scoring, while achieving similar or lower ranking error. Similar trends are observed for Qwen3.5-4B (see Fig. 17 ).

Figure 4: Larger groups reduce variance faster but can increase bias. Results for GPT-OSS-20B on HMMT (math) and LCBv6 (code). Increasing the group size allows more evaluations within the same cost budget, reducing variance faster. However, these gains can come at the expense of increased bias: on LCBv6,

K=8

yields higher bias than smaller groups. Similar trends are observed for Qwen3.5-4B (see Fig. 18 ).

Figure 5: Residual bias persists at scale with Gemini-3.1-Pro on Terminal-Bench-2.1. We generate

N=8

candidates and score them with Scorpio . Top-1 accuracy for Scorpio (

K=4

) plateaus about

9

percentage points below

\mathrm{pass@}8

. Scorpio ’s bias plateaus with compute, while its variance continues to decrease. Results are averaged across

8

seeds.

Figure 6: Test-time search using Scorpio matches or outperforms prior methods without any training. Results for GPT-OSS-20B on AIME 2024 and LCBv6. We compare Scorpio , which jointly scores partial solutions during search, with ThinkPRM-1.5B ( Khalifa et al., 2025 ) , which uses a trained process reward model, and DeepConf, which uses confidence scores derived from token probabilities. We vary the number of initial solutions:

N\in\{8,32,128\}

for Scorpio and ThinkPRM, and

N\in\{16,64,256\}

for DeepConf to keep total costs comparable. Top-1 accuracy is plotted against total inference cost. Similar trends are observed for Qwen3.5-4B (see Fig. 20 ).

Panel of Figure 7 — Figure 7: Pruning candidates with Scorpio allows RSA to scale with pool size $N$ . Using GPT-OSS-20B on LCBv6, we plot (a) accuracy across $L=1\to 10$ aggregation rounds for different pool sizes $N$ . Increasing $N$ degrades RSA performance, whereas RSA + Scorpio improves with larger pools. One failure mode contributing to this degradation is shown in (b) : RSA amplifies rare incorrect responses over successive rounds on problem 120. LCBv6 requires a complete Python solution in a single turn and does not permit tool calls. However, RSA amplifies rare incorrect responses containing tool calls over successive rounds, with the effect becoming more severe as $N$ increases (see Fig. 22 ). After each round, Scorpio scores all $N$ candidates, and only the top $N/2$ are used to construct aggregation sets for the next round, limiting this bad-mode amplification. Results are averaged over 8 seeds, with $95\%$ CI.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Increasing compute does not change the mean of the score distribution, but can reduce the bias in rankings. At low compute (solid curves), overlapping score distributions allow the incorrect candidate to receive a higher score than the correct one. Averaging more evaluations (dotted curves) concentrates the scores around their respective means. Since the correct candidate has a higher mean score in this example, this makes an incorrect ordering less likely, reducing the pair’s squared bias

(\mathbb{E}[Z_{ij}])^{2}

Figure 9: Residual bias dominates ranking error as compute grows. Variance and squared bias of the ranking error of Scorpio (

K=4

) versus the number of scoring rounds

M

for Qwen3.5-4B (top) and GPT-OSS-20B (bottom). Black circles show the squared bias, which flattens as

M

grows, and teal squares the variance; curves show least-squares fits of

a+b/\sqrt{M}

to the variance. The box gives the fitted

a

b

, and

R^{2}

for the variance, and the share of the ranking error due to squared bias at

M=20

. Using the asymptotic variance

a

together with the squared bias at

M=20

, we estimate that bias would account for

91\%

99\%

of the ranking error as

M

grows.

Figure 10: A judge from a different model can reduce ranking bias. Candidates generated by Qwen3.5-4B are scored with Scorpio (

K=4

) either by Qwen3.5-4B itself (the self judge) or by one of four other judges, on HMMT Feb 2025 (top) and LCBv6 (bottom). We plot bias 2 , variance, ranking error, and top-1 accuracy as the number of scoring rounds

M

increases, averaged over

20

repetitions; shaded regions show

95\%

confidence intervals for top-1 accuracy. At

M=20

, replacing the self judge with GPT-OSS-20B reduces the ranking error from

0.17

0.07

on HMMT and from

0.23

0.13

on LCBv6, and improves top-1 accuracy from

67.6\%

75.0\%

and from

57.0\%

71.6\%

, respectively.

Figure 11: Different models have different biases. For every pair of a correct and an incorrect candidate generated by Qwen3.5-4B, we record whether the self judge and another judge rank the correct candidate higher after

M=20

scoring rounds of Scorpio (

K=4

), on HMMT Feb 2025 (top) and LCBv6 (bottom). Fractions are averaged over questions and

20

repetitions.

Figure 12:

\mathrm{V}_{1}

( Singh et al., 2026 ) accuracy plateaus well below

\mathrm{pass@}N

, and the gap widens as

N

increases. Results for GPT-OSS-20B on LCBv6. (a) Increasing the candidate pool from

N=8

N=128

yields modest gains in selection accuracy, while the gap to

\mathrm{pass@}N

grows from roughly

5

11

percentage points. Points show means over

20

seeds, with error bars indicating

95\%

confidence intervals. (b) Accuracy plateaus as inference compute increases for each pool size, remaining below the corresponding

\mathrm{pass@}N

ceiling (dashed lines). Inference compute is measured as the total cost of the input and output tokens of each LLM call; see Sec. D.2 for further details.

Figure 13: RSA’s ( Venkatraman et al., 2025 ) final accuracy decreases as the candidate pool grows. Results for GPT-OSS-20B on LCBv6. (a) Increasing

N

from

8

128

lowers final accuracy, widening the gap to

\mathrm{pass@}N

from roughly

12

26

percentage points. Points show means over

8

seeds, with error bars indicating

95\%

confidence intervals. (b) Each curve tracks accuracy over successive aggregation rounds for a fixed

N

. Larger pools plateau at lower accuracy despite higher compute costs. Inference compute is measured as the total cost of the input and output tokens of each LLM call; see Sec. D.2 for further details.

Figure 14:

\mathrm{V}_{1}

( Singh et al., 2026 ) also exhibits a widening selection gap on HMMT Feb 2025. Results for GPT-OSS-20B. (a) Increasing

N

from

8

128

yields modest gains in selection accuracy, while the gap to

\mathrm{pass@}N

grows from roughly

6

11

percentage points. (b) Selection accuracy plateaus as inference compute increases, remaining below the corresponding

\mathrm{pass@}N

ceiling (dashed lines). Inference compute is measured as the total cost of the input and output tokens of each LLM call; see Sec. D.2 for further details.

Figure 15: Scorpio achieves higher accuracy on math and similar accuracy at lower cost on code compared with PPT ( Kwok et al., 2026 ) . Selection accuracy versus total inference cost for GPT-OSS-20B with

N=128

candidates. On HMMT Feb 2025, Scorpio (

K=4

) improves peak accuracy by 8 percentage points over PPT. On LCBv6, it reaches PPT’s accuracy at 1.5

\times

lower total cost. PPT’s strategy proceeds in two phases: an initial ring phase ensures that every candidate is evaluated, followed by a tournament phase. Points labeled “ring” show accuracy after the first phase, while

k=1

and

k=2

indicate whether one or two evaluations are used per pair. Curves show mean accuracy, with shaded regions indicating

95\%

confidence intervals. Inference compute is measured as the total dollar cost of all input and output tokens across LLM calls; see Sec. D.2 for details.

Figure 16: Random pairing matches or outperforms tournament-based pairing. Results for GPT-OSS-20B on HMMT and LCBv6. Bias-variance decomposition of ranking error for

\mathrm{V}_{1}

( Singh et al., 2026 ) and Scorpio (

K=2

) using GPT-OSS-20B. On both datasets, variance decreases quicker with Scorpio as compute increases.

\mathrm{V}_{1}

attains lower bias on LCBv6, while Scorpio attains lower bias on HMMT. At higher budgets, Scorpio achieves similar or lower ranking error, demonstrating that a simpler pairing strategy can be equally or more effective than tournament based strategies. Inference compute is measured as the total dollar cost of all input and output tokens across LLM calls; see Sec. D.2 for details.

Figure 17: Decomposition of the ranking error as a bias term and a variance term. Results for Qwen3.5-4B on HMMT, Reasoning-Gym and LCBv6. While the variance term generally vanishes with increased compute, the bias term converges to a plateau (see: Sec. C.3 ). Scorpio ’s groupwise scoring leads to a higher variance reduction rate than pointwise or V1’s ( Singh et al., 2026 ) pairwise scoring. Compute is measured as the total dollar cost of all input and output tokens across LLM calls; see Sec. D.2 for further details.

Figure 18: Larger groups reduce variance faster but can increase bias. Results for Qwen3.5-4B on HMMT and LCBv6. Increasing group size allows more evaluations within the same cost budget, reducing variance faster. However, these gains can come at the expense of bias: on LCBv6,

K=8

yields higher bias than smaller groups. Compute is measured as the total dollar cost of all input and output tokens across LLM calls; see Sec. D.2 for further details.

Figure 19: DeepConf’s ( Fu et al., 2025 ) pruning threshold depends on the model. Top-1 Accuracy versus total inference cost for DeepConf with aggressive (

\eta=10

) and conservative (

\eta=90

) pruning. Aggressive pruning generally works better for Qwen3.5-4B, while conservative pruning works better for GPT-OSS-20B. Compute is measured as the total dollar cost of all input and output tokens across LLM calls; see Sec. D.2 for further details.

Figure 20: Test-time search using Scorpio matches or outperforms prior methods. Results for Qwen3.5-4B on AIME 2024 and LCBv6. Top-1 accuracy of test-time search versus total inference cost on AIME 2024 and LCBv6 for Qwen3.5-4B. We vary the number of solutions we start with:

N\in\{8,32,128\}

for Scorpio and ThinkPRM, and

N\in\{16,64,256\}

for DeepConf to keep total costs comparable. Inference compute is measured as the total dollar cost of all input and output tokens across LLM calls; see Sec. D.2 for details.

Figure 21: Repeated evaluation of partial solutions improves search accuracy. Top-1 accuracy of test-time search using Scorpio versus total inference cost on AIME 2024, HMMT Feb 2025, and LCBv6. We fix the number of solutions we start with at

N=128

and vary the number of times the random grouping and scoring process is repeated,

M\in\{1,2,3\}

. Repeated evaluations improve accuracy over

M=1

across all three datasets. Inference compute is measured as the total dollar cost of all input and output tokens across LLM calls; see Sec. D.2 for details.

Figure 22: Larger pools amplify tool-call responses in RSA, while Scorpio limits their amplification. Proportion of responses containing tool calls across aggregation rounds for a single LCBv6 problem using GPT-OSS-20B. This is a single-turn task that requires a solution in a Python code block. Some responses attempt tool calls to test their logic, which we identify using the pattern to=[A-Za-z_] . The harness does not support tool calls, and

95\%

of these responses receive zero reward. Tool-call responses remain uncommon with

N=8

but become increasingly common with

N=32

and

N=128

as aggregation rounds progress. In RSA + Scorpio , candidates are scored using Scorpio after each round, and only the top half of candidates are used to build aggregation sets for the next round. This selection reduces the proportion of tool-call responses to nearly zero across all three pool sizes.

Language Model Self-Assessment

Language model self-assessment studies how models judge their own confidence, capabilities, errors, and behavior.

See all 96 papers

Mar 4, 2026

$V_1$ : Unifying Generation and Self-Verification for Parallel Reasoners

Test-time scaling for complex reasoning tasks shows that leveraging inference-time compute, by methods such as independently sampling and aggregating multiple solutions, results in significantly better task outcomes. However, a critical bottleneck is verification: sampling is only effective if correct solutions can be reliably identified among candidates. While existing approaches typically evaluate candidates independently via scalar scoring, we demonstrate that models are substantially stronger at pairwise self-verification. Leveraging this insight, we introduce

V_1

, a framework that unifies generation and verification through efficient pairwise ranking.

V_1

comprises two components:

V_1

-Infer, an uncertainty-guided algorithm using a tournament-based ranking that dynamically allocates self-verification compute to candidate pairs whose relative correctness is most uncertain; and

V_1

-PairRL, an RL framework that jointly trains a single model as both generator and pairwise self-verifier, ensuring the verifier adapts to the generator's evolving distribution. On code generation (LiveCodeBench, CodeContests, SWE-Bench) and math reasoning (AIME, HMMT) benchmarks,

V_1

-Infer improves Pass@1 by up to

10%

over pointwise verification and outperforms recent test-time scaling methods while being significantly more efficient. Furthermore,

V_1

-PairRL achieves

7

9%

test-time scaling gains over standard RL and pointwise joint training, and improves base Pass@1 by up to 8.7% over standard RL in a code-generation setting.

Harman Singh, Xiuyu Li, Kusha Sareen +14

UC Berkeley · Mila · Together AI +1

Language Model Self-Verification Test-Time Scaling RL for Language Model Reasoning RL for Code Generation

Ask PDF

Jul 30, 2026

SVR: Self-Verifying Refinement via Joint Verdict-Confidence Reinforcement Learning for Adaptive Test-Time Compute

Scaling test-time computation can improve language-model reasoning, but uniform budgets waste computation on easy inputs, while verifier-guided refinement relies on external feedback. We introduce Self-Verifying Refinement (SVR), an oracle-free multi-turn reinforcement learning framework that learns to use self-verification as a compute-control policy. At each turn, the model produces a solution together with a discrete correctness verdict and a confidence score; it retains the current answer only when the verdict is Correct and confidence exceeds a threshold, and otherwise continues refinement using its own self-verification. Ground-truth correctness is used only to construct training rewards and is never exposed to the policy through refinement prompts or required at inference. SVR is trained with GRPO on fixed-horizon trajectories using rewards that promote solution correctness, calibration-aware self-verification, and stop-ready correct states; adaptive stopping is activated only at inference. On seven mathematical reasoning benchmarks with Qwen3.5-2B, SVR achieves a macro-average accuracy of 0.563 with only 2.99 inference turns on average. In the evaluated complete-system comparison, it exceeds standard GRPO, strong multi-turn baselines, and a fixed-budget oracle-guided score-feedback reference while requiring substantially fewer turns than fixed ten-turn inference. These results demonstrate that learned self-verification can serve as an effective internal control signal for answer retention and adaptive test-time compute allocation.

Hongyu Chen, Liang Lin, Guangrun Wang

1Sun Yat-sen University · 2Guangdong Key Laboratory of Big Data Analysis and Processing · 3X-Era AI Lab

Language Model Self-Verification RL for Language Model Reasoning Language Model Self-Assessment Inference-Time Adaptation of Language Models

Ask PDF

May 30, 2026

FineVerify: Scaling Test-Time Compute with Fine-Grained Self-Verification for Agentic Search

Agentic search requires language model agents to explore many sources and answer complex information-seeking questions. Scaling test-time compute is a promising way to improve these agents, but current approaches can fail, because correct answers are often sparse and score-based selection depends on model calibration. We propose FineVerify, a fine-grained self-verification framework that decomposes each question into checkable sub-questions, verifies sampled candidates against each sub-question, and selects the candidate with the highest aggregated score. This per-check structure turns selection into simpler local judgments and produces scores under the same explicit criteria. Across four agentic search benchmarks and two models, FineVerify consistently outperforms standard scaling baselines. With only four sampled trajectories, it improves GPT-5-mini by 8.2 accuracy points and Gemini-3-flash by 5.6% on average. With 12 samples, FineVerify enables GPT-5-mini to surpass frontier GPT-5 on BrowseComp-Plus. Beyond accuracy, FineVerify produces interpretable verification traces that help audit benchmark errors, suggesting broader applications for inspecting agentic search systems. Code and data are available at https://github.com/XuZhao0/fineverify

James Xu Zhao, Hui Chen, Bryan Hooi +1

National University of Singapore

LLM Answer Verification Agentic Search Test-Time Scaling LLM Agent Verification AI Agent Benchmarks

Ask PDF

Abstract

Language Model Self-Assessment

Explore similar work

V1V_1V1​: Unifying Generation and Self-Verification for Parallel Reasoners

SVR: Self-Verifying Refinement via Joint Verdict-Confidence Reinforcement Learning for Adaptive Test-Time Compute

FineVerify: Scaling Test-Time Compute with Fine-Grained Self-Verification for Agentic Search

$V_1$ : Unifying Generation and Self-Verification for Parallel Reasoners