Test-time scaling for complex reasoning tasks shows that leveraging inference-time compute, by methods such as independently sampling and aggregating multiple solutions, results in significantly better task outcomes. However, a critical bottleneck is verification: sampling is only effective if correct solutions can be reliably identified among candidates. While existing approaches typically evaluate candidates independently via scalar scoring, we demonstrate that models are substantially stronger at pairwise self-verification. Leveraging this insight, we introduce V1, a framework that unifies generation and verification through efficient pairwise ranking. V1 comprises two components: V1-Infer, an uncertainty-guided algorithm using a tournament-based ranking that dynamically allocates self-verification compute to candidate pairs whose relative correctness is most uncertain; and V1-PairRL, an RL framework that jointly trains a single model as both generator and pairwise self-verifier, ensuring the verifier adapts to the generator's evolving distribution. On code generation (LiveCodeBench, CodeContests, SWE-Bench) and math reasoning (AIME, HMMT) benchmarks, V1-Infer improves Pass@1 by up to 10 over pointwise verification and outperforms recent test-time scaling methods while being significantly more efficient. Furthermore, V1-PairRL achieves 7--9 test-time scaling gains over standard RL and pointwise joint training, and improves base Pass@1 by up to 8.7% over standard RL in a code-generation setting.
Figures & tables
Figure 1: (Left) Pairwise self-verification (using V1 -Infer, § 4 ) outperforms pointwise self-verification in self-verification measured on problems which have both correct and incorrect solutions in their parallel generations (Results with GPT-OSS-20B on LiveCodeBench-V6 prompts). (Middle and Right) Recursive self-aggregation on LiveCodeBench benchmarks shows declining Pass@N (diversity collapse) for both GPT-OSS-20B and Qwen3-4B-Instruct. See § 3 for more details.
Figure 2: Swiss Refinement Overview. Increasing pairwise verifications enables LLMs to better self-verify for selecting the best response among N self-generated solutions. See Section 4
Figure 3: Performance after self-verification using V1 -Infer compared with pointwise self-verification across benchmarks and models at N=16 base generations. Results presented for GPT-OSS-20B and Qwen3-4B-Instruct-2507. Results for GPT-OSS-120B and Qwen3-4B-Thinking-2507 are in Fig. 12 and show similar trends.
Figure 4: Accuracy vs. total budget (generation + verification calls). Stars, circles, and squares denote N=8 , N=16 , and N=32 base generations from the LLM, while the total budget =N+V where V is the number of verification calls. V1 -Infer consistently outperforms pointwise self-verification at equivalent budgets and shows monotonic performance scaling with compute. See Fig. 15 for per-benchmark results.
Figure 5: Comparison with Recursive Self-Aggregation (RSA) [ 44 ] on LCB-v6. V1 -Infer achieves higher accuracy with fewer model calls.
Figure 6: Pairwise comparison with vanilla solution generation and pointwise self-verification on SWE-bench Lite (300 instances, N=8 candidates, Gemini 2.5 Flash, mini-swe-agent [ 51 ] pipeline).
Figure 7: Accuracy improvement with increasing verification budget across problem difficulty levels (GPT-OSS-20B on LCB-v6, N=16). V1 -Infer provides the largest gains on hard problems (+23.7%).
Figure 8: Combining self-verification with RSA on AIME 2025.
Figure 9: V1 -PairRL: Unified RL training for co-evolving generation and pairwise verification. A single LLM is trained with two objectives: JGen optimizes solution generation using correctness rewards, while JPairVerif optimizes pairwise verification accuracy. The generator produces G solutions per problem, which are evaluated for correctness and paired for verification training. The verifier evaluates solutions in a paired setting by providing correctness scores for both solutions. Since we are in a verifiable setting and know from ground truth (such as test case execution during code generation) which of the generator solutions are correct, we can calculate correctness rewards for the verifier as well. See Section 5 for more details on how optimization proceeds and rewards are calculated in this framework.
Figure 10: V1 -PairRL training results (N=16). (left) Test-time scaling: V1 -PairRL outperforms V1 -PointRL and improves with increased verification budget. (middle) With V1 -Infer as the test-time scaling algorithm (at 2x budget): V1 -PairRL outperforms RL baseline even when both use pairwise verification. (right) Base Pass@1 accuracy without any test-time scaling: Co-training with pairwise verification improves generation quality over the RL baseline, even without test time scaling.
Figure 11: Co-evolving vs. non-co-evolving pairwise verification. Both use V1 -Infer (2x budget).
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Qwen3-4B-Inst
Learning rate
1×10−6
Batch size
64
Rollouts per prompt ( G )
4 (8 for baseline RL)
Judge rollouts
4 (0 for baseline RL)
Max prompt length
10240
Max response length
24576
Appendix
Table 1: Training hyperparameters for V1 .
Figure 12: Self-verification bar plots across benchmarks and models (LiveCodeBench-v5, LiveCodeBench-v6, and CodeContests). Each plot compares Pass@1, pointwise verification, and pairwise verification (budgets 1 × /2 × /3 × ) at N∈{8,16} . For Qwen3-4B-Thinking (bottom row), we run experiments only on LiveCodeBench at N=8 to manage compute.
Figure 13: Comparing Pairwise Verification with RSA. Top row: LiveCodeBench-v6. Bottom row: LiveCodeBench-v5. Columns (left → right): gpt-oss-20b, Qwen3-4B-Instruct-2507. Here number of generations by the base model is 16 , followed by verification (pointwise, pairwise) or aggregation (RSA).
Figure 14: Comparing Pairwise Verification with RSA. Top row: LiveCodeBench-v6. Bottom row: LiveCodeBench-v5. Columns (left → right): gpt-oss-20b, Qwen3-4B-Instruct-2507. Here number of generations by the base model is 8 , followed by verification (pointwise, pairwise) or aggregation (RSA).
Figure 15: Budget vs accuracy per benchmark. Accuracy vs. total budget (generation + verification calls) for CodeContests (top row), LiveCodeBench-v5 (middle row), and LiveCodeBench-v6 (bottom row). Left column: GPT-OSS-20B. Right column: Qwen3-4B-Inst. Stars denote N=8 base generations, circles denote N=16 , squares denote N=32 (pointwise only).
Figure 16: V1 -PairRL vs RL baseline across benchmarks (N=16). Left: LiveCodeBench-v5. Middle: LiveCodeBench-v6. Right: CodeContests. Co-training with pairwise verification consistently improves Pass@1 accuracy across all benchmarks.
Parallel reasoning, where a generator samples many candidate solutions and an aggregator selects the best, is one of the most effective forms of test-time scaling in large language models, and pairwise self-verification has become its strongest aggregation primitive. Yet pairwise verification carries a heavy cost: each judgment reads two complete solutions in full, and existing methods perform tens of such judgments per problem regardless of whether the comparison is informative. We introduce CAPS (Cascaded Adaptive Pairwise Selection), an inference-only framework that allocates verifier compute non-uniformly along two orthogonal axes: an evidence axis that adapts how much of each candidate the judge sees, and a distribution axis that adapts how comparisons are spread across the pool. CAPS instantiates these into a four-stage cascade with an optional rescue subroutine, and admits a closed-form verifier-token cost in which the per-candidate marginal cost is roughly halved relative to uniform full-evidence schedules. On four self-verifying models (Qwen3-14B, GPT-OSS-20B, Qwen3-4B-Instruct/Thinking) and five reasoning benchmarks spanning code (LiveCodeBench-v5/v6, CodeContests) and math (AIME 2025, HMMT 2025), CAPS outperforms the leading pairwise verifier on 14 of 20 suites while using 25.4% of its verifier-token budget on code, and outperforms pointwise self-verification on all 20. The trade-off suites admit an interpretable diagnostic in terms of the verifier's accuracy at partial versus full evidence, providing a concrete pre-deployment check for cascade suitability.
Fangzhou Lin, Shuo Xing, Peiran Li +6
Texas A&M University · Georgia Institute of Technology · Tohoku University +1
Self-improvement at scale has been a longstanding goal for reasoning models, and there are two natural places to do it: at test time, through verification-refinement (V-R) loops; and at training time, through self-training methods. Both are gated by the same bottleneck: the verifier. V-R loops stall when verifier scores inflate while accuracy stagnates, and when feedback is too generic to act on; self-training fails similarly when bad self-generated data are added to training. Better verification would unlock both, but the capability we want to train, i.e., catching self-generated errors, lacks training signal. To address this challenge, we propose self-trained verification (STV). Our key observation is that, while a model cannot catch these errors alone, it can when shown the reference solution. We turn this asymmetry into a supervision target and train the verifier to imitate a more informed version of itself. At test time, STV substantially improves V-R loops on hard problems, while alternatives (e.g., SFT, RL on verifier scores, and even meta-verifiers) do not. STV roughly doubles accuracy on hard math and lifts it 14x on scientific reasoning tasks (1.5% to 21%). At training time, we additionally train the generator using RL with STV verifier's feedback inside the V-R loop - a procedure we call verifier-in-the-loop training (ViL). Starting from an RL-converged generator, ViL yields a further 33% gain in pass@1. More notably, the generator's standalone pass@1, with no verifier at test time, climbs 30% relative past where standard RL had converged. Hence, the next frontier in reasoning on hard problems may lie in how we train for and with verification. Website: https://ar-forum.github.io/stv-webpage
Scaling test-time computation can improve language-model reasoning, but uniform budgets waste computation on easy inputs, while verifier-guided refinement relies on external feedback. We introduce Self-Verifying Refinement (SVR), an oracle-free multi-turn reinforcement learning framework that learns to use self-verification as a compute-control policy. At each turn, the model produces a solution together with a discrete correctness verdict and a confidence score; it retains the current answer only when the verdict is Correct and confidence exceeds a threshold, and otherwise continues refinement using its own self-verification. Ground-truth correctness is used only to construct training rewards and is never exposed to the policy through refinement prompts or required at inference. SVR is trained with GRPO on fixed-horizon trajectories using rewards that promote solution correctness, calibration-aware self-verification, and stop-ready correct states; adaptive stopping is activated only at inference. On seven mathematical reasoning benchmarks with Qwen3.5-2B, SVR achieves a macro-average accuracy of 0.563 with only 2.99 inference turns on average. In the evaluated complete-system comparison, it exceeds standard GRPO, strong multi-turn baselines, and a fixed-budget oracle-guided score-feedback reference while requiring substantially fewer turns than fixed ten-turn inference. These results demonstrate that learned self-verification can serve as an effective internal control signal for answer retention and adaptive test-time compute allocation.
Hongyu Chen, Liang Lin, Guangrun Wang
1Sun Yat-sen University · 2Guangdong Key Laboratory of Big Data Analysis and Processing · 3X-Era AI Lab