Parallel Test-Time Scaling with Multi-Sequence Verifiers

Yegon Kim; Seungyoo Lee; Chaeyun Jang; Hyungi Lee; Juho Lee

cs.CRMar 3, 2026

Parallel Test-Time Scaling with Multi-Sequence Verifiers

Authors: Yegon Kim, Seungyoo Lee, Chaeyun Jang, Hyungi Lee, Juho Lee

Organizations: Graduate School of AI, KAIST, Seoul, South Korea

Abstract

Parallel test-time scaling, which generates multiple candidate solutions for a single problem, is a powerful technique for improving large language model performance. However, it is hindered by two key bottlenecks: accurately selecting the correct solution from the candidate pool, and the high inference latency from generating many full solutions. We argue that both challenges are fundamentally linked to verifier calibration, as a well-calibrated verifier improves answer selection and enables early-stopping strategies to reduce latency. However, existing non-generative verifiers are limited as they score each candidate in isolation, overlooking rich contextual information across the set of candidates. To address this, we introduce the Multi-Sequence Verifier (MSV), a lightweight verifier that predicts each candidate's correctness conditioned on the full sampled set. MSV achieves improved calibration, which directly enhances best-of-N selection performance and empowers a novel early-stopping framework. Across challenging mathematical reasoning benchmarks, MSV improves best-of-64 accuracy by up to 6% relative to strong baselines, and in the early-stopping setting reaches the same accuracy as baselines with less than half the latency.

Figures & tables

Figure 1: Illustration of our Multi-Sequence Verifier (MSV) that uses multiple attention masks in its Multi-Mask Transformer Block (MMTB) to predict the correctness of each answer. The different attention masks allow MSV to flexibly leverage information both across and within sequences.

Figure 2: Best-of-N accuracy vs.

N

, in Terminal Answers setting. Full results are in Table 13 .

Figure 5: Accuracy-token tradeoff curves in Streaming Answers , with

N=64

. Curves that lie higher represent superior tradeoff. Accuracy-latency tradeoff curves are in Figure 10 .

Figure 8: Accuracy-latency tradeoff with

N=64

Table 7

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Setting (ours)
Optimizer	AdamW
LR schedule	constant-with-warmup (HF defaults)
LR sweep	$\{1\text{e}{-5},\,5\text{e}{-5},\,1\text{e}{-4},\,5\text{e}{-4},\,1\text{e}{-3},\,5\text{e}{-3}\}$ (CV on validation)
Final LRs	Probe: $1\text{e}{-3}$ ; Others: $5\text{e}{-5}$ ; $w$ : $1\text{e}{-1}$ ; $e^{(n)}$ : $1\text{e}{-3}$
Probe arch.	2-layer MLP, hidden size 1024
MSV depth	1 Multi-Mask Transformer Block

Appendix

Table 3: Summary of implementation and hyperparameters.

Table 4: Average latency breakdown and end-to-end latency, in seconds.

Metric	MSV 16	w/o full	w/o equiv.	w/o within-seq	w/o within-ans
Brier ↓	0.038	0.040	0.045	0.048	0.050
AUROC ↑	0.953	0.953	0.922	0.945	0.949
NLL ↓	0.150	0.157	0.236	0.192	0.169

Appendix

Table 5: Mask ablation results on AIME with MSV 16 in the Streaming Answers setting.

Method	AUROC $\uparrow$	Brier $\downarrow$	NLL $\downarrow$
MSV 1	0.906 ±0.008	0.064 ±0.003	0.241 ±0.011
MSV 16	0.953 ±0.007	0.038 ±0.003	0.150 ±0.011
Ablation	0.936 ±0.0016	0.047 ±0.009	0.190 ±0.0333

Appendix

Table 6: Ablation of MSV architectural choices in the Streaming Answers setting. We report calibration metrics on the AIME dataset.

Panel of Figure 9 — Figure 9: Accuracy-latency tradeoff curves in Streaming Answers on AIME. We use the grouping strategy with MSV 64 for large $N$ .

Delimiter	Probe	MSV 1	MSV 16
“Wait”	1024.2	954.8	1078.3
“Alternatively”	985.9	980.0	1012.4
“But”	1039.3	1023.6	1070.7
Every 200 tokens	910.9	—	1013.9

Appendix

Table 7: Area Under the Tradeoff Curve with different delimiters on the AIME dataset with

N=16

parallel sequences. “Every 200 tokens” is a delimiter-free baseline that extracts answers at fixed token intervals.

Dataset	GenSelect	MSV 16
MATH	0.724	0.747
OlympiadBench	0.500	0.515
AMC12	0.455	0.540
AIME	0.356	0.462
Omni-MATH	0.296	0.320

Appendix

Table 8: Best-of-16 accuracy comparison between GenSelect and MSV 16 on five mathematical reasoning benchmarks in the Terminal Answers setting.

# of problems	Brier score
56	0.0833
112	0.0821
224	0.0809
448	0.0812

Appendix

Table 9: Brier score of Probe on the AIME dataset as a function of training set size

Method	$N=1$	$N=4$	$N=16$	$N=64$
MSV N	0.0819	0.0687	0.0572	0.0387
MSV N (Mean)	0.0756	0.0641	0.0544	0.0382

Appendix

Table 10: Brier score comparison between MSV N using the last token versus the mean of answer tokens on the AIME dataset in the Terminal Answers setting.

	MATH		OlympiadBench		AMC12		AIME		Omni - MATH
Method	AUROC $\uparrow$	Brier $\downarrow$	AUROC $\uparrow$	Brier $\downarrow$	AUROC $\uparrow$	Brier $\downarrow$	AUROC $\uparrow$	Brier $\downarrow$	AUROC $\uparrow$	Brier $\downarrow$
Token Probs	0.5320 ±0.0000	0.2504 ±0.0000	0.7704 ±0.0000	0.3696 ±0.0000	0.7350 ±0.0000	0.3540 ±0.0000	0.8999 ±0.0000	0.3073 ±0.0000	0.7814 ±0.0000	0.4467 ±0.0000
Probe	0.8790 ±0.0052	0.1346 ±0.0107	0.9022 ±0.0038	0.1271 ±0.0144	0.8994 ±0.0062	0.1438 ±0.0214	0.9461 ±0.0034	0.0859 ±0.0147	0.8836 ±0.0008	0.1340 ±0.0252
MSV 1	0.8566 ±0.0166	0.1460 ±0.0063	0.8926 ±0.0048	0.1310 ±0.0061	0.8890 ±0.0080	0.1498 ±0.0099	0.9469 ±0.0025	0.0819 ±0.0066	0.8711 ±0.0062	0.1460 ±0.0103
PRM	0.8156 ±0.0000	0.1747 ±0.0000	0.8958 ±0.0000	0.1689 ±0.0000	0.8521 ±0.0000	0.1807 ±0.0000	0.9374 ±0.0000	0.1472 ±0.0000	0.8826 ±0.0000	0.1584 ±0.0000
Token Probs + WV 4	0.7922 ±0.0000	0.1678 ±0.0000	0.8392 ±0.0000	0.2166 ±0.0000	0.7978 ±0.0000	0.2448 ±0.0000	0.8223 ±0.0000	0.2239 ±0.0000	0.8085 ±0.0000	0.2620 ±0.0000

Appendix

Table 11: AUROC (

\uparrow

) and Brier score (

\downarrow

) of verifiers on terminal answers.

	MATH		OlympiadBench		AMC12		AIME		Omni - MATH
Method	ECE $\downarrow$	NLL $\downarrow$	ECE $\downarrow$	NLL $\downarrow$	ECE $\downarrow$	NLL $\downarrow$	ECE $\downarrow$	NLL $\downarrow$	ECE $\downarrow$	NLL $\downarrow$
Token Probs	0.1855 ±0.0000	0.9190 ±0.0000	0.4124 ±0.0000	1.1444 ±0.0000	0.3935 ±0.0000	1.1384 ±0.0000	0.4198 ±0.0000	0.8669 ±0.0000	0.5389 ±0.0000	1.2505 ±0.0000
Probe	0.0726 ±0.0401	0.4391 ±0.0485	0.0700 ±0.0517	0.4172 ±0.0437	0.1064 ±0.0581	0.4680 ±0.0730	0.1024 ±0.0483	0.3115 ±0.0488	0.1206 ±0.0589	0.4394 ±0.0817
MSV 1	0.0849 ±0.0190	0.4787 ±0.0291	0.0806 ±0.0232	0.4363 ±0.0213	0.1279 ±0.0304	0.4913 ±0.0364	0.1043 ±0.0250	0.2993 ±0.0221	0.1446 ±0.0288	0.4881 ±0.0345
PRM	0.1817 ±0.0000	0.5349 ±0.0000	0.1834 ±0.0000	0.5231 ±0.0000	0.1474 ±0.0000	0.5484 ±0.0000	0.2468 ±0.0000	0.4773 ±0.0000	0.2160 ±0.0000	0.5010 ±0.0000
Token Probs + WV 4	0.1615 ±0.0000	1.1805 ±0.0000	0.2416 ±0.0000	1.2018 ±0.0000	0.2616 ±0.0000	1.4551 ±0.0000	0.2677 ±0.0000	1.1803 ±0.0000	0.3306 ±0.0000	1.4633 ±0.0000

Appendix

Table 12: ECE (

\downarrow

) and NLL (

\downarrow

) of verifiers on terminal answers.

N	Method	MATH	OlympiadBench	AMC12	AIME	Omni - MATH
$1$	—	0.6755 ±0.0000	0.4105 ±0.0000	0.3965 ±0.0000	0.2748 ±0.0000	0.2432 ±0.0000
$4$	Self-consistency 4	0.7049 ±0.0000	0.4484 ±0.0000	0.4286 ±0.0000	0.3251 ±0.0000	0.2633 ±0.0000
	Probe	0.7197 ±0.0006	0.4725 ±0.0008	0.4861 ±0.0041	0.3771 ±0.0012	0.2885 ±0.0005
	MSV 1	0.7155 ±0.0024	0.4681 ±0.0011	0.4854 ±0.0043	0.3758 ±0.0008	0.2878 ±0.0018
	PRM	0.7203 ±0.0000	0.4741 ±0.0000	0.4879 ±0.0000	0.3744 ±0.0000	0.2927 ±0.0000
	Probe + WV 4	0.7250 ±0.0009	0.4770 ±0.0003	0.4856 ±0.0030	0.3847 ±0.0010	0.2910 ±0.0009

Appendix

Table 13: Best-of-N accuracy (

\uparrow

), with verifiers at

N=1,4,16,64

		MATH		OlympiadBench		AMC12		AIME		Omni - MATH
N	Method	ECE $\downarrow$	Brier $\downarrow$	ECE $\downarrow$	Brier $\downarrow$	ECE $\downarrow$	Brier $\downarrow$	ECE $\downarrow$	Brier $\downarrow$	ECE $\downarrow$	Brier $\downarrow$
$4$	Self-consistency 4	0.1613 ±0.0000	0.3638 ±0.0000	0.2586 ±0.0000	0.4983 ±0.0000	0.2843 ±0.0000	0.5504 ±0.0000	0.2805 ±0.0000	0.5316 ±0.0000	0.3708 ±0.0000	0.6031 ±0.0000
	PRM	0.1308 ±0.0000	0.3283 ±0.0000	0.2016 ±0.0000	0.3508 ±0.0000	0.1386 ±0.0000	0.3888 ±0.0000	0.2363 ±0.0000	0.3276 ±0.0000	0.2241 ±0.0000	0.3614 ±0.0000
	PRM + WV 4	0.1607 ±0.0000	0.3645 ±0.0000	0.2554 ±0.0000	0.5029 ±0.0000	0.2687 ±0.0000	0.5582 ±0.0000	0.2785 ±0.0000	0.5357 ±0.0000	0.3724 ±0.0000	0.6174 ±0.0000
	Probe	0.1168 ±0.0448	0.3001 ±0.0341	0.1510 ±0.0689	0.3376 ±0.0653	0.1801 ±0.0639	0.3806 ±0.0666	0.1703 ±0.0579	0.2587 ±0.0594	0.2298 ±0.0777	0.3997 ±0.0995
	MSV 1	0.1316 ±0.0206	0.3316 ±0.0176	0.1675 ±0.0276	0.3624 ±0.0257	0.1985 ±0.0293	0.3927 ±0.0311	0.1661 ±0.0269	0.2459 ±0.0242	0.2514 ±0.0319	0.4372 ±0.0374

Appendix

Table 14: ECE (

\downarrow

) and Brier score (

\downarrow

) of confidence output on best-of-

N

answers, with verifiers at

N=4,16,64

	MATH		OlympiadBench		AMC12		AIME		Omni - MATH
Method	AUROC $\uparrow$	Brier $\downarrow$	AUROC $\uparrow$	Brier $\downarrow$	AUROC $\uparrow$	Brier $\downarrow$	AUROC $\uparrow$	Brier $\downarrow$	AUROC $\uparrow$	Brier $\downarrow$
Token Probs	0.7910 ±0.0000	0.3399 ±0.0000	0.8091 ±0.0000	0.4266 ±0.0000	0.7856 ±0.0000	0.3979 ±0.0000	0.8858 ±0.0000	0.3217 ±0.0000	0.7974 ±0.0000	0.4746 ±0.0000
PRM	0.8692 ±0.0000	0.1902 ±0.0000	0.8542 ±0.0000	0.1607 ±0.0000	0.8658 ±0.0000	0.1621 ±0.0000	0.9309 ±0.0000	0.1185 ±0.0000	0.8227 ±0.0000	0.1497 ±0.0000
Probe	0.8690 ±0.0135	0.1592 ±0.0101	0.8317 ±0.0089	0.1478 ±0.0232	0.8621 ±0.0108	0.1413 ±0.0195	0.9120 ±0.0141	0.0796 ±0.0105	0.8229 ±0.0142	0.1527 ±0.0242
MSV 1	0.8443 ±0.0223	0.1601 ±0.0125	0.8169 ±0.0135	0.1425 ±0.0076	0.8546 ±0.0104	0.1343 ±0.0028	0.9056 ±0.0076	0.0642 ±0.0028	0.7951 ±0.0125	0.1381 ±0.0084
Token Probs + WV 4	0.7743 ±0.0000	0.3266 ±0.0000	0.7837 ±0.0000	0.4051 ±0.0000	0.7596 ±0.0000	0.3775 ±0.0000	0.8601 ±0.0000	0.2990 ±0.0000	0.7757 ±0.0000	0.4522 ±0.0000

Appendix

Table 15: AUROC (

\uparrow

) and Brier score (

\downarrow

) of verifiers on streaming answers.

	MATH		OlympiadBench		AMC12		AIME		Omni - MATH
Method	ECE $\downarrow$	NLL $\downarrow$	ECE $\downarrow$	NLL $\downarrow$	ECE $\downarrow$	NLL $\downarrow$	ECE $\downarrow$	NLL $\downarrow$	ECE $\downarrow$	NLL $\downarrow$
Token Probs	0.3732 ±0.0000	1.1043 ±0.0000	0.5255 ±0.0000	1.1625 ±0.0000	0.4944 ±0.0000	1.1466 ±0.0000	0.4886 ±0.0000	0.8724 ±0.0000	0.6045 ±0.0000	1.2632 ±0.0000
PRM	0.2060 ±0.0000	0.5667 ±0.0000	0.1883 ±0.0000	0.5046 ±0.0000	0.1775 ±0.0000	0.5076 ±0.0000	0.2456 ±0.0000	0.4127 ±0.0000	0.2298 ±0.0000	0.4819 ±0.0000
Probe	0.0934 ±0.0416	0.5184 ±0.0528	0.1317 ±0.0382	0.5206 ±0.0914	0.1292 ±0.0461	0.4841 ±0.0689	0.1160 ±0.0385	0.2847 ±0.0421	0.1656 ±0.0470	0.5194 ±0.0945
MSV 1	0.0939 ±0.0163	0.5356 ±0.0441	0.0911 ±0.0163	0.5055 ±0.0208	0.0862 ±0.0164	0.4532 ±0.0039	0.0511 ±0.0229	0.2412 ±0.0113	0.1157 ±0.0132	0.4974 ±0.0232
Token Probs + WV 4	0.3499 ±0.0000	0.9724 ±0.0000	0.5001 ±0.0000	1.0713 ±0.0000	0.4680 ±0.0000	1.0312 ±0.0000	0.4595 ±0.0000	0.8013 ±0.0000	0.5843 ±0.0000	1.1794 ±0.0000

Appendix

Table 16: ECE (

\downarrow

) and NLL (

\downarrow

) of verifiers on streaming answers.

Figure 10: Accuracy-latency tradeoff curves in Streaming Answers , with

N=64

. Curves that lie higher are superior in that they require lower latency to achieve the same accuracy. Accuracy-token tradeoff curves are in Figure 5 .

Panel of Figure 11 — Figure 11: Accuracy-latency tradeoff curves in Streaming Answers , for $N\in\{4,16\}$ .

Base LM	Dataset	Metric	Probe	MSV 1	MSV 64
R1-8B	AIME	Brier $\downarrow$	0.0673 ±0.0104	0.0738 ±0.0151	0.0363 ±0.0090
		AUROC $\uparrow$	0.9672 ±0.0017	0.9750 ±0.0043	0.9901 ±0.0012
		ECE $\downarrow$	0.0479 ±0.0272	0.0694 ±0.0201	0.0367 ±0.0148
		NLL $\downarrow$	0.2555 ±0.0407	0.3240 ±0.0708	0.1868 ±0.0352
Qwen3-1.7B	AIME	Brier $\downarrow$	0.0848 ±0.0333	0.0770 ±0.0059	0.0509 ±0.0246
		AUROC $\uparrow$	0.9530 ±0.0174	0.9557 ±0.0053	0.9863 ±0.0011

Appendix

Table 18: Brier score, AUROC, ECE, and NLL of verifiers across different base language models. R1-8B is abbreviation for DeepSeek-R1-Distill-Llama-8B.

Base LM	Dataset	Probe	Probe+WV	MSV 1	MSV 1 +WV	MSV 64
R1-8B	AIME	0.6116 ±0.0098	0.6357 ±0.0131	0.6047 ±0.0088	0.6492 ±0.0091	0.6625 ±0.0054
Qwen3-1.7B	AIME	0.5223 ±0.0130	0.5089 ±0.0179	0.4545 ±0.0097	0.5288 ±0.0089	0.5357 ±0.0109
Llama-3.2-1B	MATH	0.2393 ±0.0143	0.3375 ±0.0051	0.1670 ±0.0201	0.2946 ±0.0192	0.3777 ±0.0051

Appendix

Table 19: Best-of-64 accuracy across different base language models. R1-8B is abbreviation for DeepSeek-R1-Distill-Llama-8B.

Explore similar work

CAPS: Cascaded Adaptive Pairwise Selection for Efficient Parallel Reasoning
May 15, 2026Fangzhou Lin, Shuo Xing, Peiran Li +6Pairwise Comparison Pairwise Preference Evaluation
Texas A&M University · Georgia Institute of Technology · Tohoku University +1
$V_1$ : Unifying Generation and Self-Verification for Parallel Reasoners
Mar 4, 2026Harman Singh, Xiuyu Li, Kusha Sareen +14Test-Time Scaling RL for Language Model Reasoning
UC Berkeley · Mila · Together AI +1
Towards Better Exploration in Sequential Test-Time Scaling
Sep 30, 2026Joseph Rance, Fabio Pizzati, Juil Sock +4Test-Time Scaling
University of Oxford · MBZUAI · BBC R&D

May 15, 2026J/K move · Enter open · S save

CAPS: Cascaded Adaptive Pairwise Selection for Efficient Parallel Reasoning

Fangzhou Lin, Shuo Xing, Peiran Li, Siyuan Yang+5

Texas A&M University · Georgia Institute of Technology · Tohoku University+1

Pairwise Comparison Pairwise Preference Evaluation Test-Time Optimization Test-Time Scaling Efficient Language Model Reasoning Efficient Language Model Inference+1

Open paper AskPDF arXiv

Parallel reasoning, where a generator samples many candidate solutions and an aggregator selects the best, is one of the most effective forms of test-time scaling in large language models, and pairwise self-verification has become its strongest aggregation primitive. Yet pairwise verification carries a heavy cost: each judgment reads two complete solutions in full, and existing methods perform tens of such judgments per problem regardless of whether the comparison is informative. We introduce CAPS (Cascaded Adaptive Pairwise Selection), an inference-only framework that allocates verifier compute non-uniformly along two orthogonal axes: an evidence axis that adapts how much of each candidate the judge sees, and a distribution axis that adapts how comparisons are spread across the pool. CAPS instantiates these into a four-stage cascade with an optional rescue subroutine, and admits a closed-form verifier-token cost in which the per-candidate marginal cost is roughly halved relative to uniform full-evidence schedules. On four self-verifying models (Qwen3-14B, GPT-OSS-20B, Qwen3-4B-Instruct/Thinking) and five reasoning benchmarks spanning code (LiveCodeBench-v5/v6, CodeContests) and math (AIME 2025, HMMT 2025), CAPS outperforms the leading pairwise verifier on 14 of 20 suites while using 25.4% of its verifier-token budget on code, and outperforms pointwise self-verification on all 20. The trade-off suites admit an interpretable diagnostic in terms of the verifier's accuracy at partial versus full evidence, providing a concrete pre-deployment check for cascade suitability.

Abstract

Explore similar work

CAPS: Cascaded Adaptive Pairwise Selection for Efficient Parallel Reasoning

V1V_1V1​: Unifying Generation and Self-Verification for Parallel Reasoners

Towards Better Exploration in Sequential Test-Time Scaling

$V_1$ : Unifying Generation and Self-Verification for Parallel Reasoners