Advancing the Pareto Frontier of Test-Time Scaling with Multi-Path Plan Aggregation

Siheng Xiong; Ali Payani; Faramarz Fekri

cs.CLOct 13, 2025

Advancing the Pareto Frontier of Test-Time Scaling with Multi-Path Plan Aggregation

Authors: Siheng Xiong, Ali Payani, Faramarz Fekri

Organizations: Georgia Institute of Technology · Cisco Research

Abstract

Test-time scaling seeks to improve reasoning through additional inference-time computation, commonly using solution sampling with majority voting to select the final answer. However, repeated sampling can reproduce correlated errors, limiting the accuracy gains from additional computation. We introduce Multi-Path Plan Aggregation (MPPA), which learns planning and answer selection using separate LoRA adapters on a shared frozen backbone. The planner jointly generates $K$ short plans, each guiding a separate solution generated by the frozen executor; the aggregator then scores the resulting solutions and selects the final answer. Joint planning encourages complementary approaches suited to the executor's capabilities, while learned selection uses reasoning evidence to identify correct candidates. Alternating optimization improves planning and aggregation through supervised finetuning and reinforcement learning. Across five mathematics and science benchmarks and three backbones, MPPA improves accuracy--token trade-offs compared with training-free and training-based baselines. With three plans, it achieves higher average accuracy than eight directly sampled solutions with majority voting or the same learned aggregator, while reducing output tokens by 47--54%. Evaluation with task-provided subtasks further demonstrates higher final-task accuracy than direct sampling with majority voting or learned aggregation at either the task or subtask level.

Figures & tables

Figure 1: Average accuracy versus generated-token cost across five benchmarks. Training-free includes maj@8, PS+ maj@8, L2M maj@8, ESC, and USC; Other training-based includes Base, SFT, and GRPO with agg@8, where agg denotes selection using the same learned aggregator. MPPA connects the one- and three-plan settings. Costs are normalized to a single Base model solve; See details in Table 1 .

Figure 2: Majority-vote accuracy and candidate coverage under repeated solution sampling with Qwen2.5-7B-Instruct on MATH500 and GPQA Diamond. Shaded regions show confidence intervals.

Figure 4: Overview of MPPA. (a) Selected answers propagate along task-provided subtask dependencies. (b) Within each subtask, the planner jointly generates plans, the frozen executor solves once per plan, and the aggregator selects the highest-scoring candidate.

Method	Mathematics			Science		Avg	Token
Method	MATH500	AIME24	AIME25	GPQA	MMLU-Pro	Avg	Token
Qwen2.5-7B-Instruct
Base	75.6	12.6	6.7	35.1	64.6	38.9	$1.00\times$
Base maj@8	81.6	14.4	13.3	34.8	69.6	42.8	$8.00\times$
PS+ maj@8	80.6	15.6	10.0	37.9	69.6	42.7	$8.39\times$
L2M maj@8	80.4	18.9	11.1	33.8	71.8	43.2	$32.42\times$

Table 1: Accuracy (%) and output tokens relative to Base. GPQA and MMLU-Pro denote GPQA Diamond and MMLU-Pro-sci500. Column-wise best accuracies within each model are bold.

Generation	Selection	Mathematics			Science		Avg
Generation	Selection	MATH500	AIME24	AIME25	GPQA	MMLU-Pro	Avg
No plan	Majority vote	77.2	13.3	7.8	35.5	67.3	40.2
No plan	Aggregator	83.6	17.8	13.3	54.0	78.3	49.4
Single plan	Aggregator	88.0	18.9	16.7	67.2	80.1	54.2
Independent $K$ plans	Aggregator	87.2	20.0	13.3	69.0	81.4	54.2
Joint $K$ plans	Majority vote	84.8	15.6	17.8	46.5	74.2	47.8

Table 2: Generation and selection ablations on Qwen2.5-7B-Instruct with the fixed number of candidates

K=3

. Single plan reuses one plan across

K

executions; independent plans use

K

separate planning calls.

Planner	Aggregator	Mathematics			Science		Avg
Planner	Aggregator	MATH500	AIME24	AIME25	GPQA	MMLU-Pro	Avg
SFT	Stage 1	80.5	14.4	10.0	39.6	70.0	42.9
SFT $\rightarrow$ RL	Stage 1	83.6	16.7	12.2	43.4	73.2	45.8
SFT $\rightarrow$ RL	Stage 1 $\rightarrow$ Stage 2	86.6	18.9	15.6	46.0	76.2	48.6
Alternating optimization ( $T=3$ )		89.2	22.2	24.4	72.2	84.0	58.4

Table 3: Training ablations on Qwen2.5-7B-Instruct. Stage 1 uses pointwise supervision; Stage 2 adds listwise training. Alternating optimization updates both adapters for

T=3

rounds.

Table 8

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Planner GRPO training curve for Qwen2.5-7B-Instruct, showing batch mean reward over three alternating optimization rounds.

Generation	Mathematics			Science		Avg
Generation	MATH500	AIME24	AIME25	GPQA	MMLU-Pro	Avg
Base	83.6 / 84.2	17.8 / 18.9	13.3 / 14.4	54.0 / 54.0	78.3 / 78.3	49.4 / 50.0
Single plan	88.0 / 89.6	18.9 / 24.4	16.7 / 18.9	67.2 / 68.2	80.1 / 80.2	54.2 / 56.3
Independent $K$ plans	87.2 / 89.3	20.0 / 23.3	13.3 / 15.6	69.0 / 70.2	81.4 / 83.5	54.2 / 56.4
Joint $K$ plans	89.2 / 92.3	22.2 / 26.7	24.4 / 24.4	72.2 / 73.2	84.0 / 86.0	58.4 / 60.5

Appendix

Table 6: Candidate coverage and selection accuracy (%) for Qwen2.5-7B-Instruct with

K=3

candidates. Each cell reports agg@K / pass@K, where agg@K uses the same learned aggregator.

Figure 9: Accuracy–output-token trade-offs for Qwen2.5-7B-Instruct on each dataset and on average. MPPA uses

K=1,3,5,8

plans; Base maj@K and Base agg@K use

K=1,3,8,16

directly sampled solutions, with agg@K using the same learned aggregator. Token usage is relative to one directly sampled solution on each dataset; the Avg panel averages accuracies and relative token usage across datasets.

Method	MATH500	AIME24	AIME25	GPQA	MMLU-Pro
Qwen2.5-7B-Instruct
Base	75.6 [71.9, 78.0]	12.6 [3.6, 20.6]	6.7 [2.1, 12.2]	35.1 [30.3, 39.5]	64.6 [61.1, 67.6]
Base maj@8	81.6 [77.1, 83.5]	14.4 [4.9, 28.9]	13.3 [3.1, 23.1]	34.8 [31.0, 42.2]	69.6 [66.0, 73.3]
Base agg@8	87.9 [85.2, 90.4]	21.1 [8.8, 35.1]	20.0 [8.1, 33.5]	69.4 [64.5, 75.5]	86.7 [83.8, 89.0]
MPPA@1	83.6 [80.7, 86.3]	16.7 [7.4, 26.7]	14.4 [5.6, 26.3]	45.1 [39.7, 50.7]	71.9 [68.5, 75.1]
MPPA@3	89.2 [86.4, 91.8]	22.2 [10.0, 36.7]	24.4 [12.2, 37.8]	72.2 [65.7, 78.3]	84.0 [80.8, 87.2]

Appendix

Table 7: Accuracy (%) with confidence intervals for the principal sampling and aggregation settings. Each cell reports the mean followed by the interval bounds; means match Table 1 . GPQA and MMLU-Pro denote GPQA Diamond and MMLU-Pro-sci500.

Test-Time Scaling

Test-time scaling studies how extra inference-time computation can improve model answers, decisions, and generated outputs.

See all 223 papers

Planned Test-Time Scaling with Coordinated Reasoning Paths
Sep 23, 2026Xueqing Wu, Langxing Bai, Hritik Bansal +5RL for Language Model Reasoning Test-Time Scaling
University of California, Los Angeles · Amazon
TMAS: Scaling Test-Time Compute via Multi-Agent Synergy
May 11, 2026George Wu, Nan Jing, Qing Yi +7Multi-Agent LLM Systems RL for Language Model Reasoning
1IQuest Research · 2Beihang University
CoBa: Cost-Effective Test-Time Scaling via Compute-Balanced Routing
Aug 7, 2026Yan Zhou, Yue Ouyang, Kaiyang Zheng +1Cost-Aware Inference Test-Time Scaling
School of Mathematics and Statistics, Changsha University of Science and Technology, Changsha, China · School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China.

Sep 23, 2026J/K move · Enter open · S save

Planned Test-Time Scaling with Coordinated Reasoning Paths

Xueqing Wu, Langxing Bai, Hritik Bansal, Po-Nien Kung+4

University of California, Los Angeles · Amazon

RL for Language Model Reasoning Test-Time Scaling LLM Reasoning

Open paper AskPDF arXiv

Test-time scaling with parallel branches is widely adopted to improve performance on challenging reasoning tasks. The predominant approach, repeated sampling, draws branches independently from a single policy, which can produce redundant attempts and thereby limit the gains from additional inference compute. To address this limitation, we propose Planned Test-Time Scaling (PTTS), which replaces independent sampling with a coordinated joint policy: a planner generates a solution outline for each branch, steering the branches toward distinct reasoning paths, and an executor produces a full solution conditioned on each outline. Formally, we show that PTTS strictly generalizes repeated sampling and, in a stylized setting, provably promotes coverage of complementary reasoning modes and yields better pass@k scaling. We instantiate PTTS on top of strong reasoning models, keeping them fixed as executors while replacing repeated sampling with PTTS inference to further enhance test-time scaling. Concretely, we develop two variants: PTTS-ZS prompts a model to jointly generate outlines for all branches in a single autoregressive pass, while PTTS-RL directly optimizes the planner against the pass@k reward using truncated execution rollouts for efficient training and a sharper reward signal. Across five mathematical reasoning benchmarks with Qwen3-1.7B and 4B, PTTS-ZS improves pass@64 over repeated sampling by up to 6.7 points, while PTTS-RL further increases the gain to up to 13.4 points. Further analysis indicates that broader coverage of distinct reasoning paths contributes to these gains. Overall, PTTS provides a general framework for improving test-time scaling by coordinating reasoning branches, with zero-shot and trainable instantiations that yield substantial performance gains.

Abstract

Test-Time Scaling

Explore similar work

Planned Test-Time Scaling with Coordinated Reasoning Paths

TMAS: Scaling Test-Time Compute via Multi-Agent Synergy

CoBa: Cost-Effective Test-Time Scaling via Compute-Balanced Routing