Test-Time Compute Scales Better with Parallel Agent Teams

Junlin Wang; Haozhe Chen; Ang Li; Rahul Chalamala; Zhuang Liu

cs.AINov 6, 2026

Test-Time Compute Scales Better with Parallel Agent Teams

Authors: Junlin Wang, Haozhe Chen, Ang Li, Rahul Chalamala, Zhuang Liu

Organizations: Duke University · Modal Labs · Princeton University · New York University

Abstract

Test-time scaling improves the performance of a fixed model by spending more computation at inference. There are two common ways to spend it on agents when a verifier can score every attempt. Best-of-N sampling runs many independent agents and keeps the best result but saturates quickly. One large team of collaborating agents can score higher but stops improving as it grows. Can we get the best of both? Our method, parallel agent teams, does so by sampling many small teams in parallel and keeping the best team's result. We test it on 12 long-horizon tasks from GPU kernels and formal proofs to exploit development. With a 27B model and a budget of 100 agents, parallel teams of ten beat parallel agents on 10 of 12 tasks and beat one team of 100 on all four tasks where we ran such large teams. At equal dollar cost the same parallel teams beat a frontier model on half the tasks. A simple statistical model shows that individual agents lose because their many samples have a low mean and that one large team loses because a single sample gets no gain from variance. Sampling many small teams in parallel balances a high mean with the gain from variance. Parallel agent teams thus offer a more effective way to scale test-time compute.

Figures & tables

Figure 1: Sample small teams, not agents or one big team. (a) Three ways to spend the same agents. A verifier keeps the best solution. (b) Qwen3.8-27B on four tasks. Parallel agents (blue) flatten quickly, one big team (green) rises and falls with its size, and parallel teams of ten (orange) keep improving. Brackets give the gain of parallel teams over each baseline at the largest budget both reach.

Figure 2: Parallel teams beat parallel agents on 10 of 12 tasks at 100 agents. Qwen3.8-27B parallel agents (blue) and parallel teams of ten (orange). Brackets give the teams’ gain at the largest budget both reach.

Task	Benchmark	Domain	Metric
Dabic Gravity Inversion	EdgeBench	Geophysics	points 0–100
Borden Source Inversion	EdgeBench	Hydrogeology	points 0–100
Graph Node Classification	EdgeBench	Machine learning	points 0–100
Ad Placement Optimization	EdgeBench	Heuristic optimization	points 0–100
Carleson Formalization	EdgeBench	Math formalization	points 0–100
MoE Load Balance	MLS-Bench	ML systems	reward 0–1

Table 1: The 12 tasks. Higher is better for every metric.

Figure 3: Averaged over 12 tasks, parallel teams score higher and keep improving for longer. Qwen3.8-27B parallel agents (blue) and parallel teams of ten (orange). (a) Expected best score normalized by that of ten agents. The gap widens with the budget. (b) Parallel teams gain more from each added agent.

Figure 4: At a fixed budget, the best team size lies in between. 100 Qwen3.8-27B agents form

100/k

teams of size

k

. The ends are parallel agents (

k=1

; blue) and one big team (

k=100

; green).

Figure 5: At equal cost, parallel teams of a 27B model overtake a frontier model. Qwen3.8-27B parallel agents (blue) and parallel teams of ten (orange), priced by GPU time, against GPT-5.6-sol agents (green), priced by tokens. The dotted line marks the price of one GPT-5.6-sol agent.

Figure 6: How the mean, standard deviation and cost of a sample shape the best-of-

N

curve. Expected best score in the statistical model for four sampling methods. A higher mean shifts the curve up, a larger standard deviation makes it rise faster, and a higher cost delays its start.

Figure 7: The statistical model predicts the measured curves when scores are close to normal. Left of each task: score distributions of one Qwen3.8-27B agent (blue) and one team of ten (orange) with normal fits. Right: predicted (solid) and measured (dotted) expected best score.

	One agent			One team of ten			Gain at 100 agents
Task	$\mu$	$\sigma$	$W$	$\mu$	$\sigma$	$W$	predicted	measured
Dabic Gravity Inversion	9.5	8.6	0.79	22.3	5.4	0.73	-1.4%	+4.6%
Borden Source Inversion	49.1	18.9	0.96	76.5	19.5	0.93	+10.5%	+22.1%
Graph Node Classification	52.1	11.8	0.89	79.0	8.5	0.96	+12.7%	+17.0%
Ad Placement Optimization	62.0	17.9	0.66	74.3	1.8	0.99	-28.0%	+0.7%
Carleson Formalization	3.5	4.5	0.78	23.2	15.7	0.93	+222.7%	+187.7%

Table 2: The statistical model on all 12 tasks. Mean

\mu

and standard deviation

\sigma

of one Qwen3.8-27B agent and one team of ten.

W

is the Shapiro–Wilk statistic ( Shapiro and Wilk, 1965 ) , which equals 1 for a normal sample. Gain is the expected best of 10 teams over that of 100 agents. Italic marks a predicted gain with the wrong sign.

Panel of Figure 8 — Figure 8: Two agents for 6 hours beat one agent for 12 hours. Qwen3.8-27B on Graph Node Classification. (a) Expected best score against agent-hours. (b) A longer session barely shifts the score distribution, while a team of two at the same cost shifts it to the right.

Figure 9: With DeepSeek V4.1 Flash, teams gain less on tasks that are easy for the model. Parallel agents (blue) and parallel teams of ten (orange) on four of the eight tasks in Figure 12 . Brackets give the teams’ gain at the largest budget.

Figure 10: A manager makes teams worse. Qwen3.8-27B parallel agents (blue), parallel flat teams of ten (orange) and parallel orchestrated teams of ten (purple), in which one agent assigns the work. Brackets give the difference from flat teams. We omit Dabic Gravity Inversion because its evaluator often crashed.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

$N$	2	10	20	100	200
Exact $a_{N}$	0.564	1.539	1.867	2.508	2.746
$\Phi^{-1}\big((N-0.375)/(N+0.25)\big)$	0.589	1.547	1.868	2.499	2.735
$\sqrt{2\ln N}$	1.177	2.146	2.448	3.035	3.255

Appendix

Table 3: The expected maximum

a_{N}

N

samples from the standard normal distribution: exact values, the approximation of Blom (1958) , and the leading-order approximation used in Section 4 . These values apply to any team size

k

, with

N

denoting the number of independent samples.

Figure 11: One team of

k

against the best of

k

independent agents. The mean over team runs (orange line) and its range from minimum to maximum (band) against the expected best of

k

independent agents (blue), for team sizes

k\leq 10

, on the eight tasks with team-size sweeps. Both arms spend exactly

k

agents, so the budget grows along the axis; Figure 4 holds it fixed instead. Qwen3.8-27B, 12 h per agent.

	10 agents		50 agents		100 agents
Task	best-of-10	one team	best-of-50	5 teams	best-of-100	10 teams
Dabic Gravity Inversion	21.0	22.3	28.7	30.5	33.1	34.6
Borden Source Inversion	79.9	76.5	87.9	98.2	89.3	109.0
Graph Node Classification	69.1	79.0	76.8	90.0	81.5	95.1
Ad Placement Optimization	74.5	74.3	76.6	76.7	77.3	77.8
Carleson Formalization	12.2	23.2	17.7	45.8	19.0	54.8

Appendix

Table 4: One team of ten against the expected best of ten independent agents, and the expected best of five and ten teams of ten against the expected best of 50 and 100 independent agents, on all 12 tasks (Qwen3.8-27B, 12 h per agent). Team size is

k=10

throughout. The winner of each pair is bold.

Figure 12: DeepSeek V4.1 Flash: sampling teams against sampling agents. Expected best score of parallel agents (blue) and parallel teams (team size

k=10

; orange) on the eight tasks run with DeepSeek V4.1 Flash, as in Figure 2 ; the bracket gives the teams’ gap at the largest budget both reach. About 100 solo runs and ten team runs per task. 12 h per agent (6 h on Browser Exploit).

Figure 13: How the conventions come into being: the share of a run’s posts carrying each marker by three-hour window, averaged over runs, for every Qwen3.8-27B forum run (left; 655–689 runs post in each window) and for the Borden Source Inversion runs alone (right; 96–101). Team sizes range from 2 to 32. The reservation protocol is a Borden Source Inversion phenomenon (a convention in 86% of Borden Source Inversion runs, 40% of Kimi-Linear Decode runs and at most 20% elsewhere), and so, largely, is banking a teammate’s solution; thanks rise on every task.

Figure 14: Forum posts have their own register. Four surface statistics of agents’ posts in forums of team size

k=10

by three-hour window of the run (solid; each agent’s posts in the window are pooled, then averaged over agents: 950–1,000 Qwen3.8-27B agents from 110 runs and 270–320 DeepSeek V4.1 Flash agents from 39 runs per window), against the prose that solo agents of the same model write in their own transcripts (dashed; 1,144 and 500 solo runs). Tags are words of three or more capital letters.

Figure 15: The ten most frequent ALLCAPS tags in forum posts. Qwen3.8-27B team sizes range from 2 to 32; DeepSeek V4.1 Flash team size is 10. Hollow bars are tags the forum note itself writes in capitals ( win , new , fail ); the rest are the agents’ own. Every run converges on tags: the first appears a median 0.4 h (Qwen3.8-27B) and 0.2 h (DeepSeek) into the run, and a majority of agents tag in 99% and 100% of runs.

Figure 16: Stance toward peers in the Borden Source Inversion and Graph Node Classification forum runs (Qwen3.8-27B: 106,465 posts in 203 runs; DeepSeek V4.1 Flash: 2,293 posts in 20 runs). Qwen3.8-27B team sizes range from 2 to 32; DeepSeek team size is 10. Agreement and rejection are directed: the post names the peer or cites the post.

Figure 17: Qwen3.8-27B team best score over the Borden Source Inversion forum run f with team size

k=5

(judge log; the curve starts at 7.0 and enters the frame after the first hour). Light shading: the hours in which agent_2 posts coordination (1.6–6.7 h); dark: the interval in which agent_4 was told to hold. Numbered dots mark the behaviours listed below at the hour of the quoted post. The team best is 60.3 when agent_2 takes charge, 67.9 at its last post and 73.05 at the end.

Figure 18: Equal dollars on the other six tasks: Qwen3.8-27B teams beat GPT-5.6-sol on Graph Node Classification and Active Learning. As Figure 5 : expected best score against total spend, for Qwen3.8-27B parallel agents (blue) and parallel teams (team size

k=10

; orange), priced as a flat share of the GPU that served them,

18 per agent-run (

22 on KDA Chunk Forward), and GPT-5.6-sol parallel agents (green squares), priced by tokens. The dotted line marks the price of one GPT-5.6-sol agent.

Test-Time Scaling

Test-time scaling studies how extra inference-time computation can improve model answers, decisions, and generated outputs.

See all 235 papers

Scaling Inference-Time Compute with Teams of Coding Agents
Oct 12, 2026Keenan Samway, Daniel D'souza, Omar Darwiche Domingues +3Inference-Time Scaling Inference-Time Compute Allocation
Cohere Labs · Cohere
TEX: Test-Time Scaling Testing Agents via Execution-based Cross-Validation
Oct 19, 2026Akash Gokul, Srijan Bansal, Yingbo Zhou +1Test-Time Scaling Software Engineering Agents
NVIDIA · Salesforce Research · Recursive
Scaling Discovery through Test-Time Communication
Sep 17, 2026Jongho Park, Vasilis Kontonis, Shivam Garg +2Multi-Agent Collaboration Multi-Agent Communication
UC Berkeley · Microsoft Research

Oct 12, 2026J/K move · Enter open · S save

Scaling Inference-Time Compute with Teams of Coding Agents

Keenan Samway, Daniel D'souza, Omar Darwiche Domingues, Marzieh Fadaee+2

Cohere Labs · Cohere

Inference-Time Scaling Inference-Time Compute Allocation Multi-Agent Coding Multi-Agent Resource Allocation Software Engineering Agents Cost-Aware Inference

Open paper AskPDF arXiv

Multi-agent systems promise a new axis for scaling inference: rather than having one agent think longer or sample more attempts, a task is split across many agents that work in parallel and coordinate. However, in practice, more agents can easily produce duplicate work, coordination failures, and higher costs. We therefore formulate team design as a compute allocation problem: starting from a solo agent, where should additional inference compute go to reach the reward-cost Pareto frontier? We build a minimal multi-agent harness from the ground up and trace this frontier by varying, one at a time, the workflow stage at which a lead delegates to its workers (planning, implementation, or verification), the models that form the team, and the number of workers. On ProgramBench, teams outperform solo agents but cost three to seven times as much. With the right compute allocation, however, a heterogeneous 17-agent team comes within 1.4 points of a strong homogeneous 5-agent team at 70% lower cost. This efficiency stems from three findings. First, planning is the most valuable use of workers: in a homogeneous team whose lead decides how many workers to spawn, a lead that delegates only planning to its workers, and then implements and verifies the code itself, matches the reward of a lead that delegates to workers throughout planning, implementation, and verification, at half the cost. Second, cheap models pay off as planners but not as implementers: a strong lead with cheap planners recovers 47.5% of the reward gain of an all-strong team for only 13.9% of its additional cost, whereas cheap implementers lower reward. Third, planners keep improving reward as the team grows, whereas implementers and verifiers plateau beyond four workers. Together, our results offer a guide to spending an agent budget where it counts, given the task, cost, and latency constraints a user has.

Abstract

Test-Time Scaling

Explore similar work

Scaling Inference-Time Compute with Teams of Coding Agents

TEX: Test-Time Scaling Testing Agents via Execution-based Cross-Validation

Scaling Discovery through Test-Time Communication