Scaling Inference-Time Compute with Teams of Coding Agents

Keenan Samway; Daniel D'souza; Omar Darwiche Domingues; Marzieh Fadaee; Tan Yi-Chern; Nouha Dziri

cs.MAOct 12, 2026

Scaling Inference-Time Compute with Teams of Coding Agents

Authors: Keenan Samway, Daniel D'souza, Omar Darwiche Domingues, Marzieh Fadaee, Tan Yi-Chern, Nouha Dziri

Organizations: Cohere Labs · Cohere

Abstract

Multi-agent systems promise a new axis for scaling inference: rather than having one agent think longer or sample more attempts, a task is split across many agents that work in parallel and coordinate. However, in practice, more agents can easily produce duplicate work, coordination failures, and higher costs. We therefore formulate team design as a compute allocation problem: starting from a solo agent, where should additional inference compute go to reach the reward-cost Pareto frontier? We build a minimal multi-agent harness from the ground up and trace this frontier by varying, one at a time, the workflow stage at which a lead delegates to its workers (planning, implementation, or verification), the models that form the team, and the number of workers. On ProgramBench, teams outperform solo agents but cost three to seven times as much. With the right compute allocation, however, a heterogeneous 17-agent team comes within 1.4 points of a strong homogeneous 5-agent team at 70% lower cost. This efficiency stems from three findings. First, planning is the most valuable use of workers: in a homogeneous team whose lead decides how many workers to spawn, a lead that delegates only planning to its workers, and then implements and verifies the code itself, matches the reward of a lead that delegates to workers throughout planning, implementation, and verification, at half the cost. Second, cheap models pay off as planners but not as implementers: a strong lead with cheap planners recovers 47.5% of the reward gain of an all-strong team for only 13.9% of its additional cost, whereas cheap implementers lower reward. Third, planners keep improving reward as the team grows, whereas implementers and verifiers plateau beyond four workers. Together, our results offer a guide to spending an agent budget where it counts, given the task, cost, and latency constraints a user has.

Figures & tables

Figure 1: Overview of our multi-agent harness. A lead agent delegates subtasks to workers operating in a shared sandbox. We vary team composition (a), whether the lead blocks or keeps working while workers run (b), and which stages of the task the lead is prompted to delegate (c).

Figure 2: Teams outperform solo agents, but incur higher inference and wall-clock costs. Mean ProgramBench reward versus total inference cost (left) and total recorded wall-clock time (right) for Sol solo agents and homogeneous blocking and asynchronous teams under minimal and heavy (PIV) guidance. Points are means over three runs; error bars are 95% intervals over tasks of each team’s paired difference from the solo agent with the same guidance.

Figure 5: Stage guidance shifts delegation timing, but nominal labels do not partition edit-defined phases. The figure covers 600 ProgramBench rollouts per guidance setting (200 tasks

\times

three runs) from homogeneous Sol blocking teams, with the lead’s successful delegate_agent calls marked. The four left rasters show PIV, P, I, and V guidance from top to bottom. Each row represents one rollout. Within each panel, rows are ordered by implementation start (the first detected source write; defined in Appendix B ), with the earliest at the bottom; rows are not aligned across panels. The bars on the right give the mean share of a rollout’s delegation calls in the planning (P), implementation (I), and verification (V) phases (one P-guidance rollout without delegation calls is excluded); error bars are 95% bootstrap intervals from resampling tasks. Per-guidance call counts vary by at most 3% across runs.

Figure 8: Where additional agents are deployed determines the reward–cost scaling curve. Fixed teams pair a GPT-5.6 Sol lead with 2–16 Luna workers (numbers give team size); dynamic teams are all-Sol teams sized by the lead, shown at their mean number of workers. Scaling the heterogeneous team to 16 workers with planning guidance comes within 1.4 points of the homogeneous team at 70% lower cost. All teams use asynchronous push. Fixed teams are a single run; error bars (left) are paired 95% intervals relative to Solo PIV.

Figure 9: Stronger models communicate more, but messaging does not raise reward. Homogeneous asynchronous teams under minimal and PIV guidance. Hatched segments are rejected worker calls. Right: reward change relative to the same team without the tool, with paired 95% intervals.

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Policy	How it works	Pros	Cons
Blocking delegation	The lead delegates to a worker and remains blocked until the worker returns its result.	Simple coordination; provides a natural synchronization point; allows the lead to reason from a completed result; lowers the risk of acting on incomplete information.	The lead is idle while the worker runs; prevents lead–worker parallelism; can substantially increase wall-clock time.
Async push: full message	The worker runs concurrently. Once finished, its full result is appended to the lead’s next tool result.	Requires no explicit retrieval call; makes completed work difficult to overlook; preserves concurrency; allows results to be incorporated quickly.	May interrupt the lead at an awkward point; unexpectedly consumes context; multiple messages may overload the next tool output; may destabilize the lead’s current plan.

Appendix

Table 1: Comparison of scheduling and message-delivery policies.

			Price ($ per million tokens)
Model	Provider	Reasoning setting	Input	Output	Cache read	Cache write	Evaluation
GPT-5.6 Sol	OpenAI	effort: medium	4.00	20.00	0.40	–	Full 200, 3 runs
GPT-5.6 Terra	OpenAI	effort: medium	2.00	12.00	0.20	–	Full 200, 3 runs
GPT-5.6 Luna	OpenAI	effort: medium	0.20	1.20	0.02	–	Full 200, 3 runs
Claude Opus 4.8	Bedrock	adaptive, effort: medium	5.00	25.00	0.50	6.25	Balanced-30
Claude Opus 4.7	Bedrock	adaptive, effort: medium	5.00	25.00	0.50	6.25	Full 200, 1 run

Appendix

Table 2: Model configurations and prices. Reasoning settings are OpenAI’s reasoning effort, Anthropic’s extended thinking (adaptive with an effort level, or a fixed token budget), and Gemini’s thinking level. Prompt caching is enabled explicitly for Claude on Bedrock, which charges for cache writes; OpenAI and Gemini cache prompts automatically and charge only the discounted cache-read price. Evaluation gives the ProgramBench runs in which each model appears.

Figure 10: Solo agents versus homogeneous Terra teams. Each configuration is run for three trials, and error bars are 95% intervals over tasks of each team’s paired difference from the solo agent with the same guidance.

Figure 11: Solo agents versus homogeneous Luna (bottom) teams. Each configuration is run for three trials, and error bars are 95% intervals over tasks of each team’s paired difference from the solo agent with the same guidance.

Figure 12: Solo agents versus homogeneous teams for Claude Opus 4.7. Each is a single run, and error bars are 95% intervals over tasks of each team’s paired difference from the solo agent with the same guidance.

Figure 13: Solo agents versus homogeneous teams for Claude Haiku 4.5. Each is a single run, and error bars are 95% intervals over tasks of each team’s paired difference from the solo agent with the same guidance.

Figure 14: Stage guidance for homogeneous Terra (left) and Luna (right) teams, as in Figure 4 .

Figure 15: Efficiency

\eta

of homogeneous Terra (left) and Luna (right) teams relative to each tier’s Solo PIV agent, as in Figure 4 .

Figure 16: Stage guidance for heterogeneous teams with Terra lead with Luna workers.

Figure 17: Efficiency

\eta

for a Terra lead (left) and a Sol lead (right) with Luna workers, relative to the lead’s Solo PIV agent. A missing bar marks a team that is both cheaper and higher-reward than the solo agent (it dominates it). Error bars are 95% bootstrap intervals over tasks.

Figure 18: Solo PIV agents versus homogeneous and heterogeneous (Luna-worker) teams under PIV guidance, the PIV counterpart of Figure 7 . Teams use asynchronous push.

Figure 19: Mean reward (top) and total inference cost (bottom) with workers fixed at Luna and the lead varied across Luna, Terra, and Sol, under blocking (left) and asynchronous push (right). Faint lines show implementation and verification guidance; the gray dotted line is each lead’s Solo PIV agent. Reward error bars are paired 95% intervals relative to each lead’s Solo PIV agent.

Figure 20: Task-level reward comparisons between two GPT-5.6 Sol solo agent and teams runs (top: minimal guidance; bottom: PIV guidance). Each panel pools three of runs (600 points total); the dashed diagonal line marks equal reward. Annotations give the mean pairwise correlation, the mean absolute difference, and each run’s 200-task mean.

Sol lead
		Reward
Configuration	Schedule	Mean	95% CI	SD	Range	Task $\|\Delta\|$	Task correlation
Solo minimal	–	0.521	$\pm$ 0.030	0.008	0.015	0.072	0.86
Solo PIV	–	0.577	$\pm$ 0.031	0.006	0.011	0.066	0.90
with Sol workers
Team minimal	Async	0.559	$\pm$ 0.032	0.001	0.001	0.069	0.88

Appendix

Table 3: Run-to-run variance for every GPT-5.6 configuration across three runs. All summary statistics are computed over 200-tasks across three runs. Task

|\Delta|

is the mean absolute difference in a single task’s reward between combinations of any two runs; task correlation is the mean pairwise correlation of task rewards between runs.

Figure 21: Per-task team activity for GPT-5.6 Sol teams pooled over three runs. Filled bars denote asynchronous and outlined bars blocking teams. In the delegation panel, the solid part of each bar counts delegations that spawn a new worker (and therefore equals the number of workers spawned), hatching marks delegations that resume an already spawned worker, and the dashed line marks the standard cap of five unique worker agents. In the tool-call panel, dashed lines mark Solo minimal and Solo PIV.

Sol lead
			Cost	Wall-clock	Tool	Workers		Resumed	Worker
Configuration	Schedule	Reward	($)	(min)	calls	spawned	Delegations	share	token share
Solo minimal	–	0.52	0.24	1.9	19.2	0.0	0.0	–	0.00
Solo PIV	–	0.58	0.38	2.5	22.4	0.0	0.0	–	0.00
with Sol workers
Team minimal	Async	0.56	0.91	2.6	84.0	4.2	4.4	0.03	0.46

Appendix

Table 4: Per-task team activity for GPT-5.6 models, grouped by lead model and worker model. All columns are per-task means pooled over three runs. Cost is inference cost per task; the resumed share is the fraction of delegations that resume a previously spawned worker, and the worker token share is the fraction of all tokens generated by workers.

Claude Opus 4.7 lead
			Cost	Wall-clock	Tool	Workers		Resumed	Worker
Configuration	Schedule	Reward	($)	(min)	calls	spawned	Delegations	share	token share
Solo minimal	–	0.51	2.81	9.6	59.0	0.0	0.0	–	0.00
Solo PIV	–	0.49	2.47	8.9	56.1	0.0	0.0	–	0.00
with Opus 4.7 workers
Team minimal	Async	0.51	3.23	11.3	69.1	0.4	0.4	0.05	0.24

Appendix

Table 5: Per-task team activity for Claude models on the full 200-task ProgramBench run (single run; same columns as Table 4 ).

Figure 22: Lift by difficulty tier for asynchronous Team minimal (relative to Solo minimal) and Team PIV (relative to Solo PIV), for all model setups. Lighter shades (leftwards in each group) denote easier tiers; error bars are 95% intervals over tasks.

Figure 23: Lift by difficulty tier for asynchronous Team P, I, V, and PIV relative to Solo PIV, for all model setups.

Figure 24: How work is divided as fixed-size teams grow (asynchronous Team P, I, V, and PIV; Sol lead with 2–16 Luna workers). Panels show the length of each worker’s assignment, files written by workers, total runs of the reference executable by workers, total report text returned to the lead, pairwise report similarity (word-set Jaccard; higher means more alike), and the lead’s runs of the reference executable after the last worker report. Files and reference runs are detected from shell commands, so file counts are lower bounds.

Figure 25: Change from a cap of five unique workers (mean of three runs) to twenty (one run) for asynchronous teams. Cost is shown as the percentage change in mean cost per task. Error bars on reward and cost are 95% intervals over tasks, paired by task.

Figure 26: Left: messages per task by direction when a send_message tool is available, for asynchronous Team minimal and Team PIV. Right: change in reward relative to the same team without the tool, with 95% intervals paired by task. Row configurations are aligned across panels.

Setup	W $\to$ L	L $\to$ W	W $\to$ W	W $\to$ W (Plus)	replied	calls	$\Delta$ reward
	Messages per task				Questions	Failed
Sol	14.2	2.0	0.7	1.0	74%	1%	$+0.4\pm 1.2$
Sol + Luna	12.4	1.2	0.2	0.3	16%	55%	$-1.3\pm 1.2$
Terra	11.7	1.3	0.3	0.5	28%	0%	$-1.4\pm 1.5$
Terra + Luna	12.8	1.2	0.3	0.3	20%	50%	$-3.5\pm 1.5$
Luna	10.4	0.2	0.3	0.2	9%	50%	$-7.1\pm 2.3$

Appendix

Table 6: Peer messaging for asynchronous teams under PIV guidance (cap of five workers). Messages per task count delivered messages from workers to the lead (W

\to

L), from the lead to workers (L

\to

W), and between workers (W

\to

W); Plus adds a worker instruction encouraging collaboration. Questions replied is the share of worker questions to which the lead messages that worker before the worker sends anything else. Failed calls is the share of worker send_message calls rejected by the harness.

\Delta

reward is the change relative to the same team without the tool, in points, with paired 95% intervals.

Figure 27: Balanced-30 comparison across model families. Left: solo and best-team mean reward. Middle: team lift over the solo agent with matching guidance (Solo minimal for Team minimal, Solo PIV for Team PIV). Right: workers spawned and delegations issued per task; the cap of five unique workers bounds the former but not the latter. Filled bars denote asynchronous and outlined bars blocking teams.

Figure 28: Submission structure on the 30 task subset for Solo minimal, Team minimal, Solo PIV, and Team PIV (teams asynchronous). Top: share of submissions whose dominant source language (by bytes) is the original program’s language, Python, or another language. Bottom: median number of source files (i.e. excluding benchmark-requested files such as compile.sh ).

Figure 29: Commit0-Lite lift of Team minimal over Solo minimal and of Team PIV over Solo PIV. Filled bars denote asynchronous and outlined bars blocking teams. Error bars are 95% normal intervals of the paired per-task differences over the 16 tasks.

Figure 30: Commit0-Lite lift of Team P, I, V, and PIV over Solo PIV, for asynchronous (left) and blocking (right) teams. Error bars are 95% normal intervals of the paired per-task differences over the 16 tasks.

Inference-Time Scaling

Inference-time scaling studies how extra computation during deployment can improve model performance through search, sampling, verification, or adaptive reasoning.

See all 113 papers

Sep 17, 2026

Scaling Discovery through Test-Time Communication

Science advances not in isolation but through collaboration, yet existing agentic systems capture little of this. Whether communicating agents help remains an open question with mixed prior results. We show that test-time communication can substantially outperform independent parallel attempts on challenging tasks, where sharing a breakthrough can push the whole group forward. We first study the effect of scaling multi-agent test-time communication, where agents have no predefined roles and communicate via a shared directory, on ARC-AGI-3, a benchmark requiring novel problem solving. We find that a team of

k

communicating agents, team@

k

, matches the success rate of

4k

independent agents, and this advantage grows with

k

, suggesting gains compound with scale. The effect is not merely efficiency: a task that no single agent can solve, a team of agents can solve reliably. Furthermore, these gains transfer to research-oriented tasks, given sufficient compute. On polyomino packing, communicating agents outperform best@

k

and exceed the prior best-known score. On MNIST classifier compression, communication surpasses the best-known human solution. A team of four agents produced a 1,957-byte classifier submission achieving 99.4% test accuracy, smaller than both the best-known human solution and the best single-agent result. These gains are not unconditional. Independent agents may outperform communication when compute is limited or when a clear measure of progress is absent. However, under sufficient compute and clear feedback, multi-agent communication consistently yields stronger results.

Jongho Park, Vasilis Kontonis, Shivam Garg +2

UC Berkeley · Microsoft Research

Multi-Agent Collaboration Multi-Agent Communication Test-Time Scaling AI Agents for Scientific Discovery Shared-Memory Multi-Agent Systems ARC-AGI

Ask PDF

May 2, 2026

Multi-Agent Reasoning Improves Compute Efficiency: Pareto-Optimal Test-Time Scaling

Advances in inference methods have enabled language models to improve their predictions without additional training. These methods often prioritize raw performance over cost-effective compute usage. However, computational efficiency is key for real-world applications with resource constraints. We provide a systematic analysis of the inference scaling strategies self-consistency, self-refinement, multi-agent debate, and mixture-of-agents, to study their computational performance tradeoffs. We evaluate methods on two reasoning benchmarks (MMLU-Pro, BBH) and include extensive parameter configurations (e.g., scaling the number of parallel predictions, agents, and debate rounds) across different model sizes. Across 34 configurations and over 100 evaluations, we compute the Pareto-optimal front to select methods that achieve the best accuracy with the lowest computational budget. Notably, inference scaling improves accuracy by up to +7.1% points over chain-of-thought at the highest evaluated budgets (20x the CoT compute budget) on MMLU-Pro. With an equal computing budget, debate and mixture-of-agents outperform self-consistency by 1.3% and 2.7% points, respectively. While self-consistency saturates earlier, multi-agent gains persist, particularly on more complicated tasks. We identify a simple multi-agent design guideline: mixture-of-agents is most efficient when the number of parallel generations exceeds the number of sequential aggregations.

Florian Valentin Wunderlich, Lars Benedikt Kaesberg, Jan Philip Wahle +2

University of Göttingen, Germany

Self-Consistency Decoding Cost-Aware Inference Inference-Time Scaling Efficient Language Model Reasoning Multi-Agent Reasoning

Ask PDF

Oct 6, 2026

SquidAgent: Parallelize Wisely, Coordinate Efficiently

LLM-based agents solve complex multi-step tasks, but sequential execution incurs substantial latency. In principle, parallelizing work across multiple agents should yield near-linear speedups. Yet existing parallel multi-agent systems often run slower than a single-agent baseline. We attribute this gap to two hidden costs that parallel execution incurs but a serial agent avoids. First, there is a re-exploration cost: redundant effort spent by parallel workers reconstructing context that the orchestrator already possesses, such as prior decisions, that would otherwise be inherited implicitly in a serial execution. Second, there is an alignment cost: the overhead required to reconcile inconsistencies across independently generated outputs. We thus derive a principled decision criterion: a layer should be parallelized only when its critical-path cost, plus re-exploration and alignment overheads, is lower than the corresponding serial cost. While this criterion is naturally expressed in wall-clock time, we observe that LLMs are poorly calibrated when asked to estimate task duration. To address this, we instead measure cost in predicted output tokens, which we empirically find LLMs can estimate substantially more reliably than wall-clock time. Building on this token-based criterion, we propose SquidAgent. It estimates all token budgets in a single planning step, forks each worker directly from the orchestrator's session to eliminate re-exploration cost, and replaces post-hoc reconciliation with a pre-generated shared convention block that converts alignment into a bounded upfront cost. A deterministic scheduler then applies the criterion layer by layer. Empirically, SquidAgent achieves a 2.2

\times

mean throughput improvement and a 2.6

\times

mean wall-time speedup over Claude Code, and a 2.0

\times

throughput improvement over the strongest multi-agent baseline.

Yexiong Lin, Shanshan Ye, Yu Yao +3

The University of Sydney · Mohamed bin Zayed University of Artificial Intelligence · University of Technology Sydney +1

LLM Agent Orchestration LLM Agent Workflow Optimization Multi-Agent Collaboration LLM Agent Efficiency

Ask PDF

Abstract

Inference-Time Scaling

Explore similar work

Scaling Discovery through Test-Time Communication

Multi-Agent Reasoning Improves Compute Efficiency: Pareto-Optimal Test-Time Scaling

SquidAgent: Parallelize Wisely, Coordinate Efficiently