Where Should a Stuck Agent Spend Its Next Tokens? Causal Attribution of Inference-Time Resources

Saiyue Lyu; Jerrold Huang; Yunbei Zhang; Janet Wang; Zhenting Wang; Yingqiang Ge; Haibo Zhang

cs.SEOct 20, 2026

Where Should a Stuck Agent Spend Its Next Tokens? Causal Attribution of Inference-Time Resources

Authors: Saiyue Lyu, Jerrold Huang, Yunbei Zhang, Janet Wang, Zhenting Wang, Yingqiang Ge, Haibo Zhang

Organizations: University of British Columbia · University of Toronto · Tulane University · Rutgers University ·

Abstract

Inference-time scaling is often treated as a token-budget problem, but a failed coding agent can seek qualitatively different resources: computation, information, or feedback. We introduce Banzhaf Resource Attribution (BRA), a controlled factorial framework that measures each resource's marginal contribution to repair and its interactions while holding the failed state and revision protocol fixed. We evaluate six models on 194 function-level failures from MBPP+ and HumanEval+ and 126 repository-level failures from SWE-bench Verified. Task-relevant specifications consistently improve repair, whereas public-test feedback provides small gains and retrieved code little to none. The value of additional computation declines with model capability, while information remains useful; resource channels largely exhibit diminishing returns, although complementary information can reinforce itself. Yet three of four frontier models select feedback in 95-100% of cases and obtain lower gains than uniform random resource selection. BRA thus exposes a gap between the resources agents prefer and those that measurably improve repair.

Figures & tables

Figure 1: BRA overview. A frozen failed checkpoint

s=(x,y,h)

is the unit of analysis. Computation, information, and feedback enter as separate interventions, each through its own channel, and every bundle of the three is assigned to the same

s

; a common final revision and held-out scoring identify the response surface, the eight cell means

q_{cif}

. A resource’s value is its marginal contribution averaged over every bundle of the other two, its Banzhaf value; the interactions tell substitutes from complements. The fingerprint compares models on the same states and scores the agent’s own resource choice by its regret.

Figure 2: Resource effects. Average main effects on held-out utility across 194 function-level checkpoints. Bars show 95% checkpoint-bootstrap intervals. Dashed lines mark zero.

Figure 3: Information content. Points show average information effects on held-out utility across 126 repository-level checkpoints (

k=3

); bars show 95% checkpoint-bootstrap intervals.

		retrieval information				audited specification
Model	$q_{000}$	$q_{111}$	$\mathrm{AME}_{C}$	$\mathrm{AME}_{I}$	$\mathrm{AME}_{F}$	$q_{111}$	$\mathrm{AME}_{C}$	$\mathrm{AME}_{I}$	$\mathrm{AME}_{F}$
Qwen3-14B	0.026	0.059	0.016 [ $-$ 0.005, 0.039]	0.009 [ $-$ 0.008, 0.028]	0.001 [ $-$ 0.020, 0.020]	0.125	0.034 [0.012, 0.057]	0.048 [0.022, 0.078]	0.008 [ $-$ 0.011, 0.026]
DeepSeek-V3.2	0.238	0.275	0.039 [ $-$ 0.001, 0.078]	0.007 [ $-$ 0.015, 0.028]	$-$ 0.006 [ $-$ 0.027, 0.014]	0.359	0.034 [0.006, 0.064]	0.083 [0.045, 0.123]	0.011 [ $-$ 0.012, 0.035]
GPT-5.6	0.423	0.447	0.024 [ $-$ 0.005, 0.054]	0.005 [ $-$ 0.019, 0.027]	$-$ 0.004 [ $-$ 0.027, 0.020]	0.538	0.009 [ $-$ 0.016, 0.037]	0.103 [0.045, 0.159]	0.009 [ $-$ 0.010, 0.028]
Claude Sonnet 4.5	0.324	0.338	0.014 [ $-$ 0.016, 0.047]	$-$ 0.002 [ $-$ 0.021, 0.018]	0.009 [ $-$ 0.012, 0.030]	0.392	$-$ 0.024 [ $-$ 0.053, 0.004]	0.088 [0.043, 0.137]	0.001 [ $-$ 0.017, 0.019]

Table 1: Repository-level fixed-state panel (

N=126

k=3

): empty and full-resource cells and average main effects with 95% intervals. Left: BM25-retrieved repository context as the information packet. Right: the same

I{=}0

cells with the

I{=}1

cells re-run under an audited oracle specification of the fix. Bold:

p<0.05

. Interactions, artifact strata, and the native-failure panel are extended in Table 6 .

Panel of Figure 4 — Figure 4: Self-allocation probe ( $3$ rollouts/checkpoint; repository level uses the specification rung for $I$ ). Left: share of requests for each resource. Right: mean realised gain $G(\pi)$ of the model’s own choices, of always requesting information, of a uniformly random choice among $C$ , $I$ , $F$ , and of the per-checkpoint oracle $G^{\star}$ .

Panel of Figure 5 — Figure 5: Ceiling checks, function level, with 95% checkpoint-bootstrap intervals. Top: main effects on all checkpoints (open) and on the $24$ checkpoints on which every model has $q_{000}<0.5$ (filled). Bottom: pairwise interactions on all checkpoints (open) and on the checkpoints with $q_{000}<0.5$ for that model (filled; $N$ in Table 5 ). Information stays largest away from the ceiling, and the interactions remain negative or near zero; because both subsets are selected on $q_{000}$ , the size of the shifts is not an unbiased estimate (Appendix H ).

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Family	Model	$\phi_{C}$	$\phi_{I}$	$\phi_{F}$	$\beta_{C}$	$\beta_{I}$	$\beta_{F}$	$\sum\phi$	$q_{111}-q_{000}$
function level	Qwen3-4B	0.128	0.105	0.027	0.131	0.108	0.030	0.260	0.260
	Qwen3-14B	0.113	0.139	0.030	0.110	0.136	0.026	0.282	0.282
	Qwen3-32B	0.107	0.135	0.039	0.101	0.129	0.033	0.281	0.281
	DeepSeek-V3.2	0.055	0.088	0.014	0.053	0.086	0.012	0.157	0.157
	GPT-5.6	0.011	0.065	0.020	0.010	0.064	0.019	0.096	0.096
	Claude Sonnet 4.5	0.007	0.067	0.007	0.007	0.067	0.007	0.081	0.081

Appendix

Table 2: Shapley values

\phi_{X}

and Banzhaf values

\beta_{X}=\mathrm{AME}_{X}

of the three resources, averaged over checkpoints. The Shapley values sum exactly to the full-resource gain; the two values differ by at most

0.006

and rank the resources identically for every model.

Family	Model	scalar	vector	both	$C$	$I$	$F$	$C$	$I$	$F$	submod.	monot.	25–75%	100%
		non-fungibility: CV $R^{2}$			$\Delta U$ per 1k tokens			$\Delta U$ per USD			share of surfaces		greedy / optimal
function	Qwen3-4B	0.100	0.141	0.137	0.15	0.54	0.28	–	–	–	0.25	0.33	0.90	0.80
	Qwen3-14B	0.103	0.153	0.145	0.13	1.18	0.43	–	–	–	0.27	0.32	0.96	0.88
	Qwen3-32B	0.111	0.158	0.156	0.11	1.25	0.48	325	7668	2979	0.26	0.28	0.97	0.91
	DeepSeek-V3.2	0.043	0.095	0.087	0.04	0.80	0.17	36	1216	267	0.44	0.46	0.99	0.97
	GPT-5.6	0.038	0.040	0.051	0.03	0.60	0.19	–	–	–	0.62	0.61	0.99	0.99

Appendix

Table 3: Costs, non-fungibility, and allocation (repository level uses the specification arm for

I

). Non-fungibility : within-checkpoint (fixed-effect) five-fold cross-validated

R^{2}

for predicting a checkpoint’s eight cell means from a binned function of the cell’s total token cost (scalar), from the resource indicators (vector), and from both. Return per cost : utility gain of each single-resource cell over the empty cell per

1000

tokens the resource adds to the cell (input and output of the planning and revision calls; the information cost is the packet’s tokens, not the oracle’s), and per USD at list prices (“–”: local or unpriced model). Share of surfaces satisfying the exact submodularity and monotonicity conditions (no noise tolerance). Greedy / optimal : mean ratio of the gain of the greedy-by-return rule to that of the optimal affordable bundle, minimum over budgets of 25–75% and at 100% of the total cost of all three resources.

Model	$q_{000}$	$q_{111}$	$\mathrm{AME}_{C}$	$\mathrm{AME}_{I}$	$\mathrm{AME}_{F}$	share $C$ / $I$ / $F$ (%)	Het.
Qwen3-4B	0.338	0.597	0.131 [0.097, 0.164]	0.108 [0.080, 0.139]	0.030 [0.011, 0.051]	49 / 40 / 11	0.30
Qwen3-14B	0.448	0.730	0.110 [0.076, 0.143]	0.136 [0.105, 0.170]	0.026 [0.009, 0.043]	40 / 50 / 10	0.45
Qwen3-32B	0.535	0.816	0.101 [0.074, 0.131]	0.129 [0.101, 0.159]	0.033 [0.016, 0.051]	39 / 49 / 13	0.39
DeepSeek-V3.2	0.732	0.889	0.053 [0.031, 0.076]	0.086 [0.059, 0.114]	0.012 [0.003, 0.021]	35 / 57 / 8	0.34
GPT-5.6	0.813	0.909	0.010 [ $-$ 0.006, 0.026]	0.064 [0.038, 0.091]	0.019 [0.006, 0.033]	11 / 69 / 20	0.29
Claude Sonnet 4.5	0.817	0.898	0.007 [ $-$ 0.010, 0.025]	0.067 [0.043, 0.092]	0.007 [ $-$ 0.006, 0.020]	8 / 83 / 9	0.28

Appendix

Table 4: Function-level fixed-state panel (

N=194

; held-out utility

U

): empty and full-resource cells, average main effects, each effect’s share of their sum, and heterogeneity (share of checkpoints whose best single resource differs from the population’s).

Model	$N_{\text{far}}$	all	far	all	far	all	far	$\Gamma_{CIF}$	$N_{C}$	$N_{I}$	$N_{F}$
		$\Gamma_{CI}$		$\Gamma_{CF}$		$\Gamma_{IF}$
Qwen3-4B	123	0.008	$-$ 0.017	$-$ 0.090	$-$ 0.132	0.028	0.027	$-$ 0.040	0.080	0.117	$-$ 0.011
Qwen3-14B	103	$-$ 0.039	$-$ 0.098	$-$ 0.056	$-$ 0.102	$-$ 0.027	$-$ 0.047	0.044	0.073	0.114	$-$ 0.004
Qwen3-32B	88	$-$ 0.059	$-$ 0.123	$-$ 0.045	$-$ 0.114	$-$ 0.028	$-$ 0.062	0.070	0.067	0.103	0.014
DeepSeek-V3.2	53	$-$ 0.041	$-$ 0.103	$-$ 0.004	$-$ 0.009	$-$ 0.023	$-$ 0.074	0.027	0.037	0.061	0.005
GPT-5.6	35	$-$ 0.026	$-$ 0.129	$-$ 0.002	$-$ 0.077	$-$ 0.019	$-$ 0.159	0.013	$-$ 0.001	0.044	0.012

Appendix

Table 5: Interactions and conditional necessities, function level. Pairwise interactions are given on all checkpoints and on the

N_{\text{far}}

checkpoints far from the ceiling (

q_{000}<0.5

for that model; Holm-adjusted within each sample). Negative pairwise terms indicate substitution.

N_{X}

is the loss from removing

X

from the full bundle.

Panel of Figure 6 — Figure 6: Label prevalence, function level ( $\tau=0.2$ , ceiling $0.8$ ). “One resource” and “several resources” are checkpoints responsive to exactly one or to more than one single resource.

Panel of Figure 7 — Figure 7: Dose curves on the 40 hardest function-level checkpoints (DeepSeek-V3.2, $k=5$ ; empty cell $q_{000}=0.298$ , dashed). Left: the computation-only cell $q_{100}$ against the planning budget. Right: the feedback-only cell $q_{001}$ against the feedback report level. Error bars are 95% checkpoint-bootstrap intervals of the paired gain over the empty cell, drawn around the mean.

(a) Interactions (

N=126

)

retrieval information audited specification Model

\Gamma_{CI}

\Gamma_{CF}

\Gamma_{IF}

\Gamma_{CIF}

\Gamma_{CI}

\Gamma_{CF}

\Gamma_{IF}

\Gamma_{CIF}

Qwen3-14B

-

0.021

-

0.024

-

0.005 0.027 0.015

-

0.021 0.009 0.033 DeepSeek-V3.2

-

0.006

-

0.012

-

0.018

-

0.010

-

0.015

-

0.022 0.017

-

0.030 GPT-5.6 0.028

-

0.021

-

0.008

-

0.002

-

0.001

-

0.032 0.018

-

0.023 Claude Sonnet 4.5 0.053

-

0.040 0.009

-

0.031

-

0.024

-

0.019

-

0.007 0.011

(b) Artifact strata : the agent’s patch breaks the test run (

N=57

) or target tests fail (

N=69

)

Model stratum

q_{000}

q_{111}

\mathrm{AME}_{C}

\mathrm{AME}_{I}

(retr.)

\mathrm{AME}_{I}

(spec.)

\mathrm{AME}_{F}

Qwen3-14B test run broken 0.041 0.023

-

0.009 0.000 0.019

-

0.000 tests execute 0.014 0.088 0.036 0.017 0.073 0.001 DeepSeek-V3.2 test run broken 0.221 0.257 0.043

-

0.004 0.104

-

0.005 tests execute 0.252 0.289 0.035 0.016 0.066

-

0.007 GPT-5.6 test run broken 0.468 0.412 0.004

-

0.029 0.096

-

0.026 tests execute 0.385 0.475 0.040 0.033 0.108 0.014 Claude Sonnet 4.5 test run broken 0.294 0.310 0.026 0.003 0.097

-

0.006 tests execute 0.349 0.360 0.005

-

0.006 0.081 0.021

k=3

)

Information arm

q_{000}

q_{111}

\mathrm{AME}_{C}

\mathrm{AME}_{I}

\mathrm{AME}_{F}

retrieval 0.097 0.115 0.030 [

-

0.024, 0.104]

-

0.004 [

-

0.046, 0.029]

-

0.007 [

-

0.042, 0.021] audited specification 0.097 0.187 0.047 [0.002, 0.099] 0.079 [0.013, 0.159]

-

0.027 [

-

0.063, 0.003]

Appendix

Table 6: Repository-level robustness. (a) Interactions under the retrieval and specification arms (

N=126

). (b) Main effects by artifact stratum; “retr.” and “spec.” are the information effect under the two arms. (c) Native-failure panel.

Contrast	Qwen3-14B	DeepSeek-V3.2
placebo $-$ nothing	$-$ 0.016 [ $-$ 0.040, 0.007]	$-$ 0.002 [ $-$ 0.020, 0.016]
specification $-$ placebo	0.195 [0.149, 0.245]	0.126 [0.089, 0.164]
specification $-$ nothing	0.180 [0.134, 0.228]	0.124 [0.086, 0.162]

Appendix

Table 7: Placebo check, function level (paired by checkpoint). The placebo is a length-matched neutral text supplied in place of the specification.

Model	000	001	010	011	100	101	110	111
Qwen3-4B	0.338	0.389	0.418	0.517	0.499	0.481	0.608	0.597
Qwen3-14B	0.448	0.526	0.627	0.657	0.616	0.616	0.734	0.730
Qwen3-32B	0.535	0.622	0.725	0.749	0.706	0.713	0.801	0.816
DeepSeek-V3.2	0.732	0.764	0.856	0.851	0.814	0.828	0.884	0.889
GPT-5.6	0.813	0.846	0.903	0.910	0.841	0.865	0.898	0.909
Claude Sonnet 4.5	0.817	0.827	0.906	0.906	0.840	0.853	0.893	0.898

Appendix

Table 8: All eight cell means, function level.

Panel	Model	$\mathrm{AME}_{C}$	$\mathrm{AME}_{I}$	$\mathrm{AME}_{F}$
function	Qwen3-4B	0.212	0.155	0.032
	Qwen3-14B	0.174	0.200	0.024
	Qwen3-32B	0.162	0.192	0.022
	DeepSeek-V3.2	0.067	0.158	0.009
	GPT-5.6	0.010	0.136	0.009
	Claude Sonnet 4.5	0.003	0.140	0.007

Appendix

Table 9: Main effects under the binary outcome (all held-out tests pass). Information remains the largest effect wherever it was under utility; at function level computation vanishes for GPT-5.6 and Sonnet 4.5.

Inference-Time Compute Allocation

Inference-time compute allocation decides where, when, and how much computation models spend during prediction to improve quality under resource limits.

See all 82 papers

Scaling Inference-Time Compute with Teams of Coding Agents
Oct 12, 2026Keenan Samway, Daniel D'souza, Omar Darwiche Domingues +3Inference-Time Scaling Inference-Time Compute Allocation
Cohere Labs · Cohere
CodeRescue: Budget-Calibrated Recovery Routing for Coding Agents
Jul 21, 2026Qijia He, Jiayi Cheng, Chenqian Le +8AI Coding Agents Cost-Aware Inference
University of Washington · New York University · ByteDance +1
Scaling Laws for Agent Harnesses via Effective Feedback Compute
May 28, 2026Xuanliang Zhang, Dingzirui Wang, Keyan Xu +2Inference-Time Scaling Agent Harness Optimization
Harbin Institute of Technology

Oct 12, 2026J/K move · Enter open · S save

Scaling Inference-Time Compute with Teams of Coding Agents

Keenan Samway, Daniel D'souza, Omar Darwiche Domingues, Marzieh Fadaee+2

Cohere Labs · Cohere

Inference-Time Scaling Inference-Time Compute Allocation Multi-Agent Coding Multi-Agent Resource Allocation Software Engineering Agents Cost-Aware Inference

Open paper AskPDF arXiv

Multi-agent systems promise a new axis for scaling inference: rather than having one agent think longer or sample more attempts, a task is split across many agents that work in parallel and coordinate. However, in practice, more agents can easily produce duplicate work, coordination failures, and higher costs. We therefore formulate team design as a compute allocation problem: starting from a solo agent, where should additional inference compute go to reach the reward-cost Pareto frontier? We build a minimal multi-agent harness from the ground up and trace this frontier by varying, one at a time, the workflow stage at which a lead delegates to its workers (planning, implementation, or verification), the models that form the team, and the number of workers. On ProgramBench, teams outperform solo agents but cost three to seven times as much. With the right compute allocation, however, a heterogeneous 17-agent team comes within 1.4 points of a strong homogeneous 5-agent team at 70% lower cost. This efficiency stems from three findings. First, planning is the most valuable use of workers: in a homogeneous team whose lead decides how many workers to spawn, a lead that delegates only planning to its workers, and then implements and verifies the code itself, matches the reward of a lead that delegates to workers throughout planning, implementation, and verification, at half the cost. Second, cheap models pay off as planners but not as implementers: a strong lead with cheap planners recovers 47.5% of the reward gain of an all-strong team for only 13.9% of its additional cost, whereas cheap implementers lower reward. Third, planners keep improving reward as the team grows, whereas implementers and verifiers plateau beyond four workers. Together, our results offer a guide to spending an agent budget where it counts, given the task, cost, and latency constraints a user has.

Abstract

Inference-Time Compute Allocation

Explore similar work

Scaling Inference-Time Compute with Teams of Coding Agents

CodeRescue: Budget-Calibrated Recovery Routing for Coding Agents

Scaling Laws for Agent Harnesses via Effective Feedback Compute