TokenEconomizer: Experience-Driven Black-Box Cost Control for Coding Agents

Aaron J. Li; Weijia Xu; Chandan Singh; Zixiao Chen; Jeevana Priya Inala; Charlotte Siska; Acar Tamersoy; Bin Yu

cs.CLOct 15, 2026

TokenEconomizer: Experience-Driven Black-Box Cost Control for Coding Agents

Authors: Aaron J. Li, Weijia Xu, Chandan Singh, Zixiao Chen, Jeevana Priya Inala, Charlotte Siska, Acar Tamersoy, Bin Yu,

Organizations: UC Berkeley · Microsoft Research · Microsoft Security

Abstract

Advances in frontier coding agents have sharply increased the token costs of AI-assisted software engineering. Their token efficiency is typically evaluated on isolated tasks, whereas real-world developers use them to continuously build, update, and maintain large-scale projects. This mismatch motivates us to evaluate agents over sequences of realistic coding tasks, where we find that agents exhibit recurring environment-dependent token costs that existing methods struggle to reduce. To minimize these costs, we introduce TokenEconomizer, an external controller that cuts environment-specific token costs over a sequence of tasks. These cuts come from determining (i) which operational micro-skills to inject, (ii) which persistently irrelevant tools to prune, and (iii) when to intervene in unproductive failure loops. Across Claude Code, Codex, and GitHub Copilot CLI paired with frontier models, TokenEconomizer reduces the average token cost by 18.4-30.9% with comparable task success. Savings hold in every repository under every tested agent-model setting and generalize to diverse domains such as sequential web development and security auditing, where costs fall by 6.4-21.2% depending on task structure. Our work establishes adaptive black-box cost control as a promising direction for reducing token costs of coding agents in realistic settings.

Figures & tables

Panel of Figure 1 — Figure 1: Trajectory anatomy and the cost–quality tradeoff. (a) Schematic of how the persistent prefix and interaction history accumulate across agent turns. (b) Cost and Pass@1 change of each method relative to high-effort default execution, one point per method in each of the five agent–model settings; the dashed line marks the Pareto frontier.

Figure 2: Per-repository cache-aware cost change of the three fixed strategies relative to default execution on the 60 profiled tasks (negative is cheaper). Every strategy lowers cost in some repositories and raises it in others.

Panel of Figure 3 — Figure 3: Three trace-derived sources of cost-control headroom across the 60 profiled tasks: (a) actions repeated from earlier tasks in the same repository, with another repository’s history as a control; (b) loaded tools used by each repository, with frequencies indicated by square sizes; and (c) the cost share and trajectory lengths of failed tasks.

Figure 4: TokenEconomizer applies the same three meta-strategies across repositories, while completed traces refine repository-specific policies over instructions, loaded tools, and live intervention.

	Claude Code				Codex				Copilot CLI
Method	Sonnet 5		Opus 4.8		GPT-5.6 Terra		GPT-5.6 Sol		Sonnet 5
	$\Delta$ Cost $\downarrow$	Pass@1 $\uparrow$	$\Delta$ Cost $\downarrow$	Pass@1 $\uparrow$	$\Delta$ Cost $\downarrow$	Pass@1 $\uparrow$	$\Delta$ Cost $\downarrow$	Pass@1 $\uparrow$	$\Delta$ Cost $\downarrow$	Pass@1 $\uparrow$
Default	—	$74.7_{\pm 1.3}$	—	$74.0_{\pm 0.6}$	—	$73.8_{\pm 1.0}$	—	$80.1_{\pm 0.8}$	—	$75.0_{\pm 1.5}$
Default (medium effort)	$-26.6_{\pm 1.4}$	$70.0_{\pm 1.8}$	$-16.8_{\pm 2.1}$	$72.2_{\pm 0.4}$	$-25.7_{\pm 2.0}$	$69.6_{\pm 1.2}$	$-28.9_{\pm 1.9}$	$76.7_{\pm 0.7}$	$-25.4_{\pm 2.4}$	$72.6_{\pm 1.2}$
Default (low effort)	$-45.5_{\pm 1.7}$	$65.1_{\pm 0.4}$	$-31.0_{\pm 0.8}$	$70.4_{\pm 0.4}$	$-33.0_{\pm 2.2}$	$65.3_{\pm 1.4}$	$-43.9_{\pm 1.8}$	$69.8_{\pm 0.4}$	$-41.9_{\pm 1.8}$	$65.9_{\pm 0.8}$
RTK	$-1.1_{\pm 2.9}$	$73.1_{\pm 0.6}$	$-3.6_{\pm 3.5}$	$74.0_{\pm 1.4}$	$-4.7_{\pm 2.1}$	$74.3_{\pm 1.1}$	$+4.5_{\pm 2.5}$	$79.2_{\pm 1.2}$	$-8.9_{\pm 3.7}$	$76.9_{\pm 1.0}$

Table 1: SWE-Bench cost and Pass@1 by method. Cost is the paired relative change from high-effort default execution and includes all controller overhead; Pass@1 is the absolute rate. Every setting runs the full six-repository set with three seeds. All values are percentages; subscripts give the standard deviation across the three seeds. Bold marks, per setting, the highest Pass@1 and the lowest cost among methods whose Pass@1 exceeds default execution; grey costs come with a Pass@1 below default execution.

Panel of Figure 5 — Figure 5: SWE-Bench results relative to default execution. (a) Cost change in each of the 30 repository–setting cells. (b) Cumulative cost change of TokenEconomizer over each repository’s task stream, averaged over the two Claude Code settings. The cost–quality frontier is in Fig. 1(b) .

Table 2: Cost and quality relative to default execution (%); subscripts are standard deviations over three seeds. (a) Each meta-strategy alone and the full stack on SWE-Bench with Claude Code. (b) TokenEconomizer (TE) on Web-Bench (102 tasks; tasks after an early stop fail; cost paired over tasks both arms reached) and the security benchmarks (100 tasks each); Claude Code averages Sonnet 5 and Opus 4.8.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

	Default	RTK		Serena		RepoMap
Repository	Pass@1 $\uparrow$	$\Delta$ cost $\downarrow$	Pass@1 $\uparrow$	$\Delta$ cost $\downarrow$	Pass@1 $\uparrow$	$\Delta$ cost $\downarrow$	Pass@1 $\uparrow$
sklearn	10/10	$-6.9\%$	10/10	$+28.6\%$	10/10	$+30.5\%$	10/10
SymPy	10/10	$+7.3\%$	10/10	$+30.8\%$	10/10	$+9.8\%$	8/10
Django	9/10	$-19.9\%$	9/10	$-15.4\%$	8/10	$-16.2\%$	9/10
Sphinx	6/10	$+0.6\%$	8/10	$+2.1\%$	6/10	$-7.5\%$	8/10
Matplotlib	8/10	$-4.9\%$	9/10	$-4.8\%$	8/10	$+5.0\%$	9/10

Appendix

Table 3: Cost-profiling results by repository: tasks passed out of 10, and each strategy’s cost change relative to default execution on the same tasks.

Config	Value
Micro-skill injection
Skill sources	proposer
Proposer cadence	every 5 tasks
Confirmation	3 confirmations on non-overlapping edited files
Need gate	rate $\geq 0.4$ over $\geq 4$ exercises
Support $\gamma$ , $\lambda$	$0.8$ , $5$

Appendix

Table 4: Controller configuration, shared by all three harnesses.

State	Injected into $\Delta P_{i}$	Leaves the state when
Candidate	never	confirmed on 3 tasks with non-overlapping edited files $\rightarrow$ confirmed
Confirmed	on each task, if its need rate is $\geq 0.4$ over $\geq 4$ exercised tasks; otherwise withheld, and rechecked before the next task	$q_{i}(f)<0$ , or any contradiction of a high-stakes skill $\rightarrow$ retired
Retired	never	never; the proposer may propose a replacement under a new identifier

Appendix

Table 5: Micro-skill lifecycle. Only a confirmed skill can enter

\Delta P_{i}

, and retirement is permanent.

	Micro-skills				Tool schemas		Failure loops
Method	Proposed	Confirmed	Needed	Retired	Masked	Re-audits	Judged	Redirects	Aborts
Claude Code, Sonnet 5
Micro-skills	11.4	7.2	268	2.1	0.0	0.0	0.0	0.0	0.0
Tool pruning	0.0	0.0	0	0.0	15.1	3.3	0.0	0.0	0.0
Failure-loop guard	0.0	0.0	0	0.0	0.0	0.0	10.8	4.4	1.4
Combined	12.1	8.4	301	2.6	14.8	3.1	9.9	3.9	2.0

Appendix

Table 6: Meta-strategy activity of the ablation variants in the two Claude Code settings, as per-stream means. Table 7 defines the columns, and Table 2(a) gives the cost and Pass@1 of each variant.

Group	Column	Unit	Definition
Micro-skills	Proposed	skills / stream	candidate skills in the stream’s final ledger
	Confirmed	skills / stream	candidates that met the confirmation condition and were not retired
	Needed	tasks / stream	exercised tasks that needed a skill, summed over skills
	Retired	skills / stream	retired skills
Tool schemas	Masked	schemas / stream	tools still masked at the last task, not a cumulative count
	Re-audits	tasks / stream	tasks run as a full-catalog re-audit

Appendix

Table 7: Column definitions for Table 6 . Every entry is a mean over the six repository streams of a variant.

Token Efficiency

Token efficiency studies how to reduce language-model token use, cost, and latency while preserving task quality and equitable access.

See all 30 papers

Aug 26, 2026

Can your AI agent be cheaper? Investigating the effects of task specifications on token spend in agentic coding tasks

Agentic coding workflows are now widely deployed in real-world systems. With long-horizon reasoning and tool use, token usage has become an important consideration for both cost and efficiency. Two engineers using AI will solve the same problem differently. How the specification of a task shapes an agent's token spend, and whether that spend can be predicted in advance, are open questions. Here, we study the effects of different task specifications on agentic token spend with the Kimi K3 model at three thinking efforts. Across

2,700

runs, we show that reducing a full task specification to a bare user story raises token spend by

29.7\%

, while run-to-run variance remains unaffected by any prompt changes. We show that prompt-sensitivity is task-dependent, running from

13\%

115\%

. We fit a simple predictor that can price a full distribution of task specifications and thinking effort configurations from a single cheap probe on an unseen task within

36\%

, improving over prior work in predicting token spend. Our work provides initial results quantifying the effects of task specification on agentic token spend and introduces a method that can be used to systematically evaluate the cost of AI coding workflows.

Jakub Smékal

Stanford University

Coding Agents Agentic Code Generation Cost-Aware Inference Prompt Sensitivity

Ask PDF

Apr 24, 2026

How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks

The wide adoption of AI agents in complex human workflows is driving rapid growth in LLM token consumption. When agents are deployed on tasks that require a significant amount of tokens, three questions naturally arise: (1) Where do AI agents spend the tokens? (2) Which models are more token-efficient? and (3) Can agents predict their token usage before task execution? In this paper, we present the first systematic study of token consumption patterns in agentic coding tasks. We analyze trajectories from eight frontier LLMs on SWE-bench Verified and evaluate models' ability to predict their own token costs before task execution. We find that: (1) agentic tasks are uniquely expensive, consuming 1000x more tokens than code reasoning and code chat, with input tokens rather than output tokens driving the overall cost; (2) token usage is highly variable and inherently stochastic: runs on the same task can differ by up to 30x in total tokens, and higher token usage does not translate into higher accuracy; instead, accuracy often peaks at intermediate cost and saturates at higher costs; (3) models vary substantially in token efficiency: on the same tasks, Kimi-K2 and Claude-Sonnet-4.5, on average, consume over 1.5 million more tokens than GPT-5; (4) task difficulty rated by human experts only weakly aligns with actual token costs, revealing a fundamental gap between human-perceived complexity and the computational effort agents actually expend; and (5) frontier models fail to accurately predict their own token usage (with weak-to-moderate correlations, up to 0.39) and systematically underestimate real token costs. Our study offers new insights into the economics of AI agents and can inspire future research in this direction.

Longju Bai, Zhemin Huang, Xingyao Wang +5

University of Michigan · Stanford University · Microsoft AI +3

Coding Agents Cost-Aware Inference

Ask PDF

Sep 25, 2026

Analyzing and Mitigating Cost-Inefficient Behaviors in Coding Agents

Although effective, coding agents often incur substantial monetary costs. Their recurring cost-inefficient behaviors remain underexplored. We conduct the first study of behavioral cost inefficiencies in coding agents, analyzing 1,200 trajectories from Claude Code and Mini-SWE-Agent across four configurations on SWE-bench Verified. We identify three cost-inefficient behaviors: subsumed retrieval, similar script generation, and test re-execution. We then evaluate three mitigation strategies: structure-aware retrieval, agent-synthesized skills, and developer-designed skills, over 10k trajectories on held-out SWE-bench Verified and Pro tasks. Our main findings are: (1) The three behaviors affect 79.00%--98.00% of coding tasks and account for up to 22.75% of task cost. (2) Structure-aware retrieval can introduce retrieval overhead and alter agent delegation, causing inconsistent improvements in retrieval efficiency and cost increases of up to 28.14%. (3) Agent-synthesized skills tend to produce low-level, trace-specific guidance, limiting their effectiveness and generality. (4) In contrast, developer-designed skills provide high-level, trace-agnostic guidance, reducing cost by up to 41.73%, roughly twice the maximum gain from agent-synthesized skills.

Yiran Hu, Nan Jiang, Shanchao Liang +3

Purdue University

Coding Agents Cost-Aware Inference Agent Skill Learning

Ask PDF

Abstract

Token Efficiency

Explore similar work

Can your AI agent be cheaper? Investigating the effects of task specifications on token spend in agentic coding tasks

How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks

Analyzing and Mitigating Cost-Inefficient Behaviors in Coding Agents