The Physical Cost of Context: Quality-Constrained Resource Accounting for Coding Agents

Zhongming Yu; Yitong Ma; Hengjia Yu; Boqin Yuan; Xiangye Lin; Hejia Zhang; Jishen Zhao

cs.SEOct 26, 2026

The Physical Cost of Context: Quality-Constrained Resource Accounting for Coding Agents

Authors: Zhongming Yu, Yitong Ma, Hengjia Yu, Boqin Yuan, Xiangye Lin, Hejia Zhang, Jishen Zhao

Organizations: University of California, San Diego · University of Massachusetts Amherst

Abstract

Coding agents resend a growing context on every model call, and context policies are usually judged by the prompt tokens they save. Executed work also depends on call count and prefix reuse: a caching runtime prefills only the tokens after the longest cached prefix. A shorter prompt can therefore cost more if it rewrites cached history. Our quality-gated protocol splits executed prefill into logical prompt minus reused prefix and measures KV residency, time, and energy. On 450 localization cases derived from SWE-rebench V2, preloading ten retrieved blocks (Eager) and adding anchored history compaction (Compact) both pass a five-point non-inferiority gate. In a separate 900-cell study, Eager cuts model calls by 35.7% and the GPU-energy proxy by 33.2% against on-demand search (Grep); Compact saves nothing further. Enabling prefix caching leaves Eager's prompt volume similar (+8.3%; its 95% CI spans zero) but cuts its uncached prefill by 67.7%. Deleting history can turn a token saving into extra work. With mini-SWE-agent on 100 SWE-bench Verified instances, keeping only the last five observations (Mask-5) cuts Eager's prompt volume by 33.3% but loses 135.9k reusable tokens per task. Uncached prefill rises by 608.5%, the energy proxy by 125.6%, and the policy misses the quality gate. Policies thus save physical work by shortening the trajectory and can multiply it by editing reusable history.

Figures & tables

Figure 1: Experimental boundary from repository to device. (a) Frozen dense retrieval selects top-10 blocks $R$ . (b) Policies alter evidence delivery and visible history. (c) Request and prefix-reuse boundaries translate logical prompt $P$ into uncached work $U$ and device cost, which we compare only after checking quality.

Figure 2: Quality gates and resource effects. (a) Quality contrasts

\Delta Q_{p:c}=Q(p)-Q(c)

for localization (

n=450

instances) and patch generation (

n=100

, 32-call budget); only

E_{\mathrm{loc}}{:}G

C_{\mathrm{loc}}{:}G

, and

M_{5}{:}E_{\mathrm{patch}}

face the gate. (b) Cache-off effects relative to

G

; wall resamples 25 repositories and the other metrics resample three run-block shards. (c)

M_{5}/E_{\mathrm{patch}}

resource ratios over eight paired shards.

G/N

receive no retrieved seed. Whiskers are paired-bootstrap 95% intervals.

Task quality
Metric	Grep	Eager	Compact
	control	vs. $G$	vs. $G$
Recall@5	.470	.478	.494
Full@5	.307	.318	.329
Format success	.904	.976	.987
Runtime completion	1.000	1.000	1.000

Table 1: Localization-only profile for

\mathcal{P}_{\mathrm{loc}}=\{G,E,C\}

. Quality uses 450 instances; resources use the 100-task, three-repeat cache-off study. “Full” means recall one. Parenthesized changes are relative to

G

(green: reduction); failures remain charged. Per-full-recall rows divide by the full-recall rate of the 100-task instrumented study.

Task outcome (of 100 attempts)
Metric	Mini	Eager	Compact	Mask-5
	control	vs. $N$	vs. $E$	vs. $E$
Resolved	30	29	30	26
Submitted / call-capped	45 / 55	44 / 56	45 / 55	40 / 60
Generation resource per attempt
Prompt ktok	220.7	225.6 (+2.2%)	210.8 (−6.5%)	150.5 (−33.3%)

Table 2: Patch-generation-only profile for

\mathcal{P}_{\mathrm{patch}}

: mini-SWE-agent on 100 SWE-bench Verified instances under a 32-call budget. Resources are generation-only means per attempt; parenthesized changes use the baseline under each arm (green: reduction; red: increase).

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Field	Captured value
Host	NVIDIA GB10, 128 GB unified memory, aarch64
OS / driver	Ubuntu 24.04.4 LTS; kernel 6.17.0-1026-nvidia ; NVIDIA driver 580.159.03; CUDA 13.0
Primary target	Qwen3.6-35B-A3B-FP8, revision 95a723d08a94
Primary runtime	vLLM 0.19.2rc1.dev134+gfe9c3d6c5
Cache intervention	Same Qwen3.6/vLLM stack; treatment adds only --enable-prefix-caching , with a reset before each policy–repository block
External target	Qwen3.8-27B-FP8 [ 49 ] , revision 017b9c7af6b5

Appendix

Table 3: Frozen execution metadata. Readable versions and abbreviated revisions are shown here; the artifact manifest records full revisions and integrity checks.

Policy	Recall@5	Prompt	Uncached	Req.	Wall	KV % $\cdot$ s	kJ
Grep	.772	98.779k	18.245k	10.34	56.76	165.8	1.474
Eager	.800	68.456k	13.749k	7.15	42.71	118.9	1.080
Compact	.802	84.226k	19.192k	8.87	52.16	133.8	1.362

Appendix

Table 4: Exploratory external SGLang stack (100 tasks, one repetition). Prompt and uncached-prefill values are SGLang counters per attempted task; KV is integrated normalized full-attention-pool occupancy; energy is the same idle-subtracted GPU proxy as in the main paper.

Regime	Policy	Final %	Output/call	Uncached/call	TTFT s	E2E/call s
Primary	Grep	7.9	160.4	–	2.288	5.218
	Eager	12.0	190.2	–	2.312	5.721
	Compact	10.7	182.3	–	2.244	5.468
Exploratory (SGLang)	Grep	9.7	126.2	1.765k	4.049	5.410
	Eager	14.0	137.7	1.923k	4.161	5.840
	Compact	11.3	126.5	2.164k	4.288	5.692

Appendix

Table 5: Per-successful-server-call decomposition. Final is the fraction of captured trace calls that produce the final answer; output and TTFT use aggregate server counters. E2E/call is mean server end-to-end latency per successful model request. In this table, uncached prefill is reported only for the exploratory SGLang rows. These ratios are descriptive, not paired estimates.

Repository	Lang.	Commit	Repository	Lang.	Commit
astropy/astropy	Py	0df94ff70979	tokio-rs/tokio	Rust	ab53bf0c4727
matplotlib/matplotlib	Py	6786f437df54	uutils/coreutils	Rust	a40884758329
pydata/xarray	Py	45c0a114e2b7	axios/axios	TS/JS	c6cce43cd944
scikit-learn/scikit-learn	Py	97523985b39e	babel/babel	TS/JS	35116224d21e
sympy/sympy	Py	d7c304511569	facebook/docusaurus	TS/JS	5e9e1d051b22
caddyserver/caddy	Go	4512be49a9fa	preactjs/preact	TS/JS	c77e6284c199

Appendix

Table 6: Repository snapshots used by the frozen localization matrix, five per language group (Py: Python; TS/JS: TypeScript/JavaScript). Revisions are abbreviated; the artifact manifest stores full commits.

LLM Agent Efficiency

See all 58 papers

Same Task, Different Work: Prompt-Induced Waste in Coding Agents
Aug 2, 2026Sarel Weinberger, Amir HozezLLM Inference Efficiency Overthinking in Language Models
PointFive
Cache Engineering for Agents
Oct 20, 2026Tianyu Fu, Ruoxi Wang, Zihan Min +6KV-Cache Management Memory Management for LLM Agents
Tsinghua University · The Chinese University of Hong Kong
An Empirical Cost Attribution of Context-Compression Gateways in Multi-Turn Coding Agents
Aug 19, 2026Luzhuo Chen, Jiayu ShiContext Compression LLM Agent Workflow Optimization
Paritok

Aug 2, 2026J/K move · Enter open · S save

Same Task, Different Work: Prompt-Induced Waste in Coding Agents

Sarel Weinberger, Amir Hozez

PointFive

LLM Inference Efficiency Overthinking in Language Models AI Coding Agents AI Agent Benchmarks Coding Agent Benchmarks

Open paper AskPDF arXiv

Two prompts can request the same code change and produce the same correct patch, yet cause a coding agent to perform radically different kinds and amounts of work. We study this effect in a preregistered benchmark spanning 4,644 valid runs, 24 deterministic coding tasks, seven reasoning models, and two real agent harnesses. The central finding is that prompt wording does not merely scale total effort; it changes where that effort is spent. Multiple approaches and deep thinking primarily inflate reasoning. Multiple approaches increases reasoning by 2.4x to 7.4x across all six open models and creates about three elaborated but discarded solution branches, while still yielding only one implemented solution and no success gain. Maximum certainty activates a different pathway: repeated verification propagates into extra test runs, tool calls, turns, latency, and context growth. Runs with high redundant verification cost 18x the clean-run median, execute 2.5x more tool calls, and take 3x longer, again without a success gradient. These mechanisms therefore have distinct cost carriers: some prompts are reasoning-heavy and token-borne, while others are tool-heavy and system-borne. Harness design amplifies both effects and changes cost per successful task by 5x to 30x in our setting. The findings survive a frozen holdout, paraphrase tests, a Kimi-K3 replication, and a first-party Claude Sonnet 5 study. In contrast, bounded-efficiency wording preserves diagnosis and final validation while avoiding the measured waste mechanisms. Prompt engineering for coding agents is therefore work design: it determines what the agent thinks through, what it executes, and when it stops.

Abstract

LLM Agent Efficiency

Explore similar work

Same Task, Different Work: Prompt-Induced Waste in Coding Agents

Cache Engineering for Agents

An Empirical Cost Attribution of Context-Compression Gateways in Multi-Turn Coding Agents