cs.AINov 17, 2026

Context Has a Price: Adding and Removing Context for LLM Coding Agents under Prefix Caching

Authors: Zerui Yang, Miaomiao Dong, Quan Xue, Chaorui Zhang, Xuanyu Zheng, Xiyuan Wang, Haolin Ruan, Xueyan Niu, 

Organizations: Huawei Technologies Ltd.

Abstract

Coding agents repeatedly send their conversation history to a language model. Adding useful code may reduce search, while removing old tool outputs may shorten later prompts. But do these changes reduce the cost of solving a task? We study this as a measurement problem in more than 20,000 runs of open models on SWE-bench Verified, SWE-bench and Loc-Bench. We account separately for uncached input, cached input and output tokens. Three findings emerge. First, in our main evidence-package experiments, retrieved code reduces read and search commands by 18--34%, but increases cost by 33--42% when cached input costs one tenth of uncached input, without a measurable improvement in resolution. Second, removing old observations can either save money or increase cost: the same approach saves 19--36% without a cache discount but increases cost by 38--116% with a tenth-price cache. Third, shorter prompts do not guarantee cheaper runs because agents may respond by taking more steps. We also find that concurrent experiments can share cached prefixes and distort cost comparisons. These results show why context-management methods should be evaluated using both token prices and the agent's subsequent behavior.

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. The Physical Cost of Context: Quality-Constrained Resource Accounting for Coding Agents

    Oct 26, 2026Zhongming Yu, Yitong Ma, Hengjia Yu +4LLM Agent EfficiencyLLM Context Management

  2. Token Reduction Is Not Cost Reduction

    Jul 13, 2026Sarel Weinberger, Amir HozezAI Coding AgentsCost-Aware Inference

  3. When Does Restricting a Coding Agent to execute_code Help? A Regime ×\times Agent-Design Ablation

    Jul 12, 2026Hong Yang, Qi Yu, Travis DesellCoding AgentsTool-Using Agents