cs.CLAug 19, 2026

An Empirical Cost Attribution of Context-Compression Gateways in Multi-Turn Coding Agents

Authors: Luzhuo Chen, Jiayu Shi

Organizations: Paritok

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Context compression is widely proposed as a way to cut the token bill of LLM coding agents, and public benchmarks report that aggressive compression preserves task-solving quality. These two facts do not imply the third one commonly assumed: that compressing file reads saves money in a real multi-turn agent. We instrument a production compression gateway (Paritok) between coding agents (Claude Code, Codex) and frontier LLMs (Claude Sonnet, GPT-5), and decompose the token bill of real sessions into three independent levers: tool-schema filtering, content compression of file reads and tool output, and history summarization. Measured in isolation under controlled A/B runs, the three save at fundamentally different rates. Tool-schema filtering removes a fixed block every turn, roughly 21K-57K tokens on a typical turn; it is linear in the turn count N and the only unambiguously and reproducibly positive lever. Content compression saves only about 2% of the cache-priced prefix per turn, but compressed reads accumulate in history and are re-sent on every later turn, so its cumulative saving grows quadratically, about 3350*N^2 tokens (measured), overtaking the fixed tool-filter saving within roughly 6 turns until the context window caps it. A non-destructive gateway lets the agent pull original bytes back on demand; each recall re-sends exactly the one segment just compressed away, so its cost is fixed and bounded rather than a multiplicative blowup, and heavy recall spends the accumulated saving back one segment at a time. Finally, a strong single-shot compression benchmark - 86.5% of SWE-bench quality retained at a 25.7% compression rate, achieved by the model this gateway deploys (Paritok-4B, reported separately) - is orthogonal to multi-turn agent cost and must not be cited as a cost-saving argument. We distill the results into an actionable recipe for where token-saving effort pays off.

Figures & tables

SessionTool-filter savedContent savedRatio
jinja2 ( filters.py , 14K)392,000392,00018,00018,000∼\sim 20:1
rich ( text.py , 10K, 34 req)1,220,0001,220,00028,00028,000∼\sim 33:1
Table 1: Isolated per-session savings: tool filtering vs. content compression (MCP attached). Tool filtering is billed every request; content compression is counted once per unique file. Even after multiplying content compression by the turns it rides in history, tool filtering dominates.
TurnCompress inputNo-op inputΔ\Delta
023,27523,27523,27523,2750
634,96834,96836,61736,617−- 1,6491,649
836,61236,61238,74438,744−- 2,1322,132
Table 2: Turn-aligned prefix, content compression only (Sonnet/Claude Code, sparkline task). The per-turn saving is the compressed file slice ( ∼\sim 2K), all cache-read.
Turn NNCompress in.No-op in.Saved (turn)Saved (cum.)
172,04172,04175,50775,5073,4663,4663,4663,466
251,09351,09361,67861,67810,58510,58514,05114,051
353,87753,87771,53871,53817,66117,66131,71231,712
456,68156,68181,38181,38124,70024,70056,41256,412
559,69759,69786,99586,99527,29827,29883,71083,710
cumulative content saving at N=5N{=}5 :83,71083,710 ( 22.2%22.2\% )
Table 3: Five consecutive read-only turns, one session, with tool filtering left on for both arms so the A/B isolates content compression. “Saved” is the measured input difference (No-op −- Compress); its running sum fits ≈3,350 N2\approx 3{,}350\,N^{2} . The tool-filter saving is not in this A/B (on for both arms); it is the fixed ∼\sim 21K/turn established in § 3 .
Figure 1: Measured content-compression savings (markers, from the Table 3 A/B) and fitted scaling laws (curves). The tool filter is a fixed per-turn cut (linear); content compression compounds (quadratic) and overtakes near turn 6, before the ∼\sim 200K context window caps the quadratic around turn 12–15.
Context sourceQuality retainedCompression rate
Uncompressed baseline100.0%100.0%
gpt-4.1-mini (compressor)85.6%50.2%
gpt-5 (compressor)93.6%61.9%
Paritok-4B86.5%25.7%
Table 4: SWE-bench Lite, single-shot: context passed through each compressor, one API call, model emits a unified diff. Quality retained == the compressed arm’s solve quality normalized to the uncompressed baseline, both under the identical single-shot harness.
More in this topic

Context Compression

Context compression reduces the information or memory a model must carry while preserving what it needs to answer questions and act.

See all 90 papers

Explore similar work

CardsList
Jul 13, 2026

Token Reduction Is Not Cost Reduction

Context-reduction layers for API-based coding agents, including command-output compressors, retrieval rankers, and API-boundary proxies, are commonly evaluated by how much context or tool output they remove. We ask a different question: which interventions actually reduce end-to-end billed cost while preserving task success? Our primary evidence is a pre-specified, hash-frozen, paired campaign of 2,908 provider-billed Claude Code runs, of which 2,848 were analyzed, covering 103 tasks, seven repositories, and three models. The campaign compared a baseline with two generations of hook-based compression and an API-boundary proxy within a broader measured program of roughly 5,500 billed executions. Three findings emerge. First, prompt-cache traffic dominated cost composition, accounting for about 87% of reconstructed four-component cost (about 80% of the actual bill), with an 8.7% dollar-weighted residual not attributable from retained telemetry. Second, local payload reduction was not a reliable predictor of end-to-end billed cost. An arm that removed 38% of estimated raw tool-output tokens incurred 6.8% higher paired cost (95% CI: +2.8% to +11.3%), while per-task reduction showed only a weak association with cost change (Pearson r = 0.15). Third, aggressive compression can remove action-critical evidence: on SWE-bench-derived Go tasks, compression reduced successful patch application from 27/40 to 15/40 by corrupting verbatim edit anchors. We propose evaluating context-reduction systems by success-adjusted billed cost rather than token reduction alone.

Sarel Weinberger, Amir Hozez

PointFive

AI Coding AgentsCost-Aware InferenceCode Generation EvaluationContext Compression
AskPDF
Aug 25, 2026

Paritok-4B: Intent-Conditioned Context Compression for Coding Agents

Coding agents re-send large file reads and tool outputs to a frontier LLM every turn, and this context dominates their token bill. General-purpose prompt compressors are trained on prose and suit code poorly: they paraphrase identifiers and drop the exact spans an agent needs to edit. We present Paritok-4B, a 4B LoRA compressor for coding-agent trajectories built on two commitments. It is extractive: it selects spans rather than rewriting them, and 96.0% of the identifiers, paths, and numbers it emits already appear in its input, holding at 96.2% on held-out SWE-bench Lite output. It is intent-conditioned: told the agent's current task, it acts chiefly inside a retained segment, selecting which lines survive (retained lines are +0.067 more intent-relevant than removed ones, paired 95% CI [+0.056, +0.078]) rather than changing how much is retained. We distil a gpt-4.1-mini teacher over 67,074 real OpenHands trajectories into 40,606 validated examples and fine-tune Qwen3-4B. On all 300 SWE-bench Lite instances, Paritok-4B compresses agent context to 25.7% of its size, 2.0x harder than a gpt-4.1-mini compressor (50.2%) and 2.4x harder than gpt-5 (61.9%), while retaining 86.5% of uncompressed single-shot solve quality. Fed the cat -n line-numbered input real agents produce, it compresses slightly less (27.8%) and retains more (89.3%); there the paired test is informative, with 30 instances solved only uncompressed and 17 only compressed, an exact McNemar p=0.079, so at this sample size compressing context to roughly a quarter of its size does not significantly reduce the solve rate. The model is a 264 MB adapter that self-hosts on one 24 GB GPU with no per-token compressor fee, which at list prices decides the economics: gpt-5 as a compressor is net-negative, costing more than the downstream tokens it saves. Weights, data, and evaluation scripts are open (Apache 2.0).

Jiayu Shi, Luzhuo Chen

Paritok

Context Compaction for LLM AgentsContext CompressionLLM CompressionSoftware Engineering Agents
AskPDF
Sep 26, 2026

Beyond Token Savings: A Systematic Study of Context Compression in LLM Agents

As LLM agents tackle longer tasks, they increasingly compress growing histories of reasoning, actions, and tool outputs. Compression can reduce token use, but it also changes the information available for later decisions. Existing agentic harnesses bundle decisions about what to compress, when to compress, and how much to remove into fixed policies. A systematic characterization is needed to disentangle these decisions and reveal how each affects task success and execution cost. We systematically vary these decisions across three open-weight models on SWE-bench Verified and Terminal-Bench 1.0. Across nearly 35,000 agent runs, we measure task success, token use, end-to-end latency, and estimated cost. We find that fewer tokens need not mean faster or cheaper execution: on Terminal-Bench with Qwen, policies using roughly one-third as many tokens can take 20-80% longer than the uncompressed agent. Policies with similar overall success can solve different tasks, while the same policy can perform quite differently across models. Our results motivate evaluating compression by its effects on agent execution and tailoring policies to the task, model, and workload.

Ritul Satish, Prasoon Sinha, Akiho Kawada +1

The University of Texas at Austin

Context Compaction for LLM AgentsContext CompressionLLM Agent HarnessesLLM Agent Evaluation
AskPDF