The economics of prompt compression depend not only on reducing input tokens but on how compression changes output length, which is typically priced several times higher. We evaluate this in a pre-registered six-arm randomized controlled trial of prompt compression on production multi-agent task-orchestration, analyzing 358 successful Claude Sonnet 4.5 runs (59-61 per arm) drawn from a randomized corpus of 1,199 real orchestration instructions. We compare an uncompressed control with three uniform retention rates (r=0.8, 0.5, 0.2) and two structure-aware strategies (entropy-adaptive and recency-weighted), measuring total inference cost (input+output) and embedding-based response similarity. Moderate compression (r=0.5) reduced mean total cost by 27.9%, while aggressive compression (r=0.2) increased mean cost by 1.8% despite substantial input reduction, consistent with small mean output expansion (1.03x vs. control) and heavy-tailed uncertainty. Recency-weighted compression achieved 23.5% savings and, together with moderate compression, occupied the empirical cost-similarity Pareto frontier, whereas aggressive compression was dominated on both cost and similarity. These results show that "compress more" is not a reliable production heuristic and that output tokens must be treated as a first-class outcome when designing compression policies.
Figures & tables
Paper
Year
Output Measured
Total Cost
Gap
LLMLingua ( Jiang et al., 2023 )
2023
No
No
Yes
LLMLingua-2 ( Pan et al., 2024 )
2024
No
No
Yes
FrugalGPT ( Chen et al., 2023 )
2023
Formula only
No
Yes
Selective Context ( Li et al., 2023 )
2023
No
No
Yes
RECOMP ( Xu et al., 2023 )
2023
No
No
Yes
Gist Tokens ( Mu et al., 2023 )
2023
No
No
Yes
Table 1: Output token dynamics gap in the prompt compression literature. The overwhelming majority of compression research measures only input token reduction, ignoring output tokens that cost 3–5 × more per token. This systematic omission undermines cost-effectiveness claims.
Task Type
Count
%
Mean Length (chars)
Est. Tokens
Implementation
707
59.0
895
∼ 224
Breakdown
159
13.3
133
∼ 33
Validation
153
12.8
709
∼ 177
Post-orchestration
78
6.5
430
∼ 108
Review
68
5.7
669
∼ 167
Execution
23
1.9
702
∼ 176
Table 2: Corpus composition by task type and length tercile ( N=1,199 ).
Arm
Mean In Tok
Mean Out Tok
Mean Cost ($)
Total Cost ($)
Latency (ms)
Control
59
609
0.0093
0.0466
13,122
Light
49
811
0.0123
0.0616
16,285
Moderate
41
613
0.0093
0.0466
13,671
Aggressive
30
161
0.0025
0.0125
4,124
Adaptive
46
420
0.0064
0.0322
9,370
Recency
41
504
0.0077
0.0384
11,583
Table 3: Pilot run summary statistics ( n=30 ; 5 trials per arm).
Figure 1: CONSORT flow diagram. Enrollment: 1,577 records → 1,337 after exclusions → 1,199 after deduplication and randomization. All 1,199 trials were submitted to the API; 841 failed after retries (primarily credit-balance errors), leaving a complete-case inferential set of 358 successful trials (59–61 per arm).
Metric
Full Randomized ( N=1,199 )
Complete-Case ( N=358 )
Mean input tokens (original)
179.6
76.0
Median input tokens (original)
126.0
78.5
Implementation tasks, n (%)
707 (59.0%)
176 (49.2%)
Breakdown tasks, n (%)
159 (13.3%)
159 (44.4%)
Execution tasks, n (%)
23 (1.9%)
23 (6.4%)
All other task types, n (%)
310 (25.9%)
0 (0.0%)
Table 4: Full randomized set vs complete-case successful-response set.
Arm
Assigned
Successful
Mean Cost ($)
Successes/$
Cost Reduction vs Control
Control
197
59
0.004682
64.0
—
Light
199
60
0.003992
75.5
14.7%
Moderate
201
60
0.003256
91.7
30.5%
Aggressive
202
61
0.004419
68.3
5.6%
Adaptive
199
59
0.003878
76.4
17.2%
Recency
201
59
0.003389
86.6
27.6%
Table 5: Assignment-level deployment sensitivity across all randomized submissions.
Prompt compression is often evaluated by input-token reduction, but its real deployment impact depends on how compression changes output length and total inference cost. We present a controlled replication and extension study of benchmark-dependent output dynamics under aggressive compression, covering 5,400 API calls across three benchmarks and multiple providers. To explain conflicting prior observations, we formalize instruction survival probability (Psi), a structural metric that captures whether task-critical prompt segments remain after truncation. Results show a strong benchmark effect: under r=0.3, DeepSeek exhibits severe output expansion on MBPP (56x, Psi approx 0.15) but substantially lower expansion on HumanEval (5x, Psi approx 0.72), while GPT-4o-mini is comparatively stable across benchmarks. This reconciles the apparent discrepancy between previously reported extreme explosion and lower replication effects by identifying prompt structure, not provider identity alone, as the primary moderator. We introduce the Compression Robustness Index (CRI) for cross-benchmark evaluation and show that single-benchmark assessments can produce misleading conclusions about compression safety and efficiency. To contextualize energy claims, we incorporate companion direct NVML measurements from rented RunPod GPUs and show that token savings can overstate joule savings. These findings motivate benchmark-diverse testing and structure-aware compression policies for reliable, energy-conscious LLM deployment.
As LLM agents tackle longer tasks, they increasingly compress growing histories of reasoning, actions, and tool outputs. Compression can reduce token use, but it also changes the information available for later decisions. Existing agentic harnesses bundle decisions about what to compress, when to compress, and how much to remove into fixed policies. A systematic characterization is needed to disentangle these decisions and reveal how each affects task success and execution cost. We systematically vary these decisions across three open-weight models on SWE-bench Verified and Terminal-Bench 1.0. Across nearly 35,000 agent runs, we measure task success, token use, end-to-end latency, and estimated cost. We find that fewer tokens need not mean faster or cheaper execution: on Terminal-Bench with Qwen, policies using roughly one-third as many tokens can take 20-80% longer than the uncompressed agent. Policies with similar overall success can solve different tasks, while the same policy can perform quite differently across models. Our results motivate evaluating compression by its effects on agent execution and tailoring policies to the task, model, and workload.
Context-reduction layers for API-based coding agents, including command-output compressors, retrieval rankers, and API-boundary proxies, are commonly evaluated by how much context or tool output they remove. We ask a different question: which interventions actually reduce end-to-end billed cost while preserving task success? Our primary evidence is a pre-specified, hash-frozen, paired campaign of 2,908 provider-billed Claude Code runs, of which 2,848 were analyzed, covering 103 tasks, seven repositories, and three models. The campaign compared a baseline with two generations of hook-based compression and an API-boundary proxy within a broader measured program of roughly 5,500 billed executions. Three findings emerge. First, prompt-cache traffic dominated cost composition, accounting for about 87% of reconstructed four-component cost (about 80% of the actual bill), with an 8.7% dollar-weighted residual not attributable from retained telemetry. Second, local payload reduction was not a reliable predictor of end-to-end billed cost. An arm that removed 38% of estimated raw tool-output tokens incurred 6.8% higher paired cost (95% CI: +2.8% to +11.3%), while per-task reduction showed only a weak association with cost change (Pearson r = 0.15). Third, aggressive compression can remove action-critical evidence: on SWE-bench-derived Go tasks, compression reduced successful patch application from 27/40 to 15/40 by corrupting verbatim edit anchors. We propose evaluating context-reduction systems by success-adjusted billed cost rather than token reduction alone.