The Compression Paradox in LLM Inference: Provider-Dependent Energy Effects of Prompt Compression

Warren Johnson

cs.CLMar 6, 2026

The Compression Paradox in LLM Inference: Provider-Dependent Energy Effects of Prompt Compression

Authors: Warren Johnson

Organizations: Plexor Labs Principal Researcher

Abstract

The rapid proliferation of Large Language Models has created an environmental paradox: the very technology that could help solve climate challenges is itself becoming a significant contributor to global carbon emissions. We test whether prompt compression improves inference energy efficiency in 28,421 successful API trials (28,428 planned) across three providers (OpenAI GPT-4o-mini, Anthropic Claude-3.5-Sonnet, and DeepSeek-Chat), five benchmarks (HumanEval, MBPP, GSM8K, MATH, MMLU), and four compression ratios (r in {1.0, 0.7, 0.5, 0.3}). Energy is estimated with a token-based proxy calibrated against local direct measurements, and quality is tracked with benchmark pass rates. Compression produced substantial quality loss (overall pass rate 26.0% at baseline vs. 1.5% at r=0.7) and strongly provider-dependent energy behavior. DeepSeek exhibited output expansion under compression (21 to 798 tokens at r=0.3), corresponding to energy increases up to +2,140%, while GPT-4o-mini showed mixed effects including a reduction at r=0.5. These results indicate that input-token reduction alone is not a reliable energy optimization strategy in production inference. For the evaluated settings, model selection and output-length control provided more consistent energy-quality tradeoffs than prompt compression.

Figures & tables

Provider	Est. PUE	Hardware	Efficiency Factor
OpenAI (Azure)	1.20	H100/A100	1.00 (baseline)
Anthropic (GCP)	1.10	TPU v4/v5	0.85
Google (Gemini)	1.09	TPU v5	0.80
Mistral	1.15	H100	0.95
xAI	1.30	H100	1.10
DeepSeek	1.25	H800	1.05

Table 1: Provider-specific energy adjustment factors

Provider/Model	Energy (J)	Cost ($/query)	Overall Pass (%)	Avg Out Tokens
GPT-4o-mini	0.0066	$0.000019	6.97	20.3
DeepSeek-Chat	0.1134	$0.000088	6.87	20.6 (baseline)

Table 2: Proxy-estimated active inference energy per trial in the released matched comparison snapshot

Ratio	n	Pass@1 (%)	Quality Score	$\Delta$ Energy (DeepSeek)	$\Delta$ Energy (GPT)
1.0 (baseline)	4,001	26.0	0.62	N/A	N/A
0.7	3,999	1.5	0.51	+257%	+2.4%
0.5	3,997	0.1	0.37	+753%	–26%
0.3	3,996	0.2	0.29	+2,140%	+22%

Table 3: Effect of prompt compression on energy consumption (N=28,421 trials)

Model	Pass Rate (%)	Energy/Trial (J)	Energy/Success (J)	GAS ${}^{\text{trial}}$	GAS ${}^{\text{success}}$
GPT-4o-mini	6.97	0.006613	0.0949	100.0	100.0
DeepSeek-Chat	6.87	0.114543	1.6667	5.8	5.7

Table 4: Provisional Green AI Scores from collected repository data

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Problems	Avg. Tokens	Task Type
HumanEval	164	180	Code generation
MBPP	500	120	Code generation
GSM8K	1,319 (200 used)	250	Math reasoning
MATH	5,000 (100 used)	350	Math reasoning
MMLU-STEM	3,000 (200 used)	150	Knowledge QA

Appendix

Table 5: Benchmark dataset statistics

LLM Inference Efficiency

Large language model inference efficiency reduces the time, memory, energy, and cost of generating useful outputs.

See all 345 papers

Compression Method Matters: Benchmark-Dependent Output Dynamics in LLM Prompt Compression
Mar 6, 2026Warren JohnsonPrompt Compression LLM Inference Efficiency
Plexor Labs Principal Researcher
Prompt Compression in the Wild: Measuring Latency, Rate Adherence, and Quality for Faster LLM Inference
Apr 3, 2026Cornelius Kummer, Lena Jurkschat, Michael Färber +1Prompt Compression LLM Inference Efficiency
ScaDS.AI Dresden/Leipzig, CIDS, TU Dresden
Towards Green AI: Evaluating Energy Efficiency for Quantized LLM Inference
Oct 13, 2026Petar BastaLLM Inference Efficiency LLM Quantization
University of Waterloo Computer Science Waterloo, Ontario, Canada

Mar 6, 2026J/K move · Enter open · S save

Compression Method Matters: Benchmark-Dependent Output Dynamics in LLM Prompt Compression

Warren Johnson

Plexor Labs Principal Researcher

Prompt Compression LLM Inference Efficiency Efficient Language Model Inference LLM Evaluation

Open paper AskPDF arXiv

Prompt compression is often evaluated by input-token reduction, but its real deployment impact depends on how compression changes output length and total inference cost. We present a controlled replication and extension study of benchmark-dependent output dynamics under aggressive compression, covering 5,400 API calls across three benchmarks and multiple providers. To explain conflicting prior observations, we formalize instruction survival probability (Psi), a structural metric that captures whether task-critical prompt segments remain after truncation. Results show a strong benchmark effect: under r=0.3, DeepSeek exhibits severe output expansion on MBPP (56x, Psi approx 0.15) but substantially lower expansion on HumanEval (5x, Psi approx 0.72), while GPT-4o-mini is comparatively stable across benchmarks. This reconciles the apparent discrepancy between previously reported extreme explosion and lower replication effects by identifying prompt structure, not provider identity alone, as the primary moderator. We introduce the Compression Robustness Index (CRI) for cross-benchmark evaluation and show that single-benchmark assessments can produce misleading conclusions about compression safety and efficiency. To contextualize energy claims, we incorporate companion direct NVML measurements from rented RunPod GPUs and show that token savings can overstate joule savings. These findings motivate benchmark-diverse testing and structure-aware compression policies for reliable, energy-conscious LLM deployment.

Abstract

LLM Inference Efficiency

Explore similar work

Compression Method Matters: Benchmark-Dependent Output Dynamics in LLM Prompt Compression

Prompt Compression in the Wild: Measuring Latency, Rate Adherence, and Quality for Faster LLM Inference

Towards Green AI: Evaluating Energy Efficiency for Quantized LLM Inference