Prompt Compression

Follow this topic Ask about this topic

Prompt compression studies how to shorten instructions, documents, examples, and agent histories without losing information needed for a task. Methods include selecting salient sentences or tokens, rewriting text into structured forms, using learned encoders or soft tokens, and adapting compression to context structure and task needs. The goal is to lower inference cost, latency, or memory use in settings such as long-context question answering, retrieval, coding agents, and edge devices. A central tension is that fewer input tokens do not always mean lower total cost or reliable answers: omitted evidence, broken references, model-specific behavior, preprocessing overhead, and longer outputs can offset the gains.

Latest papers

35 papers2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Reproducible Evaluation as the Substrate for Safe Prompt Compression in Production Agents
Oct 22, 2026Prabal Vashisht, Zheng Hu, Ying YangPrompt Compression LLM Agent Evaluation
Mx ML, DoorDash Inc.
How Much Prompt Is Enough? A Blackbox Minimization of Few-Shots in LLMs
Sep 28, 2026Ali Alfageeh, Rahul Gopinath, Amin AlipourLLM Prompting Few-Shot Prompting
University of Houston · The University of Sydney
PC-SubMax: Efficient Prompt Compression via Regularized Submodular Maximization
Sep 26, 2026Ziyi Zhang, Shuang Cui, Haotian Zhang +1Prompt Compression Efficient Language Model Inference
Soochow University
Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression
Aug 5, 2026Zhengpei Hu, Kai Li, Dapeng Fu +5LLM Compression Long-Context Inference
School of Computer Technology and Application, Qinghai University · Tsinghua University · Ant Group Security and Intelligence Laboratory (SIL)
Control Under Compression: Reliability Frontiers for Tool-Using Agents
Aug 2, 2026Yinghan Hou, Zongyou YangAI Agent Reliability AI Agent Benchmarks
Department of Electrical and Electronic Engineering Imperial College London London, United Kingdom · Dyson School of Design Engineering Imperial College London London, United Kingdom
Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors
Jul 28, 2026Jianfei Ma, Zhaoxin Feng, Emmanuele Chersoni +1LLM Compression Efficient Language Model Inference
The Hong Kong Polytechnic University
SALT: Salience-Aware Lexical Trie for Long-Context Compression
Jul 20, 2026Oteo Mamo, Hyunjin Yi, Joydhriti Choudhury +2LLM Compression Long-Context Language Model Inference
Florida State University
Cache-Aware Prompt Compression:A Two-Tier Cost Model for LLM API Caching
Jul 17, 2026Yan SongLLM Serving LLM Inference
PayPal Inc.
Prompt Compression via Activation Aggregation
Jul 9, 2026Thibaud Ardoin, Semira Einsele, Evis Bregu +1LLM Inference Acceleration Representation Geometry in Language Models
Freie Universität Berlin
What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents
Jul 9, 2026Ashwin Gerard Colaco, Nada LahjoujiRate-Distortion Theory LLM Memory
University of California, Irvine Irvine, California, USA
CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression
Jun 23, 2026Morayo Danielle Adeyemi, Ryan A. Rossi, Franck DernoncourtLLM Evaluation LLM Compression
Independent · Adobe Research
Safe to Check, Unsafe to Use: Relinking at the Compression Boundary of LLM Agents
Jun 19, 2026Zesen Liu, Zihan Zhang, Dongdong SheAdversarial Attacks on LLMs LLM Agent Security
The Hong Kong University of Science and Technology
SoftSkill: Behavioral Compression for Contextual Adaptation
Jun 18, 2026Xijia Tao, Yihua Teng, Xinyu Fu +6Prompt Tuning LLM Agent Skill Learning
The University of Hong Kong · Huawei Research · City University of Hong Kong +1
Token Factory: Efficiently Integrating Diverse Signals into Large Recommendation Models
Jun 17, 2026Xilun Chen, Shao-Chuan Wang, Baykal Cakici +6Recommender Systems Large Language Model-Based Recommendation
Google · USA
Adaptive Multi-Resolution Procedural Knowledge Compression for Large Language Models
Jun 10, 2026Changyue Wang, Weihang Su, Qingyao Ai +5LLM Compression Efficient Language Model Inference
Department of Computer Science and Technology, Tsinghua University
Closing the Social-Semantic Gap: SPSD for Edge-Based Prompt Compression in Cloud LLM Inference
Jun 10, 2026Abhinit Sen, Ajeet Kumar, Manaranjan PradhanLLM Inference Efficiency On-Device Language Model Inference
Indian School of Business, Hyderabad, India
Cross-Lingual Token Arbitrage: Optimizing Code Agent Context Windows via Local LLM Preprocessing
Jun 2, 2026Mehmet Utku ColakAI Coding Agents LLM Prompting
Department of Computer Engineering, Istanbul Technical University · Istanbul, Turkey
Token Optimization Strategies for LLM-Based Oracle-to-PostgreSQL Migration
May 27, 2026Oleg Grynets, Dmytro Babarytskyi, Vasyl LyashkevychToken Efficiency Prompt Compression
EPAM Systems McLean, Virginia, USA · EPAM Systems Kharkiv, Ukraine · EPAM Systems Lviv, Ukraine
AGORA: Adapter-Grounded Observation-Action Retention for Inference-Free Prompt Compression in LLM Agents
May 26, 2026Haoran Zhang, Zhaohua SunLLM Compression LLM Agents
AI Agent Technologies (Hong Kong) Limited · Department of Mechanical Engineering, The University of Hong Kong
Prompt Compression in Diffusion Large Language Models: Evaluating LLMLingua-2 on LLaDA
May 18, 2026Sterling Huang, Abigayle Brown, Jiyoo Noh +4LLM Evaluation LLM Compression
University of Toronto Toronto, Canada · King Mongkut’s University of Technology Thonburi Bangkok, Thailand
From Token to Token Pair: Efficient Prompt Compression for Large Language Models in Clinical Prediction
May 12, 2026Mingcheng Zhu, Zhiyao Luo, Yu Liu +1LLM Compression Clinical Prediction
Department of Engineering Science, University of Oxford, Oxford, United Kingdom.
Telegraph English: Semantic Prompt Compression via Structured Symbolic Rewriting
May 6, 2026Mikhail L. Arbuzov, Sisong Bei, Ziwei Dong +2LLM Compression Prompt Compression
Independent Researcher · Palo Alto Networks
Mapping Text to Multiplex Graph: Prompt Compression as Lévy Walk-Guided Graph Pruning
May 4, 2026Yaxin Gao, Yao Lu, Jinhong Deng +7LLM Compression Long-Context Modeling
Less is More: Lightweight Prompt Compression for Question Answering Applications on Edge Devices
Apr 27, 2026Zihuai Xu, Ruofei Hou, Yang Xu +3Retrieval-Augmented Generation On-Device Language Model Inference
Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, China, 215123
A Self-Evolving Framework for Efficient Terminal Agents via Observational Context Compression
Apr 21, 2026Jincheng Ren, Siwei Wu, Yizhi Li +8Terminal Agents Long-Horizon Agent Tasks
University of Manchester · 2MAP · 4HKUST(GZ) +2
Compressing Sequences in the Latent Embedding Space: $K$ -Token Merging for Large Language Models
Apr 16, 2026Zihao Xu, John Harvill, Ziwei Fan +3Token Merging LLM Compression
♢Rutgers University · ♡Mistral AI · ♠AWS AI Labs +3
Local-Splitter: A Measurement Study of Seven Tactics for Reducing Cloud LLM Token Usage on Coding-Agent Workloads
Apr 14, 2026Justice Owusu Agyemang, Jerry John Kponyo, Elliot Amponsah +2LLM Agent Workflow Optimization LLM Inference Efficiency
Sperix Labs · VIA Cybersecurity Lab, KNUST · Quantum and Assistive Technologies Lab, KNUST
DiffuMask: Diffusion Language Model for Token-level Prompt Pruning
Apr 8, 2026Caleb Zheng, Jyotika Singh, Fang Tu +6Prompt Compression LLM Context Compression
University of Washington · Oracle AI
Prompt Compression in the Wild: Measuring Latency, Rate Adherence, and Quality for Faster LLM Inference
Apr 3, 2026Cornelius Kummer, Lena Jurkschat, Michael Färber +1Prompt Compression LLM Inference Efficiency
ScaDS.AI Dresden/Leipzig, CIDS, TU Dresden
Compressed-Sensing-Guided, Inference-Aware Structured Reduction for Large Language Models
Mar 22, 2026Andrew KirulutaLLM Inference Acceleration LLM Pruning
UC Berkeley School of Information

Oct 22, 2026J/K move · Enter open · S save

Reproducible Evaluation as the Substrate for Safe Prompt Compression in Production Agents

Prabal Vashisht, Zheng Hu, Ying Yang

Mx ML, DoorDash Inc.

Prompt Compression LLM Agent Evaluation AI Agent Evaluation LLM Agent Harnesses Prompt Optimization

Open paper AskPDF arXiv

Evaluating conversational agents that act on live, mutable enterprise data is hard: replayed sessions drift as backend state changes, and a benchmark built today can silently go stale as the data it describes moves on. We present an offline evaluation harness for a production conversational agent that stays reproducible despite the live data it operates on changing, and that can be regenerated on demand once it goes stale. We then use this harness as the fitness function for GEPA-based prompt optimization under a correctness gate that guards against reward hacking. Applied to a production merchant-facing agent, correctness-gated compression shrinks the agent's full prompt surface by 3.5-7.6% without regressing held-out task correctness, consistently across three independent subject-model families, and for one, correctness measurably improves.