Context Compression

Follow this topic Ask about this topic

Context compression studies how to shrink long prompts, interaction histories, retrieved documents, or model memory without losing information needed for later tasks. Approaches include selecting or pruning text, rewriting it as summaries or structured records, encoding it into learned latent tokens, and adaptively retaining or retrieving history as an agent works. Methods are tested on long-context question answering, retrieval-augmented generation, coding and web agents, and long-horizon tool use, where accuracy, task success, latency, memory, and cost can all matter. The central tension is that aggressive compression can discard critical details or constraints, and fewer tokens do not always mean lower end-to-end cost or more reliable execution.

Latest papers

90 papers11 papers in the last four weeks, down 45% on the four weeks before. 0.1% of all new papers.

Decision-Aware Memory Cards: Counterfactual-Inspired Context Selection and Compression for Tool-Using LLM Agents
Jun 6, 2026Xinyu Guan, Qianyang Zhao, Yuming DengLLM Agent Memory LLM Context Management
Alibaba Group, China
Signal-Driven Observation for Long-Horizon Web Agents
Jun 4, 2026Shubham Gaur, Ian LaneActive Perception Long-Horizon LLM Agents
Department of Computer Science and Engineering, University of California, Santa Cruz, CA, USA.
From Long News to Accurate Forecast: Importance-Aware Fusion and PRM-Guided Reflection for Time Series Forecasting
Jun 2, 2026Mingyang Liu, Qingcan Kang, Yuke Wang +6Process Reward Models Financial Forecasting
Department of Computer Science, City University of Hong Kong · Work done as an intern in Huawei Noah’s Ark Lab. · Huawei Noah’s Ark Lab
Beyond Recall: Behavioral Specification as an Interpretive Layer for AI Personalization
May 27, 2026Aarik GulayaLLM Personalization AI Alignment
Base Layer
Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor
May 27, 2026Guoxin Ma, Yibing Liu, Chengzhengxu Li +7LLM Compression Long-Context Inference
Baidu Inc., Beijing, China · Xi’an Jiaotong University · City University of Hong Kong +1
Neural Router: Semantic Content Matching for Agentic AI
May 25, 2026Lauri Lovén, Abhishek Kumar, Alexander Engelhardt +5Semantic Textual Similarity LLM Inference Acceleration
Future Computing Group, University of Oulu, Finland · University of Jyväskylä, Finland · Department of Computer Science, University of Helsinki, Finland +1
SemanticZip: A Pilot Framework for Lossy Text Compression with LLMs as Semantic Decompressors
May 23, 2026Natalia Trukhina, Vadim VashkelisContext Compression
Embedded Intelligence Lab (EMILAB)
On Problems of Implicit Context Compression for Software Engineering Agents
May 11, 2026Kirill Gelvan, Igor Slinko, Felix Steinbauer +3Software Engineering Agents Long-Horizon LLM Agents
JetBrains Research · Technical University of Munich, Germany
Beyond Position Bias: Shifting Context Compression from Position-Driven to Semantic-Driven
May 10, 2026Jiwei Tang, Zhijing Huang, Xinyu Zhang +5LLM Compression Position Bias
Sun Yat-sen University · Hong Kong Polytechnic University · Beijing Normal–Hong Kong Baptist University
GRC: Unifying Reasoning-Driven Generation, Retrieval and Compression
May 9, 2026Zhongtao Miao, Qiyu Wu, Yoshimasa TsuruokaRetrieval-Augmented Generation Memory-Augmented Language Models
The University of Tokyo · Tsuruoka Lab. · Independent Researcher
MiA-Signature: Approximating Global Activation for Long-Context Understanding
May 7, 2026Yuqing Li, Jiangnan Li, Mo Yu +3Long-Context Modeling Context Compression
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · Pattern Recognition Center, WeChat AI, Tencent +1
A11y-Compressor: A Framework for Enhancing the Efficiency of GUI Agent Observations through Visual Context Reconstruction and Redundancy Reduction
May 1, 2026Michito Takeshita, Takuro Kawada, Takumi Ohashi +2GUI Agents Context Compression
Hosei University, Tokyo, Japan
Kwai Summary Attention Technical Report
Apr 27, 2026Chenglong Chu, Guorui Zhou, Guowang Zhang +35Self-Attention KV Caching
Kuaishou
ResRank: Unifying Retrieval and Listwise Reranking via End-to-End Joint Training with Residual Passage Compression
Apr 24, 2026Xiaojie Ke, Shuai Zhang, Liansheng Sun +6Learning to Rank LLM Reranking
Qwen Applications Business Group of Alibaba
A Self-Evolving Framework for Efficient Terminal Agents via Observational Context Compression
Apr 21, 2026Jincheng Ren, Siwei Wu, Yizhi Li +8Terminal Agents Long-Horizon Agent Tasks
University of Manchester · 2MAP · 4HKUST(GZ) +2
Transforming External Knowledge into Triplets for Enhanced Retrieval in RAG of LLMs
Apr 14, 2026Xudong Wang, Chaoning Zhang, Qigan Sun +7Structured RAG Retrieval-Augmented Generation
School of Computing, Kyung Hee University, Yongin-si, South Korea · School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China · School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China +2
Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learning
Apr 13, 2026Xiaozhe Li, Tianyi Lyu, Yizhao Yang +6Lost-in-the-Middle Effect LLM Context Management
Tongji University · CurrentsAI Research · Stanford University +1
HYVE: Hybrid Views for LLM Context Engineering over Machine Data
Apr 7, 2026Jian Tan, Fan Bu, Yuqing Gao +5LLM Context Management LLM Inference Efficiency
Cisco Systems, Inc., San Jose, California, USA
ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents
Apr 2, 2026Yong Wu, YanZhao Zheng, TianZe Xu +9LLM Context Management Long-Horizon LLM Agents
Zhejiang University, Hangzhou, China · Alibaba Group, Hangzhou, China
Developing Adaptive Context Compression Techniques for Large Language Models (LLMs) in Long-Running Interactions
Mar 31, 2026Payal Fofadiya, Sunil TiwariLLM Context Compression LLM Context Management
LinkedIn · eBay
Density-aware Soft Context Compression with Semi-Dynamic Compression Ratio
Mar 26, 2026Yijiong Yu, Shuai Yuan, Jie Zheng +2LLM Context Compression Context Compression
Oregon State University · DeepSolution
PoC: Performance-oriented Context Compression for Large Language Models via Performance Prediction
Mar 20, 2026Runsong Zhao, Shilei Liu, Jiwei Tang +8Language Model Performance Prediction LLM Context Compression
Northeastern University, China · Future Living Lab of Alibaba · Tsinghua University
GradMem: Learning to Write Context into Memory with Test-Time Gradient Descent
Mar 14, 2026Yuri Kuratov, Matvey Kairov, Aydar Bulatov +2Memory-Augmented Language Models Context Compression
AXXX, Cognitive AI Systems Lab, Moscow, Russia · MIRAI, Moscow, Russia · MBZUAI, Abu Dhabi, UAE +1
EnComp: Lightweight Encoder-Only Context Compression for Retrieval-Augmented Question Answering
Mar 10, 2026Thao Do, Dinh Phu Tran, An Vo +2Efficient Language Model Inference Knowledge-Intensive QA
Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, South Korea
Stacked from One: Multi-Scale Self-Injection for Context Window Extension
Mar 5, 2026Wei Han, Pan Zhou, Soujanya Poria +1Long-Context Language Model Inference Long-Context Language Modeling
Singapore University of Technology and Design (SUTD), Singapore · Singapore Management University (SMU), Singapore · Nanyang Technological University (NTU), Singapore +1
Do LLMs Benefit From Their Own Words?
Feb 27, 2026Jenny Y. Huang, Leshem Choshen, Wei Sun +5LLM Context Management Context Compression
Department of Electrical Engineering and Computer Science, MIT · MIT–IBM Watson AI Lab · IBM Research
Contextual Memory Virtualisation: DAG-Based State Management and Structurally Lossless Trimming for LLM Agents
Feb 25, 2026Cosmo SantoniLLM Context Management Agent Memory Management
Imperial College London
BudgetMem: Training-Free Selective Memory for Cost-Efficient Long-Context Processing in Language Models
Nov 7, 2025Chandra Vamsi Krishna Alla, Harish Naidu Gaddam, Manohar Kommi +1Memory-Efficient Optimization LLM Compression
University of Texas at Arlington
CompLLM: Compression for Long Context Q&A
Sep 23, 2025Gabriele Berton, Jayakrishnan Unnikrishnan, Son Tran +1LLM Compression Long-Context Language Model Inference
Amazon · Center For Research in Computer Vision, University of Central Florida
InComeS: Integrating Compression and Selection Mechanisms into LLMs for Efficient Model Editing
May 28, 2025Shuaiyi Li, Zhisong Zhang, Yang Deng +6In-Context Learning Model Editing
The Chinese University of Hong Kong · City University of Hong Kong · Singapore Management University +1

Jun 6, 2026J/K move · Enter open · S save

Decision-Aware Memory Cards: Counterfactual-Inspired Context Selection and Compression for Tool-Using LLM Agents

Xinyu Guan, Qianyang Zhao, Yuming Deng

Alibaba Group, China

LLM Agent Memory LLM Context Management Context Compression

Open paper AskPDF arXiv

Modern large language model (LLM) agents do not simply need longer contexts; they need decision-relevant evidence at the moment of action. We study decision-aware context selection: ranking retrieved files, tests, traces, rules, and memories by their expected effect on an agent's next action rather than by semantic similarity alone. We present the Counterfactual-Inspired Context Layer (CICL), which builds an instance context graph, estimates decision-oriented utility for candidate units, and compresses selected evidence into typed memory cards. The same schema can be instantiated with hosted LLM judges, local surrogates, or lightweight rankers, making the selection protocol auditable across model choices. On 50 SWE-bench Verified file-retrieval instances, Qwen3.6-Plus reranking of BM25 top-50 candidates improves hit@1 from 0.58 to 0.78 and MRR@10 from 0.634 to 0.790, with all 2,500 judgments parseable. Controlled diagnostics show that CICL identifies action-critical evidence: removing the top-utility semantic unit reduces F1 from 0.245 to 0.000. In selected-then-compressed mode, memory cards save 44.93 tokens per query while preserving selected evidence. CICL provides a practical layer for measuring, ranking, and compressing decision-critical context for tool-using agents. Code is available at https://github.com/stephen-guan-researcher/CICL.