LLM Context Compression

LLM: Large Language Model

Follow this topic Ask about this topic

Latest papers

62 papers18 papers in the last four weeks, up 350% on the four weeks before. 0.1% of all new papers.

QRAE: Query-Relevant Attention Entropy for Context Compression in DeepSearch Agents
Nov 5, 2026Yingjie Tan, Haiquan Qiu, Quanming Yao +3LLM Context Compression Context Compaction for LLM Agents
Tsinghua University · Huawei Technologies Ltd.
SpecCompact: Accelerating LLM Agent Context Compaction with Slot-Parallel Semantic Speculative Verification
Nov 4, 2026Yulu An, Yifei Feng, Xiangyang YinContext Compaction for LLM Agents LLM Context Compression
New York University
Tools Need Not Speak in Text: Latent Context Compression for Agents
Nov 4, 2026Yuetong Liu, Zhehao Zhou, Yitong Zhang +2Context Compaction for LLM Agents LLM Context Compression
Tsinghua AI Agent Group, Tsinghua University
SUMONE: A Simple and Efficient Summarizer for Plug-and-Play Agent Memory
Nov 4, 2026Jizhe Zhang, Ang Li, Chenheng Zhang +1LLM Context Compression Context Compaction for LLM Agents
State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University
Does Context Compression Preserve Refusal Alignment?
Nov 4, 2026Anthony Nguyen, Wenjun LinLLM Context Compression Refusal Behavior in Language Models
Digital Innovation Lab, Faculty of Computer Science and Technology Algoma University, ON, Canada
You don't need long context
Nov 3, 2026Peter Belcak, Saurav Muralidharan, Jindong Jiang +9Long-Context Reasoning in Language Models Long-Context Language Model Inference
ACCT: Asymmetric Cross-LLM Context Transfer
Oct 30, 2026Yuzhe Liu, Haolin Tian, Tonghan WangKV-Cache Reuse LLM Inference Acceleration
Tsinghua University, Beijing, China
In-Model Context Compression with Agentic Soft Tokens for Long-Horizon LLM Agents
Oct 30, 2026Xuning He, Kai Fan, Runyu Zhang +4Context Compaction for LLM Agents LLM Context Compression
Shanghai Jiao Tong University · Nanyang Technological University · Nankai University +2
Kernel-based Interpolatory Methods for Context Compression of Attention Mechanisms
Oct 29, 2026Michel Fabrice Serret, Laura GrigoriLLM Context Compression KV-Cache Compression
Center for Scientific Computing, Theory and Data, Paul Scherrer Institute, Switzerland. · Institute of Mathematics, EPFL, Switzerland
Merge and Branch: Efficient Tree-Structured Supervised Fine-Tuning of LLMs for Recommendation
Oct 27, 2026Yunqin Zhu, Liyang Shu, Yusi Zhang +4Large Language Model-Based Recommendation LLM Fine-Tuning
Georgia Institute of Technology, Atlanta, GA, USA · Work completed during an internship at TikTok. · TikTok, Bellevue, WA, USA +3
Balanced State Subspaces for Mamba-2 State Compression
Oct 27, 2026Shang Tingyu, Cao WenfeiLLM Compression State Space Language Models
School of Mathematics and Statistics, Shaanxi Normal University, Xi’an 710119, China
CoLink: Global Compression and Knowledge Linking for Multi-Turn Dialogue Modeling
Oct 25, 2026Lian Yan, Zhenbo Xia, Haotian Wang +4Language Modeling LLM Context Compression
Research Center of Language Technology, Faculty of Computing, Harbin Institute of Technology, Harbin, Heilongjiang 150001, China · Institute for Advanced Algorithms Research, Shanghai, 201306, China · Research Center of Artificial Intelligence of Things, Faculty of Computing, Harbin Institute of Technology, Harbin, Heilongjiang 150001, China, and also with the National Key Laboratory of Smart Farm Technologies and Systems, Harbin, Heilongjiang 150001, China
Rethinking Retrieval in the Long-Context Era: Training-Free Semantic Trees for Adaptive Context Construction
Oct 25, 2026Zhe Xu, Tianhao Liao, Zhenqing Ling +3Long-Context Retrieval Knowledge Retrieval in Language Models
Sun Yat-sen University · Alibaba Group
Benchmarks Are Surprisingly Compaction-Friendly
Oct 22, 2026Jeanne Salle, Hardik Bhatnagar, Matthias Bethge +2Context Compaction for LLM Agents LLM Context Compression
ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems · Tübingen AI Center +1
Rethinking Sequence Compression with Test-time Regression
Oct 22, 2026Sihan Yang, Jie Huang, Hanxuan Li +4LLM Context Compression Multimodal Token Compression
The Chinese University of Hong Kong · Independent Researcher · Token Foundry, Alibaba Group
Compact-Bench: Measuring What Context Compaction Costs Long-Horizon Coding Agents
Oct 22, 2026Ruopei Sun, Jianfeng Cai, Kangwen Zhao +2Context Compaction for LLM Agents LLM Context Compression
University of Science and Technology of China
Evaluating Context Compaction on the Questions That Come Later
Oct 19, 2026Junwei Yao, Yongxi Zhou, Zike Yang +4Context Compaction for LLM Agents LLM Context Compression
Independent Researcher
LatComm: Reusable Context Compression for Latent Communication between Large Language Models
Oct 19, 2026Huawei Lin, Jinting Liu, Tan Yu +1LLM Context Compression LLM Inference Acceleration
Rochester Institute of Technology · NVIDIA
KEEP-Bench: Evaluating Evidence Preservation in Context Compression
Oct 19, 2026Ziqing Man, Di Wu, Jia-Chen Gu +3LLM Context Compression LLM Evaluation
University of California, Los Angeles
TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning
Oct 19, 2026Liana Patel, Rishabh Ranjan, Ion Stoica +2Token Pruning LLM Inference Acceleration
Stanford University · UC Berkeley
Bud ContextCompact 2.0: Context Compaction for Agent Success and Accuracy with Small Language Models and Constrained Context Windows
Oct 18, 2026Jithin VG, Ditto PSLLM Context Management LLM Context Compression
Bud Ecosystem
Cache2LoRA: Understanding and Improving Context Internalization
Oct 16, 2026Yi Fang, Gaurav Srivastava, Zhenyu Bi +2LLM Context Compression Knowledge Augmentation for Language Models
Department of Computer Science, Virginia Tech, Blacksburg, VA, USA · Department of Biomedical Data Science, Stanford University, Stanford, CA, USA
Latent Compaction with Test-Time Training
Oct 14, 2026Yingfa Chen, Xingyu Shen, Chaojun Xiao +1LLM Context Compression KV-Cache Compression
NLP Group, DCST, IAI, BNRIST, Tsinghua University, Beijing, China
SemanticFold: Latent Sequence Compression SeparatesLanguage Modeling, Decodability, and Reasoning
Oct 7, 2026Mingyan Liu, Min HuangLLM Context Compression LLM Compression
Highlight-Then-Summarize: Learning to Compress Evidence for Long-Context Understanding
Sep 25, 2026Zhaoyuan Xia, Qinghongbing Xie, Yung Xiang Hue +7Query-Focused Summarization Long-Context Reasoning in Language Models
Peking University · Baidu Inc · Baidu Inc. +1
Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference
Sep 22, 2026Md Mostafizer Rahman, Md Faizul Ibne Amin, Md Shahajada Mia +2LLM Inference Efficiency Long-Context Language Model Inference
Lucy Family Institute for Data & Society, University of Notre Dame, IN, USA · The University of Aizu, Aizuwakamatsu, Japan · Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, IN, USA
Machine-Interpretable Information: Compiling Documents into Searchable and Readable Protocol States
Sep 20, 2026Yifan Wang, Dejing DouLLM Context Compression Memory-Augmented Language Models
NexusLumenLabs · Fudan University
FlexComp: One Model for Every Ratio in Context Compression
Sep 10, 2026Kaiyan Zhao, Zhongtao Miao, Akiko Aizawa +1LLM Inference Acceleration Context Compression
The University of Tokyo · National Institute of Informatics
Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAG
Sep 4, 2026Shuyu Guo, Shuo Zhang, Zhaochun RenRL for RAG LLM Inference Acceleration
Shandong University Qingdao, China · Bloomberg London, United Kingdom · Leiden University Leiden, The Netherlands
LatentPress: Context Compression Beyond Text and Vision
Sep 1, 2026Yuanda Xu, Zhengze Zhou, Hejian Sang +3LLM Inference Acceleration LLM Context Compression
Iowa State University · Cornell University

Nov 5, 2026J/K move · Enter open · S save

QRAE: Query-Relevant Attention Entropy for Context Compression in DeepSearch Agents

Yingjie Tan, Haiquan Qiu, Quanming Yao, Manyi Zhong+2

Tsinghua University · Huawei Technologies Ltd.

LLM Context Compression Context Compaction for LLM Agents RAG Context Compression

Open paper AskPDF arXiv

DeepSearch LLM agents repeatedly search and carry retrieved sources into subsequent reasoning steps. This expanding context increases token consumption and latency, although many results do not contribute to later reasoning. We propose Query-Relevant Attention Entropy (QRAE), an online context compression method that uses sparse evidence-assessment heads to decide which intact retrieved chunks should persist across reasoning steps. It scores bounded candidate batches through paired prefills: a query-free control suppresses shared attention responses, while normalized entropy measures the breadth of the positive response within each chunk. High-scoring chunks enter a persistent evidence whitelist and are included in the reasoning context. Uncertain chunks remain outside the reasoning context but are retained in a candidate cache, where they can be retrieved for later reevaluation. QRAE requires neither an additional generative compression pass nor modification of the attention operator. Across multiple DeepSearch benchmarks and reasoning models, QRAE improves task success by an average of 2.35 percentage points and reduces end-to-end latency and reported token usage by an average of 30% and 45%, respectively. These benefits generalize to a model with nearly 100B parameters.