LLM Context Compression

LLM: Large Language Model

Follow this topic Ask about this topic

Latest papers

37 papers4 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Latent Compaction with Test-Time Training
Oct 14, 2026Yingfa Chen, Xingyu Shen, Chaojun Xiao +1LLM Context Compression KV-Cache Compression
NLP Group, DCST, IAI, BNRIST, Tsinghua University, Beijing, China
SemanticFold: Latent Sequence Compression SeparatesLanguage Modeling, Decodability, and Reasoning
Oct 7, 2026Mingyan Liu, Min HuangLLM Context Compression LLM Compression
Highlight-Then-Summarize: Learning to Compress Evidence for Long-Context Understanding
Sep 25, 2026Zhaoyuan Xia, Qinghongbing Xie, Yung Xiang Hue +7Long-Context Reasoning in Language Models Long-Context Modeling
Peking University · Baidu Inc · Baidu Inc. +1
Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference
Sep 22, 2026Md Mostafizer Rahman, Md Faizul Ibne Amin, Md Shahajada Mia +2LLM Inference Efficiency Long-Context Language Model Inference
Lucy Family Institute for Data & Society, University of Notre Dame, IN, USA · The University of Aizu, Aizuwakamatsu, Japan · Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, IN, USA
Machine-Interpretable Information: Compiling Documents into Searchable and Readable Protocol States
Sep 20, 2026Yifan Wang, Dejing DouLLM Context Compression Memory-Augmented Language Models
NexusLumenLabs · Fudan University
FlexComp: One Model for Every Ratio in Context Compression
Sep 10, 2026Kaiyan Zhao, Zhongtao Miao, Akiko Aizawa +1LLM Inference Acceleration Context Compression
The University of Tokyo · National Institute of Informatics
Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAG
Sep 4, 2026Shuyu Guo, Shuo Zhang, Zhaochun RenRetrieval-Augmented Generation LLM Inference Acceleration
Shandong University Qingdao, China · Bloomberg London, United Kingdom · Leiden University Leiden, The Netherlands
LatentPress: Context Compression Beyond Text and Vision
Sep 1, 2026Yuanda Xu, Zhengze Zhou, Hejian Sang +3LLM Inference Acceleration LLM Context Compression
Iowa State University · Cornell University
TopoCompress: Long Context Compression via Graph-Wired Semantic Trajectories
Aug 31, 2026Daniel Agyei Asante, Yang LiLLM Compression Long-Context QA
1Iowa State University, United States
SeDeM: Selective Decompression of Hidden-State Memories for Long-Context Question Answering
Jul 31, 2026Maryam Haghifam, Jason Cong, Yizhou SunMemory-Augmented Language Models Long-Context QA
University of California, Los Angeles, USA
PReM: Learning What to Preserve and When to Refresh for Context Compression
Jul 15, 2026Bohan Yu, Lei Shen, Chenxi Zhou +5LLM Inference Acceleration Long-Context Language Model Inference
Alibaba Group
Fictional Worldbuilding: Multi-Agent LLM Collaboration with Hierarchical Context Compression and Iterative Review
Jul 10, 2026Jingbo Chen, He Wang, Wei Yuan +2Multi-Agent LLM Systems LLM Agent Orchestration
National University of Defense Technology, China
Prompt Compression via Activation Aggregation
Jul 9, 2026Thibaud Ardoin, Semira Einsele, Evis Bregu +1LLM Inference Acceleration Representation Geometry in Language Models
Freie Universität Berlin
MemRefine: LLM-Guided Compression for Long-Term Agent Memory
Jun 11, 2026Minjae Kim, Jinheon Baek, Soyeong Jeong +1Persistent Memory for Language Models LLM Context Compression
Korea University · KAIST
Context-Driven Incremental Compression for Multi-Turn Dialogue Generation
Jun 10, 2026Yeongseo Jung, Jaehyeok Kim, Eunseo Jung +5Memory-Augmented Language Models Long-Context Modeling
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong · NVIDIA AI Technology Center, NVIDIA · University of Tsukuba, Japan +1
End-to-End Context Compression at Scale
Jun 8, 2026Ang Li, Sean McLeish, Haozhe Chen +12LLM Compression Long-Context Language Model Inference
New York University · University of Maryland · Princeton University +3
Still: Amortized KV Cache Compaction in a Single Forward Pass
Jun 5, 2026Charles O'Neill, Alex Sandomirsky, Harry Partridge +2KV Caching Long-Context Language Model Inference
Baseten San Francisco · Baseten London
Entropy Gate: Entropy Quenching for Near-Lossless Token Compression in LLM Pipelines
Jun 2, 2026Justice Owusu Agyemang, Jerry John Kponyo, Kwame Opuni-Boachie Obour Agyekum +3LLM Compression Token Pruning
1VIA Cybersecurity Lab, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana · 2Quantum and Assistive Technologies Lab, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana
LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning
May 31, 2026Mengmeng Ji, Ravi Shanker Raju, Jonathan Lingjie Li +1LLM Compression LLM Fine-Tuning
SambaNova Systems, Inc. San Jose, CA, USA
Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor
May 27, 2026Guoxin Ma, Yibing Liu, Chengzhengxu Li +7LLM Compression Long-Context Inference
Baidu Inc., Beijing, China · Xi’an Jiaotong University · City University of Hong Kong +1
ZipRL: Adaptive Multi-Turn Context Compression with Hindsight Response Replay
May 27, 2026Zhexin Hu, Li Wang, Xiaohan Wang +4LLM Compression Reinforcement Learning with Verifiable Rewards
1Meituan · Institute of Software, Chinese Academy of Sciences
Compress the Context, Keep the Commitments: A Formal Framework for Verifiable LLM Context Compression
May 17, 2026Natalia Trukhina, Vadim VashkelisLLM Compression LLM Context Compression
Embedded Intelligence Lab (EMILAB)
Telegraph English: Semantic Prompt Compression via Structured Symbolic Rewriting
May 6, 2026Mikhail L. Arbuzov, Sisong Bei, Ziwei Dong +2LLM Compression Prompt Compression
Independent Researcher · Palo Alto Networks
From Similarity to Structure: Training-free LLM Context Compression with Hybrid Graph Priors
Apr 25, 2026Yitian Zhou, Chaoning Zhang, Jiaquan Zhang +6LLM Compression Long-Context Language Model Inference
School of Computer Science and Engineering University of Electronic Science and Technology of China · School of Information and Software Engineering, University of Electronic Science and Technology of China · Department of Computer Science and Engineering, Kyung Hee University +3
Compressing Sequences in the Latent Embedding Space: $K$ -Token Merging for Large Language Models
Apr 16, 2026Zihao Xu, John Harvill, Ziwei Fan +3Token Merging LLM Compression
♢Rutgers University · ♡Mistral AI · ♠AWS AI Labs +3
MemoSight: Unifying Context Compression and Multi Token Prediction for Reasoning Acceleration
Apr 16, 2026Xinyu Liu, Xin Liu, Bo Jin +8Multi-Token Prediction KV Caching
School of Computer Science and Engineering, Northeastern University, China · Meituan Inc. · NiuTrans Research, Shenyang, China
DiffuMask: Diffusion Language Model for Token-level Prompt Pruning
Apr 8, 2026Caleb Zheng, Jyotika Singh, Fang Tu +6Prompt Compression LLM Context Compression
University of Washington · Oracle AI
Developing Adaptive Context Compression Techniques for Large Language Models (LLMs) in Long-Running Interactions
Mar 31, 2026Payal Fofadiya, Sunil TiwariLLM Context Compression LLM Context Management
LinkedIn · eBay
Density-aware Soft Context Compression with Semi-Dynamic Compression Ratio
Mar 26, 2026Yijiong Yu, Shuai Yuan, Jie Zheng +2LLM Context Compression Context Compression
Oregon State University · DeepSolution
Large Language Model as Token Compressor and Decompressor
Mar 26, 2026Wenbing Li, Yiran Wang, Zikai Song +4LLM Context Compression LLM Compression

Oct 14, 2026J/K move · Enter open · S save

Latent Compaction with Test-Time Training

Yingfa Chen, Xingyu Shen, Chaojun Xiao, Xu Han

NLP Group, DCST, IAI, BNRIST, Tsinghua University, Beijing, China

LLM Context Compression KV-Cache Compression Memory Compression Long-Context Retrieval Test-Time Training

Open paper AskPDF arXiv

Long contexts enlarge key--value (KV) caches, increasing memory use and attention costs during generation. Compressing these caches must preserve information for unseen questions. We introduce Latent Compaction, which learns a compact document-specific cache through test-time training. We initialize cache entries using reconstruction importance, teacher attention, and key-space diversity. Training first fits attention outputs and log-normalizers to those of the full cache, with reconstruction-selected entries held fixed. It then jointly optimizes the entire cache through end-to-end distillation from the full-cache model. Model weights remain frozen throughout, and training uses only generic prompts about the document, without access to downstream questions or answers. Experiments on RULER, LongBench, and Phonebook Lookup demonstrate substantial gains over budget-matched compression baselines. On a RULER, for example, our method achieves 96.7% accuracy at 32

\times

compression, compared with 40.0% for KVZip+. The method also transfers across model scales and families, with retrieval gains on both Qwen3 and MiniCPM5. These results show that treating the KV cache as a neural network and optimizing it with test-time training can preserve retrieval accuracy under aggressive compression. The code is released at https://www.github.com/THUNLP/latent-compaction