Overview

Key-value cache compression reduces the memory and bandwidth cost of storing attention states, enabling longer contexts, larger batches, and inference on constrained hardware. Common approaches include token eviction and merging, quantization, low-rank or latent representations, and adaptive per-layer or per-head budgets; some methods also reconstruct or retrieve compressed states on demand. Work is evaluated on long-context retrieval and question answering benchmarks such as LongBench, RULER, and Needle-in-a-Haystack, as well as reasoning, multimodal tasks, and end-to-end serving. The central tension is that aggressive compression can harm accuracy, evidence use, or safety, while a high compression ratio does not guarantee faster inference or lower peak memory, so workload-aware evaluation remains essential.

Latest papers

CardsList

293 papers30 papers in the last four weeks, down 48% on the four weeks before. 0.2% of all new papers.

  1. VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization

    Mar 17, 2026Yixuan Wang, Qingyu Shi, Jiayu Zhou +3KV-Cache CompressionKV-Cache Quantization

  2. SemantiCache: Efficient KV Cache Compression via Semantic Chunking and Clustered Merging

    Mar 15, 2026Shunlong Wu, Hai Lin, Shaoshen Chen +5KV-Cache CompressionMemory-Efficient Inference

  3. Self-Indexing KVCache: Predicting Sparse Attention from Compressed Keys

    Mar 15, 2026Xu Yang, Jiapeng Zhang, Dongyang Zhao +2Binary QuantizationLLM Inference Acceleration

  4. Where Matters More Than What: Decoding-aligned KV Cache Compression via Position-aware Pseudo Queries

    Mar 12, 2026Zhenxu Tian, Yi Su, Juntao Li +1KV-Cache CompressionKV-Cache Eviction

  5. LongFlow: Efficient KV Cache Compression for Reasoning Models

    Mar 12, 2026Yi Su, Zhenxu Tian, Dan Qiao +3KV-Cache CompressionLLM Inference Acceleration

  6. OVGGT: O(1) Constant-Cost Streaming Visual Geometry Transformer

    Mar 6, 2026Si-Yu Lu, Po-Ting Chen, Hui-Che Hsu +3Streaming 3D Reconstruction3D ViTs

  7. Linear Predictability of Attention Heads in Large Language Models

    Mar 4, 2026Khalid Shaikh, Asmit Kumar Singh, Rebecca Christopher Dsouza +1Attention Head AnalysisLLM Inference Acceleration

  8. Understanding the Physics of Key-Value Cache Compression for LLMs through Attention Dynamics

    Mar 2, 2026Samhruth Ananthanarayanan, Ayan Sengupta, Tanmoy ChakrabortyAttention RoutingKV-Cache Compression

  9. Zipage: Maintain High Request Concurrency for LLM Reasoning through Compressed PagedAttention

    Mar 1, 2026Mengqi Liao, Lu Wang, Chaoyun Zhang +6LLM Inference AccelerationLLM Inference Scheduling

  10. KVSlimmer: Theoretical Insights and Practical Optimizations for Asymmetric KV Merging

    Mar 1, 2026Lianjun Liu, Hongli An, Weiqi Yan +4KV-Cache CompressionLLM Inference Acceleration

  11. Whisper-MLA: Reducing GPU Memory Consumption of ASR Models based on MHA2MLA Conversion

    Feb 28, 2026Sen Zhang, Jianguo Wei, Wenhuan Lu +6Automatic Speech RecognitionSpeech Language Models

  12. ST-Lite: Training-Free KV Cache Compression with Spatio-Trajectory Guidance for Long-Horizon GUI Agents

    Feb 27, 2026Bowen Zhou, Zhou Xu, Wanli Li +3KV-Cache CompressionGUI Agents

  13. XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression

    Feb 25, 2026Zunhai Su, Weihao Ye, Hansen Feng +5Streaming 3D ReconstructionKV-Cache Compression

  14. Whole-Piece Training for Symbolic Music Language Models via Full-Horizon Compressed Recurrence

    Feb 23, 2026Yungang Yi, Weihua Li, Matthew Kuo +2Long-Context Language ModelingSymbolic Music Generation

  15. Fast KV Compaction via Attention Matching

    Feb 18, 2026Adam Zweiger, Xinghong Fu, Han Guo +1KV-Cache ManagementLLM Context Compression

  16. Thin Keys, Full Values: Reducing KV Cache via Low-Dimensional Attention Selection

    Feb 16, 2026Hengshuai Yao, Xing Chen, Ahmed Murtadha +1Memory-Efficient AttentionKV-Cache Compression

  17. CompilerKV: Risk-Adaptive KV Compression via Offline Experience Compilation

    Feb 9, 2026Ning Yang, Chengzhi Wang, Yibo Liu +2KV-Cache CompressionLong-Context Inference

  18. ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inference

    Feb 8, 2026Jitai Hao, Qiang Huang, Yaowei Wang +2Self-AttentionKV Caching

  19. GUI-KV: Efficient GUI Agents via KV Cache with Spatio-Temporal Awareness

    Oct 1, 2025Kung-Hsiang Huang, Haoyi Qiu, Yutong Dai +2KV-Cache CompressionEfficient Inference

  20. ReFreeKV: Towards Threshold-Free KV Cache Compression

    Feb 24, 2025Xuanfan Ni, Liyan Xu, Chenyang Lyu +6KV CachingMemory-Efficient Optimization