Overview

Key-value cache compression reduces the memory and bandwidth cost of storing attention states, enabling longer contexts, larger batches, and inference on constrained hardware. Common approaches include token eviction and merging, quantization, low-rank or latent representations, and adaptive per-layer or per-head budgets; some methods also reconstruct or retrieve compressed states on demand. Work is evaluated on long-context retrieval and question answering benchmarks such as LongBench, RULER, and Needle-in-a-Haystack, as well as reasoning, multimodal tasks, and end-to-end serving. The central tension is that aggressive compression can harm accuracy, evidence use, or safety, while a high compression ratio does not guarantee faster inference or lower peak memory, so workload-aware evaluation remains essential.

Latest papers

CardsList

293 papers30 papers in the last four weeks, down 48% on the four weeks before. 0.2% of all new papers.

  1. MovieRoll: A Systematic Framework for Rolling Out Long-Form Videos

    Nov 16, 2026Yunyang Ge, Weijie Kong, Ryan Xu +9Long-Horizon Video GenerationVideo Generation

  2. GaugeRoPE: RoPE-Compatible Executable Latent Attention

    Nov 16, 2026Yuanhang Gao, Xiangrui Yang, Lingyuan Meng +2KV-Cache CompressionLLM Inference Acceleration

  3. KVPath: Connecting Key-Value Representations Across Transformer Layers

    Nov 11, 2026Xuan Luo, Zhiyuan Wang, Hanzhong Cao +1KV CachingKV-Cache Compression

  4. TraceKV: Policy-Preserving Low-Rank KV Cache Compression for Reasoning Vision-Language Models

    Nov 11, 2026Ke Li, Zishan Shao, Yiran ChenKV-Cache CompressionEfficient VLM Inference

  5. Semantically Aligned KV Cache Compression for Efficient Agentic Autoregressive Video Generation

    Nov 8, 2026Weizhong Huang, Huaijia Kou, Tianyu Xie +2KV-Cache CompressionAutoregressive Video Generation

  6. Verify Before You Fetch: Reliable KV Cache Compression for LLM Inference

    Nov 8, 2026Devanshu Jha, Gaurav Kumar, Agniva Chowdhury +2KV-Cache CompressionLong-Context Language Model Inference

  7. CALFA-KV: Joint Calibration and Fisher-Guided Anchors for 1-Bit KV Cache Compression

    Nov 7, 2026Xiancai Tian, Leichao Zhang, Shenhui Zhang +5KV-Cache CompressionKV-Cache Quantization

  8. Transformer-FSQ: Discrete Latents for Extreme KV-Cache Compression

    Nov 6, 2026Andras Szecsenyi, Thomas L. Lee, Vaisakh Shaj +2KV-Cache CompressionKV-Cache Quantization

  9. The Gating Mechanism of KV Cache Eviction

    Nov 5, 2026Hao Gu, Xintong Yang, Zhiwei Li +10KV-Cache EvictionKV-Cache Compression

  10. TACT-KV: Tri-Axis Cosine Transform for Compressing Volumetric KV Caches in Medical VLMs

    Nov 5, 2026Chongyu Qu, Ritchie Zhao, Yufan He +12KV-Cache CompressionKV-Cache Quantization

  11. LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time

    Nov 4, 2026Teresa Delgado, Hugo Hazard, Martin Benfeghoul +2KV-Cache CompressionMemory-Efficient Inference

  12. LoopKV: Loop-Aware KV Cache Compression for Serving Looped Transformers

    Nov 3, 2026Tianle Zhou, Yang Zhang, Tianyu Wu +3KV-Cache CompressionLLM Inference Acceleration

  13. Exact Fusion of Softmax Attention Heads Can Require Exponentially Many Atoms

    Nov 2, 2026Hiroyasu Usami, Keisuke Hara, Ayato Tsuboi +1Softmax AttentionMulti-Head Attention

  14. BasisKV: Value Latents for Communication and Key Routing

    Nov 2, 2026Lixun Zhang, Zishan Shao, Kangning Cui +3KV Cache SelectionKV-Cache Management

  15. EarlySelectKV: Anticipatory KV Selection via Cross-Layer Query Proxies

    Nov 1, 2026Yuhao Liu, Yuehao Li, Linghe Zhang +4KV Cache SelectionLong-Context Language Model Inference

  16. ACCT: Asymmetric Cross-LLM Context Transfer

    Oct 30, 2026Yuzhe Liu, Haolin Tian, Tonghan WangKV-Cache ReuseLLM Inference Acceleration

  17. In-Model Context Compression with Agentic Soft Tokens for Long-Horizon LLM Agents

    Oct 30, 2026Xuning He, Kai Fan, Runyu Zhang +4Context Compaction for LLM AgentsLLM Context Compression

  18. Kernel-based Interpolatory Methods for Context Compression of Attention Mechanisms

    Oct 29, 2026Michel Fabrice Serret, Laura GrigoriLLM Context CompressionKV-Cache Compression

  19. FAD-KV: Adaptive Dictionary KV Caching for Efficient Reasoning Decoding

    Oct 27, 2026Yuxuan Zhu, David H. Yang, Parisa Shiravani +5KV-Cache CompressionLLM Inference Acceleration

  20. DCKV: Learned Transform Coding of the Prompt KV Cache in Hybrid Language Models

    Oct 27, 2026Egor Maximov, Aleksei Merkelov, Vasiliy Alekseev +5KV-Cache CompressionKV-Cache Quantization

  21. Attention under Bounded Key-Query Similarity: Space Complexity and Transformer Anisotropy

    Oct 26, 2026Justin Y. Chen, Ying Feng, Piotr Indyk +3Attention ApproximationKV-Cache Compression