Overview

Key-value cache eviction studies how to shrink the tokens stored during generation so long-context models fit in limited GPU memory and serve requests faster. Common approaches rank tokens using attention, recency, value magnitude, or redundancy; newer methods estimate future or output-level utility, allocate budgets across layers and heads, or combine eviction with quantization and recoverable memory. Methods are commonly evaluated on long-context understanding, retrieval, and reasoning benchmarks such as LongBench, RULER, and Needle-in-a-Haystack, alongside latency and memory measurements. The central tension is that cheap importance signals can miss information needed later, while more accurate prediction or reconstruction may add computation; results also depend on workload, model, hardware, and the rigor of matched-budget evaluation.

Latest papers

CardsList

104 papers7 papers in the last four weeks, down 68% on the four weeks before. 0.1% of all new papers.

  1. AutoAssumption: Searching over Assumptions Discovers ML-Systems Algorithms

    Nov 10, 2026Igor Sadalski, Minghao Guo, Benjamin Jones +1Automated Algorithm DiscoveryKV-Cache Eviction

  2. The Gating Mechanism of KV Cache Eviction

    Nov 5, 2026Hao Gu, Xintong Yang, Zhiwei Li +10KV-Cache EvictionKV-Cache Compression

  3. EchoKV: Self-Corrective KV Cache Eviction via Iterative Query Feedback

    Nov 4, 2026JianKun Zou, Tongtong YuanKV-Cache EvictionKV-Cache Management

  4. WAKMan: Workload Aware KVCache Manager

    Oct 26, 2026Suhas Kumar Morisetty, Vaibhavi Lokegaonkar, Sachin Mathew Varghese +1KV-Cache ManagementLanguage Model Inference Caching

  5. MOSAIC: Multi-Objective Search for Allocating Inference Cache Across LLM Decoder Layers

    Oct 18, 2026Sravanth Kodavanti, Manjunath ArvetiLanguage Model Inference CachingKV-Cache Management

  6. TideKV: Delayed KV Eviction via Layer-Wise Chunked Prefill for Memory-Efficient Long-Context LLM Inference

    Oct 17, 2026Yu Wang, Zhu Wang, Ning Hu +4Memory-Efficient InferenceLong-Context Language Model Inference

  7. DropKV: Decoupling Residual-Output Perturbation for Near-Optimal KV-Cache Eviction

    Oct 15, 2026Aozhong Zhang, Selcuk Gurses, Yanxia Deng +8KV-Cache EvictionLLM Inference Acceleration

  8. TwinPrune: Density-Aware Two-Phase Token Pruning for Vision-Language Models

    Oct 13, 2026Kai Liu, Anqi Li, Junxian Li +4Efficient VLM InferenceVisual Token Pruning

  9. Monte Carlo Estimation for KV Cache Eviction

    Oct 6, 2026Ahsan Bilal, Muhammad Ahmed Mohsin, Muhammad Umer +5LLM Inference EfficiencyKV-Cache Eviction

  10. Behavior-Preserving KV Cache Compression

    Oct 5, 2026Doo Hwan Hwang, Junyoung Jang, Junho Na +2KV-Cache EvictionLLM Inference Acceleration

  11. DeferKV: Rethinking Eviction Timing for One-Shot KV Cache Compression

    Oct 5, 2026Zhe Wang, Jiakai Li, Yujia Sun +2Long-Context Language Model InferenceKV-Cache Eviction

  12. Mask-Guided KV Cache Eviction in Block Diffusion Language Models

    Oct 4, 2026Gleb Molodtsov, Ekaterina Alimaskina, Evgeny Uskov +2KV-Cache EvictionDiffusion Language Model Inference

  13. KV2^2: A Self-Refining KV Cache

    Oct 2, 2026Johannes Wesch, Danni Liu, Jan NiehuesKV-Cache CompressionKV-Cache Eviction

  14. Page-EntroKV: Hardware-Aligned, Entropy-Weighted KV-Cache Eviction under Grouped-Query Attention

    Oct 2, 2026Inbasekaran SKV-Cache EvictionMemory-Efficient Inference

  15. Emergent Structure in the Marginal Attention Space of Language Models

    Oct 2, 2026Valentino Maiorca, Walter Nelson, Francesco LocatelloAttention MechanismsAttention Head Analysis

  16. AvoKV-E: Payload-Aware KV Cache Eviction for Long Reasoning

    Oct 2, 2026Han Yu, Wenhui Zhu, Xiwen Chen +9KV-Cache EvictionCache Replacement

  17. One Distribution, Two Channels: Coverage-Calibrated KV Retention and Compensation

    Sep 28, 2026Shuxin Liu, Qing Liu, Yi Du +1KV Cache SelectionKV-Cache Eviction

  18. The KV Cache Is the New Memory Wall

    Sep 25, 2026Tejinder SinghKV CachingLong-Context Language Model Inference

  19. Beyond Mean Attention: Diversity-Aware, Layer-Wise Scoring for KV Cache Eviction

    Sep 25, 2026Tianfang Xie, Wei ZhuKV-Cache EvictionKV-Cache Pruning

  20. Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targets

    Sep 23, 2026Beomgu Kang, SoJin Yun, Hojoon Kim +1KV-Cache EvictionMemory-Efficient Inference

  21. DeltaS: Reading the Gated Linear Attention State for KV Cache Eviction in Streaming Video

    Sep 23, 2026Taeyoun Kwon, Seungjin Kim, Hyeonyu Kim +1Streaming Video UnderstandingKV-Cache Eviction

  22. ValueDiff: Value-Geometric KV Cache Eviction for Sink-Suppressed LLMs

    Sep 20, 2026Junyoung Park, Jungwook Choi, Mingu LeeLLM Inference EfficiencyKV-Cache Eviction