KV-Cache Eviction

Ask about this topic

Momentum

9 papers in the last four weeks, up 80% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 65

All topics
CardsList

  1. Adaptive Filtering of the KV Cache: Diagnosing and Correcting Structural-Role Bias in LLM Inference

    Jul 14, 2026Soumil MandalKV CachingLong-Context Language Model Inference

    ServiceNow

  2. MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inference

    Jul 12, 2026Venkatesha Matam, Keon KimKV CachingKV-Cache Eviction

    Independent Researcher New York City, USA

  3. KVpop -- Key-Value Cache Compression with Predictive Online Pruning

    Jul 6, 2026Lukas Hauzenberger, Niklas Schmidinger, Anamaria-Roberta Hartl +5KV CachingKV-Cache Eviction

    1NXAI · 2Johannes Kepler University Linz, Austria

  4. Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM

    Jun 28, 2026Shuvendu Roy, Mengyao Zhai, Hossein Hajimirsadeghi +1Long-Context Language Model InferenceKV-Cache Eviction

    RBC Borealis

  5. EpiKV: Epiphany-Aware KV Cache Eviction Without the Attention Matrix

    Jun 25, 2026Steven Kolawole, Virginia SmithKV CachingKV-Cache Eviction

    Language Technologies Institute Carnegie Mellon University · Machine Learning Department Carnegie Mellon University

  6. CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

    Jun 23, 2026Xiaolin Lin, Jingcun Wang, Olga Kondrateva +3LLM Inference EfficiencyKV Caching

    Technical University of Darmstadt, Darmstadt, Germany · University of Notre Dame, Notre Dame, IN, USA · Technical University of Ilmenau, Ilmenau, Germany

  7. Forget Without Compromise: Nexus Sampling for Streaming KV-Cache Eviction Under Fixed Budgets

    Jun 22, 2026Duc Duong, Hoang Anh Duy Le, Jianwen Xie +2KV CachingLong-Context Language Model Inference

    Department of Computer Science, Grinnell College · Department of Computer Science, Rice University · Lambda, Inc +1

  8. Recency/Frequency Adaptive KV Caching for Large Language Model Serving

    Jun 19, 2026Yang Shen, Meghana Madhyastha, Robert Underwood +2LLM ServingKV Caching

    Department of Computer Science, Johns Hopkins University, Baltimore, USA

  9. Exploring a Layer-Wise Design Space for KV Cache Eviction

    Jun 13, 2026Chao Fei, Kaihua Liang, Hanzhi Hu +4KV CachingKV-Cache Eviction

    King Abdullah University of Science and Technology · Fudan University

  10. Value-Aware Stochastic KV Cache Eviction for Reasoning Models

    Jun 2, 2026Ting-Yun Chang, Harvey Yiyun Fu, Deqing Fu +3LLM Inference EfficiencyKV Caching

    University of Southern California · University of Chicago

  11. TGV-KV: Text-Grounded KV Eviction for Vision-Language Models

    Jun 2, 2026Jizhihui Liu, Ruizi Han, Miao Zhang +4Vision-Language ModelsEfficient VLM Inference

    Harbin Institute of Technology, Shenzhen · Peng Cheng Laboratory

  12. MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference

    Jun 1, 2026Yu Li, Binxu Li, Tian LanKV CachingLong-Context Language Model Inference

    George Washington University · Princeton University

  13. Moment-KV: Momentum-Based Decode-Time KV Cache Compression for Long Generation

    May 28, 2026Soumyadeep Jana, Sagar Nishad, Sanasam Ranbir SinghKV CachingLong-Context Language Modeling

    Indian Institute of Technology Guwahati, India

  14. IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference

    May 25, 2026Xintong Yang, Hao Gu, Binxing Xu +6Memory-Augmented Language ModelsSelf-Attention

    The Hong Kong University of Science and Technology · Zhejiang University

  15. CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM

    May 24, 2026Yubo Li, Yidi MiaoKV CachingLong-Context Language Model Inference

    Carnegie Mellon University

  16. A Simple Plug-in for Improving Eviction-Based KV Cache Compression

    May 22, 2026Yuping Lin, Jiayuan Ding, Yue Xing +3KV CachingMemory-Efficient Optimization

    1Michigan State University · 2Hippocratic AI

  17. Meta-Soft: Leveraging Composable Meta-Tokens for Context-Preserving KV Cache Compression

    May 21, 2026Wei Luo, Yi Huang, Songchen Ma +3KV CachingKV-Cache Eviction

    Guangdong Institute of Intelligence Science and Technology · University of Macau · Chinese University of Hong Kong, Shenzhen +1

  18. ArborKV: Structure-Aware KV Cache Management for Scaling Tree-based LLM Reasoning

    May 21, 2026Yeqiu Chen, Ziyan Liu, Zhenxin Huang +3Inference-Time SearchTree of Thoughts

    University of Science and Technology of China

  19. Tensor Cache: Eviction-conditioned Associative Memory for Transformers

    May 21, 2026Kabir Swain, Sijie Han, Daniel Karl I. Weidele +2Memory-Augmented Neural NetworksSelf-Attention

    Massachusetts Institute of Technology, Cambridge, MA, USA · University of Toronto, Toronto, Canada · IBM Research, Cambridge, MA, USA

  20. Protection Is (Nearly) All You Need: Structural Protection Dominates Scoring in Globally Capped KV Eviction

    May 18, 2026Gabriel GarciaKV CachingKV-Cache Eviction

    Independent Researcher

  21. Minimal-Intervention KV Retention via Set-Conditioned Diversity

    May 14, 2026Libo Sun, Po-wei Harn, Peixiong He +1KV CachingKV-Cache Eviction

    Department of Computer Science and Software Engineering Auburn University, Auburn, AL, USA · Department of Information Management National Central University, Taoyuan, Taiwan

  22. Not All Tokens Are Worth Caching: Learning Semantic-Aware Eviction for LLM Prefix Caches

    May 12, 2026Shaoke Fang, Ziang Li, Wenfei Wu +3LLM Inference EfficiencyKV Caching

    Peking University · FirestAI · Tsinghua University +2

  23. Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction

    May 10, 2026Ngoc Bui, Hieu Trung Nguyen, Arman Cohan +1Self-AttentionKV Caching

    Department of Computer Science, Yale University · The Chinese University of Hong Kong

  24. PEEK: Predictive Queue-Informed KV Cache Management for LLM Serving

    May 10, 2026Bing Xie, Zhipeng Wang, Masahiro Tanaka +1LLM ServingKV Caching

    †Corresponding author.

  25. ProxyKV: Cross-Model Proxy Pruning for Efficient Long-Context LLM Inference

    May 9, 2026Junjie Li, Jiong Lou, Jie LiLLM PruningKV Caching

    Shanghai Jiao Tong University

  26. ReST-KV: Robust KV Cache Eviction with Layer-wise Output Reconstruction and Spatial-Temporal Smoothing

    May 9, 2026Yongqi An, Chang Lu, Kuan Zhu +5KV CachingLong-Context Language Model Inference

    Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · University of Electronic Science and Technology of China, Chengdu, China +2

  27. RDKV: Rate-Distortion Bit Allocation for Joint Eviction and Quantization of the KV Cache

    May 8, 2026Junkai Zhang, Hang Guo, Luca Benini +1KV CachingLong-Context Language Model Inference

    ETH Zurich · Tsinghua University

  28. Reformulating KV Cache Eviction Problem for Long-Context LLM Inference

    May 8, 2026Tho Mai, Joo-Young KimKV CachingLong-Context Language Model Inference

    KAIST Daejeon, South Korea

  29. When Does Value-Aware KV Eviction Help? A Fixed-Contract Diagnostic for Non-Monotone Cache Compression

    May 7, 2026Ruijie Zhang, Haozhe Liang, Da Chang +4KV CachingLong-Context Language Model Inference

  30. Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective

    Apr 28, 2026Jiaming Yang, Chenwei Tang, Liangli Zhen +1KV CachingLong-Context Language Model Inference

    College of Computer Science, Sichuan University, Chengdu, China · Institute of High Performance Computing, A*STAR, Singapore