KV Caching

KV: Key-Value

Ask about this topic

Momentum

29 papers in the last four weeks, up 16% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 276

All topics
CardsList

  1. When Fancy Eviction Fails: Rethinking Cache Replacement For LLM Prefix Reuse

    Sep 24, 2026Yiyu Liu, Minlan Yu, Juncheng YangLLM Inference EfficiencyKV Caching

    Harvard University

  2. Shared Global KV with Layer-Specific Local History

    Sep 23, 2026Xinglang XianKV CachingLong-Context Language Model Inference

  3. Written as a Record, Read as an Address: What a Forward Pass Leaves in an Operation's KV Cache

    Sep 21, 2026Lingfeng Wu, Behzad ShomaliKV CachingLLM Interpretability

    University of Bonn · Lamarr Institute

  4. DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

    Sep 17, 2026DeepSeek-AI, :, Anyi Xu +590KV CachingLong-Context Language Model Inference

  5. D-Quant: Driftable Entropy Coding for KV Cache Quantization

    Sep 17, 2026Yi Su, Hong Liu, Guanghua Yu +1KV CachingKV-Cache Compression

    Tencent

  6. PrefixBench-H100: Characterizing Prefix Reuse and Time-to-First-Token in H100 LLM Serving

    Sep 17, 2026Omkar Shewale, Deepak Kumar, Divakar Kumar YadavLLM Inference EfficiencyLLM Inference

    Illinois Institute of Technology · University of Wisconsin–Milwaukee

  7. Ask the Tool, Don't Guess: Agent Tool Calls Hold Their Progress, and the Serving System Should Read It

    Sep 16, 2026Yipeng Liu, Yingqiang Zhang, Feifei Li +1LLM Inference EfficiencyKV Caching

    Tsinghua University, China · Alibaba Cloud Computing, China · Zhejiang University, China

  8. MeshKV: A Network-on-Chip KV Cache Fabric for Scalable Transformer Decoding Accelerators

    Sep 16, 2026Dong Liu, Yanxuan YuTransformer InferenceKV Caching

    Columbia University New York, NY, USA

  9. Contiguity, Not Importance: Budgeted Repair of Stale KV Caches After Document Edits

    Sep 16, 2026Mingyang Mao, Wyatt Mackey, Xiaomin LinRetrieval-Augmented GenerationKV Caching

    University of South Florida, Electrical and Computer Engineering · DEVCOM Army Research Laboratory

  10. JustFit: Just-in-Time State Management for Local LLM Serving

    Sep 15, 2026Yuhua ChenLLM ServingOn-Device Language Model Inference

  11. Fathom: Per-Query Read Depth for Sparse Decoding over Offloaded KV Caches

    Sep 15, 2026Vivek KalyanaranganKV-Cache OffloadingKV Caching

  12. Where Should a Document Live: Context, Representations, or Parameters?

    Sep 15, 2026Nathanaël Carraz Rakotonirina, Momchil Hardalov, Gonzalo Iglesias +1Memory-Augmented Language ModelsKnowledge Augmentation for Language Models

    Amazon AGI

  13. Shared-Prefix KV Reuse Across Standard LoRA Adapters: Quality and Serving Tradeoffs

    Sep 15, 2026Dushyant RajputLLM Inference EfficiencyLLM Serving

    AltSlate Labs LLP

  14. Divergence Timing and Cumulative Disagreement under KV-Cache Eviction

    Sep 15, 2026Xinyue Luo, Fei YuKV CachingKV-Cache Eviction

    Wealth AI Lab, Ant Group

  15. T-LoopFormer: Token-Level Elastic-Depth Looped Transformers for Latent Reasoning With Dynamic Routing

    Sep 14, 2026Mingqian Yu, Wenpeng Zhang, Peilin ZhaoKV CachingRecurrent Transformers

    Institute of Automation Chinese Academy of Sciences Beijing, China · Independent Reasearcher · School of Artificial Intelligence Shanghai Jiao Tong University Shanghai, China

  16. Shared KV Caching for Replicated 27B Inference: Correctness Failures and Performance Boundaries

    Sep 14, 2026Frank LiKV CachingKV-Cache Management

    UNSW Sydney

  17. Pixel Decodability Is Not a Compression Signal: Causally Evaluating Importance Proxies for Visual KV-Cache Eviction

    Sep 14, 2026Chenyu Zhou, Qiliang Jiang, Shuning Wu +1KV CachingKV-Cache Eviction

    School of Engineering, Institute of Science Tokyo, Japan · College of Control Science and Engineering, Zhejiang University, China · Department of Electrical and Computer Engineering, National University of Singapore

  18. AgentKV: Phase-Aware KV Eviction for Agentic LLMs

    Sep 14, 2026Taowen Tony Liu, Jeffrey T. H. Wong, Can Xiao +3KV CachingKV-Cache Eviction

    Imperial College London, London, United Kingdom · Columbia University, New York, NY, USA

  19. OmniKVQuant: KV Cache Quantization for Omni-LLMs

    Sep 11, 2026Suho Yoo, Hyunjong Ok, Jongmin Choi +2Efficient Multimodal InferenceKV Caching

    KAIST · POSTECH

  20. K/V-Cache Interventions Dissociate Representation Alignment from Persona Expression in Decoder-Only Language Models

    Sep 11, 2026Yu Sun, Mengyin Lu, Cong Feng +2KV CachingDecoder-Only Language Models

    LinkedIn

  21. Building py-kvcache: A Performance Characterization of External KV Caching for vLLM with NVMe SSDs

    Sep 11, 2026Joseph Kanichai, Tiziano De Matteis, Animesh TrivediKV-Cache OffloadingKV Caching

    Vrije Universiteit Amsterdam Amsterdam, the Netherlands · IBM Research Zurich Zurich, Switzerland

  22. Composable CXL Memory as a Kubernetes-Native Shared Memory for LLM Serving

    Sep 9, 2026Hongjian Fan, Kevin Zhang, David Habinsky +1LLM Inference EfficiencyKV Caching

    Research Group, Seagate Technology, LLC

  23. KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints

    Sep 9, 2026Xi Shi, Mengxin Zheng, Qian LouKV CachingLLM Inference Acceleration

    University of Central Florida

  24. Fine-Tuning a KV Cache Concatenation-Aware Model or Recomputing KV Caches? Why Not Both?

    Sep 9, 2026Fumihiko Tachibana, Daisuke Miyashita, Jun DeguchiRetrieval-Augmented GenerationKV Caching

    Kioxia Corporation

  25. Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction

    Sep 8, 2026Vishesh Tripathi, Abhay Kumar, Ramsha KhanEfficient Transformer InferenceGrouped-Query Attention

    FrontiersMind

  26. BIO-MEMART: Biometric-Aware KV Cache Memory for Multi-User LLM Agents

    Sep 8, 2026Yanhong Qian, Xuanying He, Qingguo Meng +3Agent MemoryPrivacy-Preserving Language Models

    Anhui Provincial Key Laboratory of Secure Artificial Intelligence, School of Artificial Intelligence, Anhui University, Hefei 230093, China

  27. Jacap: Robust KV Cache Eviction via Jacobian-Based Nonlinear Information Capacity Preservation

    Sep 8, 2026Jiaming Yang, Chenwei Tang, Liangli Zhen +2KV CachingKV-Cache Eviction

    College of Computer Science, Sichuan University, Chengdu, China · 2Engineering Research Center of Machine Learning and Industry Intelligence, Ministry of Education, Chengdu, China · Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore

  28. MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference

    Sep 7, 2026Michael Wang, Keith Li, Roozbeh BostandoostLLM Inference EfficiencyLLM Inference

    Lake Washington School District · XSchool · UMass Amherst

  29. RedKnot-MLA: Multi-Head Offline-Online Reuse for DeepSeek-V4 Long-Context Serving

    Sep 7, 2026Yang Liu, Zhaokai Luo, Huayi Jin +10KV CachingLLM Inference Acceleration

    Xiaohongshu Inc., China · Huawei Cloud · Shanghai Jiao Tong University +2

  30. VestigeKV: The NoPE-MLA KV Cache Carries Its Own Sparse-Attention Signal in a Vestigial Branch

    Sep 3, 2026WenJie FanKV CachingKV-Cache Management

    Yotta Labs

  31. What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservation

    Sep 3, 2026Bo Zeng, Yu Zhao, Yefeng Liu +3KV CachingKV-Cache Eviction

    Ant Group · Alibaba Group · Universität Hamburg

  32. GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving

    Sep 3, 2026Qiankun Ma, Yanjiang Zhou, Zinan Xiong +5LLM InferenceKV Caching

    Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · Pengcheng Laboratory · University of Chinese Academy of Sciences

  33. Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning

    Sep 3, 2026Heng Wang, Jielin Qiu, Wenting Zhao +8KV CachingMemory-Efficient Optimization

    Salesforce AI Research · University of Illinois Urbana-Champaign

  34. SGD-KV: Summarization Guided KV Cache Compression

    Sep 3, 2026Zeyu Liu, Woomin Song, Xuandi Fu +5KV CachingLong-Context Language Model Inference

    USC, USA · KAIST, Korea · Amazon AGI, USA

  35. HeadWiseKV: Budgeted Per-Head Cache Residency for Hybrid Long-Context Language Models

    Sep 2, 2026Renjie Xie, Juncheng Yang, Aoting Hu +4KV CachingLong-Context Language Model Inference

    Nanjing University of Posts and Telecommunications · Tylogi AI Lab / TAIL · Anhui University of Technology +2

  36. CacheBridge: Efficient Cross-Model KV Cache Transfer

    Sep 1, 2026Xingyu Qu, Siyuan Lu, Zhiyu Chen +2Transformer InferenceKV Caching

    Westlake University. · †Work conducted during an internship at Westlake University. · Amazon. +1

  37. Faithfulness Is Not Free: Auditing Offline KV-Cache Quantization in Retrieval-Augmented Generation

    Aug 31, 2026Atta Ul Asad, Ahsan Bilal, Muhammad Ali +2KV CachingRAG Evaluation

    1Lahore University of Management Sciences (LUMS) · University of Oklahoma · 3Air University +1

  38. A Universal Context-Reuse Layer for Cross-Model KV Sharing

    Aug 31, 2026Yi Li, Dongming Jiang, Yi Zhao +1LLM InferenceKV Caching

    University of Texas at Dallas

  39. What It Costs to Compose, Rebuild, and Correct Precomputed Memory

    Aug 31, 2026Asa ShepardLLM Inference EfficiencyMemory-Augmented Language Models

    Department of Computer Science, Williams College

  40. DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving

    Aug 31, 2026Yanqi Yu, Pingwei Sun, Jianchao Tan +4KV CachingMemory-Efficient Inference

    East China Normal University · Meituan · South China University of Technology

  41. Strong Drafts Need Compact Memories: Long-Context Speculative Decoding with Compressed KV Cache

    Aug 31, 2026Tong Yuan, Chengxi Liao, Zeyi WenMemory-Augmented Language ModelsKV Caching

    Data Science and Analytics Thrust, Information Hub, The Hong Kong University of Science and Technology (Guangzhou)

  42. Compression-Aware Abstention: Teaching LLMs to Refuse When KV-Compression Masks Remove Answer Evidence

    Aug 30, 2026Mohammadali Khodabandehlou, Bhaskar KrishnamachariLLM Hallucination MitigationKV Caching

    University of Southern California

  43. Sliding-window beats linear attention

    Aug 28, 2026Alexia Jolicoeur-Martineau, Rhea Sanjay Sukthanker, Pashmina Cameron +1KV CachingLLM Inference Acceleration

    Microsoft Applied Sciences Group (ASG) · Independent

  44. Small Frequency Corrections Can Change What Survives KV Cache Compression

    Aug 27, 2026Hong Chen, Yudong Zeng, Yongwei Huang +5KV CachingKV-Cache Eviction

    The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Bosum Institute of Management Science, Shenzhen, China

  45. StepKV: Step-Aware KV Cache Compression for LLM Agents

    Aug 26, 2026Boyu Feng, Jiahong Liu, Yifan Li +7KV CachingKV-Cache Compression

    The Chinese University of Hong Kong · Huawei Technologies Co., Ltd

  46. PAGE: Partition-Aware Gated KV-Cache Eviction

    Aug 26, 2026Pankaj Kumar, Subhankar MishraLong-Context RetrievalKV Caching

    School of Computer Sciences National Institute of Science Education and Research An OCC of Homi Bhabha National Institute, India

  47. Learning how to Forget: Fine-tuning for Long-Context Sparse Attention

    Aug 20, 2026Matthias Seeger, Zeyu Zhang, Vihang Patil +2Fine-TuningKV Caching

    Amazon Web Services · University of Amsterdam · Amazon +1

  48. WhiteMatter: All-to-All Cross-Layer Connections via KV Source Mixing

    Aug 19, 2026Wenbo Zhang, Xiang RenEfficient Transformer InferenceKV Caching

    University of Southern California

  49. vToken: Token-Level Virtualization for Reclaimable KV Caches

    Aug 13, 2026Yuanhang Gao, Xiangrui Yang, Yuanfeng Chen +4KV CachingKV-Cache Eviction

    National University of Defense Technology · Changsha, China · Peking University +1

  50. QV-PIC: Query-Aware Visual Position-Independent Caching for Efficient RAG Serving

    Aug 12, 2026Yilin Liu, Rui Meng, Wangze Ni +5Retrieval-Augmented GenerationKV Caching

    Zhejiang University · Department of Statistics and Data Science, Beijing Normal-Hong Kong Baptist University · Microsoft +2

  51. Neural Introspection Gating for Adaptive KV-Cache Reuse in Vision-Language-Action Models

    Aug 11, 2026Zhijie Wu, Kento Kawaharazuka, Kei OkadaAdaptive InferenceKV Caching

    Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8656, Japan

  52. ImpactHO: Importance-Aware KV Cache Transfer for Multi-User Edge LLM Handover

    Aug 11, 2026Minwoo Kim, Soochang Song, Namyoon Lee +2KV CachingEdge Computing

    Department of Electrical Engineering, Pohang University of Science and Technology (POSTECH), Pohang 37673, South Korea · Department of Electrical and Computer Engineering, Ajou University, Suwon 16499, South Korea

  53. KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Models

    Aug 10, 2026Chen Qiu, Ziwu Liu, Chao Fei +2Long-Context Language Model EvaluationKV Caching

    KAUST Thuwal, Saudi Arabia

  54. Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference

    Aug 10, 2026Tejasvi C. AddagadaKV CachingKV-Cache Management

  55. DistillCache: KL-Guided Adaptive KV-Cache Eviction for Memory-Efficient LLM Inference

    Aug 9, 2026Asaad AlthoubiKV CachingLong-Context Language Model Inference

    Oklahoma State University Stillwater, OK USA

  56. RippleKV: Cross-Layer KV Cache Allocation via Perturbation Propagation

    Aug 9, 2026Dongjie Xu, Kai Qian, Julius +6KV CachingKV-Cache Management

    1Soochow University · 2Tencent Inc. · 3The Hong Kong University of Science and Technology +2

  57. SPECTRA: Pushing the KV Cache Beyond the 2-Bit Cliff via Spectral Transform Coding

    Aug 8, 2026Jiamu Zhang, Liang Wu, Kelly Wan +2Mixed-Precision QuantizationKV Caching

    Nokia · Rice University

  58. CommitKV: Lifecycle-Aware KV Cache Compression via Commit Transitions for Multi-Turn Agents

    Aug 8, 2026Weizhong Huang, Jinchao Zhang, Xiawu ZhengKV CachingLLM Agent Memory

    Xiamen University · WeChat AI, Tencent Inc., China

  59. CoinRAG: Contextualized Information Nugget KV Cache Reuse for Long-Context RAG

    Aug 7, 2026Gyuwan Kim, Cheoneum Park, Tao YangLong-Context RetrievalRetrieval-Augmented Generation

    University of California, Santa Barbara · Hanbat National University

  60. Addressable Memory for Video World Models

    Aug 7, 2026Xindi Wu, Sven Elflein, James Lucas +5KV CachingLong-Horizon Video Generation

    1NVIDIA · 2Princeton University · University of Toronto +1