KV-Cache Reuse

Ask about this topic

Overview

KV-cache reuse studies how language models can retain and reuse attention states instead of repeatedly processing the same documents, conversation history, or shared context, reducing latency and memory costs in retrieval-augmented generation, long-context, and multi-agent settings. Approaches include exact-prefix caching, selective recomputation and cache repair, compression, position-invariant reuse, and transferring cache states across models; serving systems add scheduling and storage tiers to make reuse practical. A central challenge is preserving answer quality when cached states are used in a different position, context, model, or request order, so many methods combine reuse with targeted correction or recomputation. Evaluation remains sensitive to workload and measurement choices, and safety, privacy, and timing leakage are additional concerns when caches are shared.

Latest papers

CardsList

73 papers19 papers in the last four weeks, up 19% on the four weeks before. 0.1% of all new papers.

  1. Shared Context, Separate Engines: Streaming Vision--Language Inference on Edge SoCs

    Oct 26, 2026Yilong Li, Chengpo Yan, Yijing Zeng +3Efficient VLM InferenceVLM Inference

    University of Wisconsin–Madison. · Meta. · Carnegie Mellon University.

  2. BeNice! Closing Looping's Inference-Memory Gap via KV-Sharing

    Oct 22, 2026Hayden Prairie, Jacob M. Ryan, Taylor Berg-Kirkpatrick +1KV-Cache ReuseMemory-Efficient Inference

    University of California, San Diego · Together AI

  3. System X: An LLM Decision Model That Replies from Its Own KV Cache

    Oct 18, 2026Syed Usama Khalid Bukhari, Syed Safwan Khalid, Muhammad AamirLLM Decision-MakingKV-Cache Reuse

    Turing Logic LTD · University of Oxford

  4. Close Enough is Nowhere Near: Fidelity-Consumability Decoupling in KV Caches

    Oct 15, 2026Adam Deng, Shicheng Liu, Yada ZhuKV-Cache ReuseMulti-Agent LLM Systems

    MIT ORC · MIT CSAIL · IBM Research

  5. LatentRemedy: Efficient Remedy For Non-Prefix KV Cache Reuse in Multi-Agent Systems

    Oct 13, 2026Zhiwei Chen, Zhongtao Rao, Waikit Xiu +4KV-Cache ReuseMulti-Agent LLM Systems

    The Hong Kong University of Science and Technology (Guangzhou) · Microsoft · The University of Hong Kong

  6. Hashrope: A Persistent Data Structure for Mechanisms of LLM Context Management

    Oct 12, 2026Muntaser Syed, Ameerah Alsulami, Marius SilaghiLLM Context ManagementLLM Inference Acceleration

    Florida Institute of Technology

  7. To Cache or Not to Cache: Understanding and Mitigating Safety Incompatibility in Cross-Model KV Cache Reuse

    Oct 11, 2026Zhiwei Lin, Jiahao Liu, Yan Liu +1KV-Cache ReuseLLM Safety Evaluation

    National University of Singapore · Ant Group

  8. CacheLink: Functional KV Transport for Zero-Prefill Cross-Model Context Handoff

    Oct 11, 2026Haomiao Qiu, Yining Tang, Zijia Song +3LLM ServingKV-Cache Reuse

    Harbin Institute of Technology, Shenzhen · Great Bay University · Zhejiang University

  9. SAFE-KV: Causal Suffix Repair for Behavior-Preserving Non-Prefix KV Cache Reuse in RAG

    Oct 10, 2026Xingjian ZhangKV-Cache ReuseKV-Cache Management

    Carnegie Mellon University, Pittsburgh, PA, USA

  10. KV-Repurpose: A Testbed for Evaluating Non-Prefix KV-Cache Reuse

    Oct 9, 2026Taiga Ikeda, Daisuke Miyashita, Jun DeguchiKV-Cache ReuseKV-Cache Management

    Kioxia Corporation, Japan

  11. RaReCache: Bridging the Gap in Cross-Model KV Cache Reuse via Rank disagreement-based Selective Recomputation

    Oct 8, 2026Sreetama Sarkar, Saptarshi Mitra, Sitao Huang +2KV-Cache ReuseLLM Inference Acceleration

    University of Southern California · University of California, Irvine · Intel Labs

  12. Real Long-Term Memory for AI: A 50-Million-Token Window That Is Faster and Cheaper Than Recompute

    Oct 7, 2026Sietse SchelpePersistent Memory for Language ModelsKV-Cache Reuse

    Corbenic AI

  13. Request Order Matters: Cache-History Sensitivity in Selective KV-Cache Reuse for Rolling Agents

    Oct 5, 2026Tiffany Gu, Annie Guan, Manshu Huang +3KV-Cache ManagementLLM Inference Acceleration

    Capital One · University of Washington

  14. KV2^2: A Self-Refining KV Cache

    Oct 2, 2026Johannes Wesch, Danni Liu, Jan NiehuesKV-Cache CompressionKV-Cache Eviction

    Karlsruhe Institute of Technology, Germany

  15. Working Around the Compute Ceiling: Byte-Exact Memory in Galahad Makes LLM Reading a One-Time Cost LLM Reading a One-Time Cost

    Sep 30, 2026Sietse SchelpeLLM ServingLong-Context Language Model Inference

    Corbenic AI

  16. ATTUNER: Recomputation-Free Agentic Context KV Reuse

    Sep 29, 2026Xinghao Chen, Junnan Dong, Cai Ke +8KV-Cache ReuseLLM Inference Acceleration

    EIT-NLP Lab, Eastern Institute of Technology, Ningbo · The Hong Kong Polytechnic University · Tencent Youtu Lab

  17. Dynamic Flow, Static Graph: KV Cache Reuse for Efficient LLM Serving on Mobile NPUs

    Sep 28, 2026Zhengxiang Huang, Shengheng Chen, Chaoyue Niu +6AI Accelerator InferenceOn-Device Language Model Inference

    Shanghai Jiao Tong University Shanghai, China · Alibaba Group Hangzhou, China · Alibaba Group Hangzhou, Zhejiang, China

  18. KVCMAS: Efficient KV cache Correction for Shared Context in Multi-Agent Systems

    Sep 28, 2026Hyesung Jeon, Hyeongju Ha, Seoyoung Lee +2Multi-Agent LLM SystemsKV-Cache Management

    Department of Electrical and Computer Engineering Seoul National University

  19. PReCache: Efficient KV Cache Sharing for Multi-LoRA Agents via Low-Rank Precomputation and Neutral Reconstruction

    Sep 28, 2026Hyesung Jeon, Hyeongju Ha, Jae-Joon KimMulti-Agent LLM SystemsMemory-Efficient Inference

    Department of Electrical and Computer Engineering Seoul National University

  20. Thinking Outside the Box: Retention and Transmission of Information in Sliding-Window KV Inference

    Sep 28, 2026Timothy DeLise, Seth CromelinLong-Context Language Model InferenceKV-Cache Reuse

  21. RelaxKV: Recomputation Guided by the Query with Sparse Context Attention for Efficient KV Cache Reuse

    Sep 27, 2026Ruoling Qi, Yirui Liu, Xuaner Wu +5Retrieval-Augmented GenerationKV-Cache Management

    Shanghai Jiao Tong University · Institute of Artificial Intelligence, China Telecom (TeleAI) · State University of New York at Buffalo

  22. Budgeted Cache Repair for Cross-Context KV-Cache Reuse

    Sep 27, 2026Haeyong Kang, Chang D. YooKV-Cache ManagementKV-Cache Reuse

    Duksung Women’s University · KAIST

  23. In-Flight KV Cache with Clean Anchors for Faster Autoregressive Video Diffusion

    Sep 26, 2026Yikai Wang, Xiao Han, Mengmeng Xu +9Long-Horizon Video GenerationDiffusion Model Caching

    Meta · S-Lab, Nanyang Technological University

  24. Evaluating the accuracy of KV cache reuse techniques

    Sep 25, 2026Samuel Cestola, Tianxiang Xia, Pengfei Zheng +4KV-Cache ReuseRAG Evaluation

    Huawei Technologies Ltd. · Department of Computer Science, ETH Zurich

  25. CacheReforge: Bounded Recovery for Stale KV Caches under Evolving Adapters

    Sep 25, 2026Yuhang Cao, Yanzhou Mu, Chunrong Fang +1KV-Cache ManagementKV-Cache Reuse

    Nanjing University

  26. Contiguity, Not Importance: Budgeted Repair of Stale KV Caches After Document Edits

    Sep 16, 2026Mingyang Mao, Wyatt Mackey, Xiaomin LinRetrieval-Augmented GenerationKV Caching

    University of South Florida, Electrical and Computer Engineering · DEVCOM Army Research Laboratory

  27. Shared-Prefix KV Reuse Across Standard LoRA Adapters: Quality and Serving Tradeoffs

    Sep 15, 2026Dushyant RajputLLM Inference EfficiencyLLM Serving

    AltSlate Labs LLP

  28. Shared KV Caching for Replicated 27B Inference: Correctness Failures and Performance Boundaries

    Sep 14, 2026Frank LiKV CachingKV-Cache Management

    UNSW Sydney

  29. Composable CXL Memory as a Kubernetes-Native Shared Memory for LLM Serving

    Sep 9, 2026Hongjian Fan, Kevin Zhang, David Habinsky +1LLM Inference EfficiencyKV Caching

    Research Group, Seagate Technology, LLC

  30. KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints

    Sep 9, 2026Xi Shi, Mengxin Zheng, Qian LouKV CachingLLM Inference Acceleration

    University of Central Florida