Kv-Cache Management

Ask about this topic

Momentum

15 papers in the last four weeks, up 400% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 63

All topics
CardsList

  1. TempoKV: Timely Staging of LLM KV Caches for Memory-Semantic Flash

    Sep 28, 2026Jay H. Park, Hyungjun Kim, Dong KimKv-Cache ManagementCache

    Samsung Semiconductor

  2. Dynamic Flow, Static Graph: KV Cache Reuse for Efficient LLM Serving on Mobile NPUs

    Sep 28, 2026Zhengxiang Huang, Shengheng Chen, Chaoyue Niu +6Kv-Cache ManagementNeural Processing Units

    Shanghai Jiao Tong University Shanghai, China · Alibaba Group Hangzhou, China · Alibaba Group Hangzhou, Zhejiang, China

  3. PulseInfer: I/O-Centric Sparse KV Cache Offloading for Efficient Long-Context LLM Decoding

    Sep 28, 2026Qiuyang Zhang, Kai Zhou, Kai Lu +6Kv-Cache ManagementLarge Language Model Decoding

    Huazhong University of Science and Technology Wuhan, China · UCloud Technology Co., Ltd. Shanghai, China

  4. DPS: Dual-Mode Precision LLM Serving with Semi-Unified Memory

    Sep 28, 2026Xuan Truong Nguyen, Tien Son Pham, Tuan Duc Chu +2Graphics Processing Unit MemoryKv-Cache Management

    Department of Next Generation Semiconductor Convergence and Open Sharing System (COSS), Seoul National University, Seoul 08826, South Korea · Van-Lang Institute of Semiconductor Technology (VIST), Hanoi, Vietnam · Efficient Computation Research Group, Moreh Vietnam +1

  5. KVCMAS: Efficient KV cache Correction for Shared Context in Multi-Agent Systems

    Sep 28, 2026Hyesung Jeon, Hyeongju Ha, Seoyoung Lee +2Kv-Cache ManagementKey-Value Cache

    Department of Electrical and Computer Engineering Seoul National University

  6. EfficientAgent: What Makes KV Cache Offloading Work for Concurrent Agents?

    Sep 27, 2026Kunming Shao, Jierun Chen, Jiangnan Yu +7Kv-Cache ManagementOffloading

    The Hong Kong University of Science and Technology · Huawei Technologies Ltd. · The University of Hong Kong +1

  7. FlashLoop: Fast and Memory-Efficient Looped Transformers via Lazy Updates

    Sep 24, 2026Wanqi Yang, Shiwei LiuBlock Sparse Flash AttentionTransformer Architectures

    ELLIS Institute Tübingen Max Planck Institute for Intelligent Systems Tübingen AI Center

  8. TierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Caching

    Sep 18, 2026Zhihao Shu, Md Musfiqur Rahman Sanim, Jie Hu +4Kv-Cache ManagementDepthweave-Kv

    University of Georgia Athens, GA, USA · Peking University Beijing, China · Western Digital Research San Jose, CA, USA

  9. Shared-Prefix KV Reuse Across Standard LoRA Adapters: Quality and Serving Tradeoffs

    Sep 15, 2026Dushyant RajputKv-Cache ManagementKey-Value Cache

    AltSlate Labs LLP

  10. Shared KV Caching for Replicated 27B Inference: Correctness Failures and Performance Boundaries

    Sep 14, 2026Frank LiKv-Cache ManagementCpu-Gpu Hybrid Designs

    UNSW Sydney

  11. DiffSynth-Music: Audio-Conditioned KV-Cache Adapters for Controllable Music Generation

    Sep 14, 2026Zhongjie Duan, Shengchuan Gao, Hong Zhang +1Song GenerationAudio Editing

    Alibaba Group · Shanghai Jiao Tong University

  12. Building py-kvcache: A Performance Characterization of External KV Caching for vLLM with NVMe SSDs

    Sep 11, 2026Joseph Kanichai, Tiziano De Matteis, Animesh TrivediKv-Cache ManagementKey-Value Cache

    Vrije Universiteit Amsterdam Amsterdam, the Netherlands · IBM Research Zurich Zurich, Switzerland

  13. Composable CXL Memory as a Kubernetes-Native Shared Memory for LLM Serving

    Sep 9, 2026Hongjian Fan, Kevin Zhang, David Habinsky +1Kv-Cache ManagementKey-Value Cache

    Research Group, Seagate Technology, LLC

  14. KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints

    Sep 9, 2026Xi Shi, Mengxin Zheng, Qian LouKv-Cache ManagementLarge Language Model Serving

    University of Central Florida

  15. BIO-MEMART: Biometric-Aware KV Cache Memory for Multi-User LLM Agents

    Sep 8, 2026Yanhong Qian, Xuanying He, Qingguo Meng +3Kv-Cache ManagementKey-Value Cache

    Anhui Provincial Key Laboratory of Secure Artificial Intelligence, School of Artificial Intelligence, Anhui University, Hefei 230093, China

  16. A Universal Context-Reuse Layer for Cross-Model KV Sharing

    Aug 31, 2026Yi Li, Dongming Jiang, Yi Zhao +1Kv-Cache ManagementKeys And Value

    University of Texas at Dallas

  17. vToken: Token-Level Virtualization for Reclaimable KV Caches

    Aug 13, 2026Yuanhang Gao, Xiangrui Yang, Yuanfeng Chen +4Kv-Cache ManagementKey-Value Cache

    National University of Defense Technology · Changsha, China · Peking University +1

  18. ImpactHO: Importance-Aware KV Cache Transfer for Multi-User Edge LLM Handover

    Aug 11, 2026Minwoo Kim, Soochang Song, Namyoon Lee +2Kv-Cache ManagementKey-Value Cache

    Department of Electrical Engineering, Pohang University of Science and Technology (POSTECH), Pohang 37673, South Korea · Department of Electrical and Computer Engineering, Ajou University, Suwon 16499, South Korea

  19. Cascade: Exploiting SLO-Aware latency budget for fair and high goodput LLM inference serving

    Aug 6, 2026Muhammad Adnan, Rohan Mahapatra, Prashant J. Nair +4Large Language Model ServingLLM Inference Optimization

    The University of British Columbia · Microsoft Azure Research · NVIDIA

  20. Output-Aware Rotation for INT2 KV-Cache Quantization

    Aug 3, 2026Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong +4Key-Value Cache QuantizationKv-Cache Management

    University of Southern California · Seoul National University · Inha University

  21. InferScale: GPU-Native KV Injection for Personalized LLM Serving

    Jul 29, 2026Peter Li, Prashant PandeyLarge Language Model MemoryKv-Cache Management

    Northeastern University Boston, Massachusetts, USA

  22. Kalypso: Relational LLM Serving

    Jul 26, 2026Hojae Son, Md Ashraful Islam, Huy Gia Cao +2Large Language Model ServingKv-Cache Management

    UMass Amherst, USA

  23. MoA-Structured Decode Attention DNF Derivation, KV-Cache Accumulation, GQA/MQA, and OpenACC Kernel

    Jul 21, 2026Lenore Mulin, Gaetan HainsMatrix MultiplicationTransformer Attention

    University at Albany, SUNY · LACL, Universit´e Paris-Est Cr´eteil

  24. SALT: Salience-Aware Lexical Trie for Long-Context Compression

    Jul 20, 2026Oteo Mamo, Hyunjin Yi, Joydhriti Choudhury +2Efficient Long-Context InferenceLarge Language Model Memory

    Florida State University

  25. PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization

    Jul 17, 2026Yuchen Yang, Yifan Zhao, Anisha Dasgupta +1Graphics Processing Unit MemoryMixture-Of-Experts Large Language Models

    University of Illinois Urbana-Champaign

  26. Smarter and Cheaper at Once: Byte-Exact KV-Cache Grafting Turns a Frozen Small Model into a Verified-Knowledge Flywheel

    Jul 15, 2026Sietse SchelpeFrozen Language ModelKv-Cache Management

    Corbenic AI

  27. MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inference

    Jul 12, 2026Venkatesha Matam, Keon KimKey-Value Cache EvictionKv-Cache Management

    Independent Researcher New York City, USA

  28. Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization

    Jul 9, 2026Jiantong Jiang, Peiyu Yang, Rui Zhang +1Kv-Cache ManagementLarge Language Model Serving

    School of Computing and Information Systems, The University of Melbourne · School of Computer Science and Technology, Huazhong University of Science and Technology (www.ruizhang.info)

  29. TriRoute: Unified Learned Routing for Joint Adaptive Attention, Experts, and KV-Cache Allocation

    Jul 7, 2026Andrii Balashov, Olena PonomarovaKey-Value Cache QuantizationMixture-Of-Experts

    Ukrainian State University of Science and Technologies

  30. When Latent Agents Lie: KV-Cache Integrity in Multi-Agent LLM Collaboration

    Jun 27, 2026Luís Brito, Carlos BaqueroLarge Language Model AgentsHidden States

    Escola Superior de Tecnologia e Gestão (ESTG), Politécnico de Viana do Castelo (IPVC), Portugal · Faculdade de Engenharia (FEUP), Universidade do Porto (UP), Portugal

  31. PersistentKV: Page-Aware Decode Scheduling for Long-Context LLM Serving on Commodity GPUs

    Jun 25, 2026Muhammad AhmedKv-Cache ManagementLarge Language Model Serving

  32. CrossPool: Efficient Multi-LLM Serving for Cold MoE Models through KV-Cache and Weight Disaggregation

    Jun 23, 2026Zhuoren Ye, Tianyu Wo, Dinghao Xue +4Kv-Cache ManagementGraphics Processing Unit Memory

    School of Software, Beihang University

  33. Execution-State Capsules: Graph-Bound Execution-State Checkpoint and Restore for Low-Latency, Small-Batch, On-Device Physical-AI Serving

    Jun 18, 2026Liang SuKv-Cache ManagementIntermediate Checkpoints

  34. KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing

    Jun 15, 2026Mufei Li, Shikun Liu, Dongqi Fu +5Kv-Cache ManagementKey-Value Cache

    Georgia Institute of Technology · Meta

  35. See What I See, Know What I Think: Dense Latent Communication Across Heterogeneous Agents

    Jun 11, 2026Siyi Chen, Xiaoyan Zhang, Meng Wu +7Kv-Cache ManagementLatent Representation Alignment

    University of Michigan · NVIDIA · University of Pennsylvania +2

  36. TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment

    Jun 11, 2026Yu Meng, Xiangyang Luo, Letian Li +5Autoregressive Video Diffusion ModelsAutoregressive Video Generation

    Tsinghua University · D-INFK, ETH Zürich

  37. RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention

    Jun 4, 2026Yang Liu, ZhaoKai Luo, HuaYi Jin +5Kv-Cache ManagementKey-Value Cache

    †Xiaohongshu Inc., China · ¶Huawei Cloud · §Peking University

  38. NetKV: Network-Aware Decode Instance Selection for Disaggregated LLM Inference

    Jun 2, 2026Mubarak Adetunji OjewaleKv-Cache ManagementTime-To-First-Token

    Cloud Competency Centre, National College of Ireland

  39. Multi-Segment Attention: Enabling Efficient KV-Cache Management for Faster Large Language Model Serving

    Jun 1, 2026Chunan Shi, Yilei Chen, Yilin Chen +2Kv-Cache ManagementCache

    Peking University Beijing, China

  40. Move the Query, Not the Cache: Characterizing Cross-Instance Latent Attention Redistribution Across GPU Fabrics

    May 31, 2026Bole Ma, Jan Eitzinger, Harald Köstler +1Kv-Cache ManagementCache

  41. A Token/KV-Cache Communication Media Selection and Resource Allocation Strategy for Multi-Agent Collaboration

    May 25, 2026Lipeng Dai, Luping Xiang, Kun YangPower AllocationKv-Cache Management

    State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, 210008, China · Institute of Intelligent Networks and Communications (NINE), Nanjing University (Suzhou Campus), Suzhou, 215163, China

  42. CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM

    May 24, 2026Yubo Li, Yidi MiaoKv-Cache ManagementKey-Value Cache Eviction

    Carnegie Mellon University

  43. KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference

    May 18, 2026Jian Lin, Jiazhi Mi, Zicong Hong +5Kv-Cache ManagementKey-Value Cache

    Hong Kong University of Science and Technology, China · Xi’an Jiaotong University, China

  44. Head Forcing: Long Autoregressive Video Generation via Head Heterogeneity

    May 14, 2026Jiahao Tian, Yiwei Wang, Gang Yu +1Autoregressive Video Diffusion ModelsAutoregressive Video Generation

    AGI Lab, Westlake University · University of California at Merced · StepFun

  45. KV-Fold: One-Step KV-Cache Recurrence for Long-Context Inference

    May 12, 2026Alireza Nadali, Patrick Cooper, Ashutosh Trivedi +1Efficient Long-Context InferenceKv-Cache Management

    Department of Computer Science University of Colorado Boulder Boulder, Colorado, 80301

  46. Nectar: Neural Estimation of Cached-Token Attention via Regression

    May 10, 2026João Monteiro, Michal Klein, Pierre Ablin +1Efficient Long-Context InferenceKv-Cache Management

    Apple

  47. KV-RM: Regularizing KV-Cache Movement for Static-Graph LLM Serving

    May 10, 2026Zhiqing Zhong, Zhijing Ye, Jian Zhang +3Kv-Cache ManagementLarge Language Model Decoding

    Stevens Institute of Technology Hoboken, New Jersey, USA · The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, Guangdong, China · Argonne National Laboratory Lemont, Illinois, USA +1

  48. PEEK: Predictive Queue-Informed KV Cache Management for LLM Serving

    May 10, 2026Bing Xie, Zhipeng Wang, Masahiro Tanaka +1Kv-Cache ManagementEviction

    †Corresponding author.

  49. PRISM: Fast Online LLM Serving via Scheduling-Memory Co-design

    May 9, 2026Xingyu Qu, Tianhao Lin, Yiqi Li +2Large Language Model ServingKv-Cache Management

    School of Computer Science, Wuhan University · Amazon

  50. Efficient Serving for Dynamic Agent Workflows with Prediction-based KV-Cache Management

    May 7, 2026Haoyu Zheng, Fangcheng Fu, Jia Wu +6Kv-Cache ManagementWorkload

    Wuhan University · Shanghai Jiao Tong University · Macquarie University +2

  51. Shallow Prefill, Deep Decoding: Efficient Long-Context Inference via Layer-Asymmetric KV Visibility

    May 7, 2026Jungsuk Oh, Hyeseo Jeon, Hyunjune Ji +2Efficient Long-Context InferencePrefill

    Graduate School of Data Science Seoul National University Seoul, Republic of Korea

  52. Requests of a Feather Must Flock Together: Batch Size vs. Prefix Homogeneity in LLM Inference

    May 7, 2026Saksham Rathi, Preeti, Mythili VutukuruLLM Inference OptimizationKv-Cache Management

    Indian Institute of Technology Bombay

  53. When Hidden States Drift: Can KV Caches Rescue Long-Range Speculative Decoding?

    Apr 29, 2026Tianyu Liu, Yuhao Shen, Xinyi Hu +8Speculative DecodingKv-Cache Management

    1Qwen Applications Business Group of Alibaba · University of Science and Technology of China · 3Zhejiang University

  54. The Recurrent Transformer: Greater Effective Depth and Efficient Decoding

    Apr 23, 2026Costin-Andrei Oncescu, Depen Morwani, Samy Jelassi +3Transformer ArchitecturesKv-Cache Management

    Harvard University

  55. Latent Phase-Shift Rollback: Inference-Time Error Correction via Residual Stream Monitoring and KV-Cache Steering

    Apr 20, 2026Manan Gupta, Dhruv KumarCorrectionReasoning Errors

    1BITS Pilani, Pilani Campus, India

  56. SLO-Guard: Crash-Aware, Budget-Consistent Autotuning for SLO-Constrained LLM Serving

    Apr 19, 2026Christian LysenstøenLarge Language Model ServingCrashes

    Inland Norway University of Applied Sciences · Visiting Student, University of California, Berkeley

  57. Predictive Multi-Tier Memory Management for KV Cache in Large-Scale GPU Inference

    Apr 19, 2026Sanjeev Rao GanjihalKv-Cache ManagementKey-Value Cache

    Independent Researcher

  58. Bit-Flip Vulnerability of Shared KV-Cache Blocks in LLM Serving Systems

    Apr 19, 2026Yuji Yamamoto, Satoshi MatsuuraKv-Cache ManagementLarge Language Model Serving

    Institute of Science Tokyo, Tokyo, Japan

  59. KV Cache Offloading for Context-Intensive Tasks

    Apr 9, 2026Andrey Bocharnikov, Ivan Ermakov, Denis Kuznedelev +2Kv-Cache ManagementEfficient Long-Context Inference

    HSE, Yandex · Yandex · Yandex, NSU

  60. Near-Oracle KV Selection via Pre-hoc Sparsity for Long-Context Inference

    Feb 9, 2026Yifei Gao, Lei Wang, Rong-Cheng Tu +3Dynamic Sparse AttentionLLM Inference Optimization

    University College London, London, UK · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences (CAS), Shenzhen 518055, China · The Chinese University of Hong Kong +1