Key-Value Cache Compression

Ask about this topic

Momentum

15 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 110

All topics
CardsList

  1. Behavior-Preserving KV Cache Compression

    Oct 5, 2026Doo Hwan Hwang, Junyoung Jang, Junho Na +2Key-Value Cache Compression

    Kim Jaechul Graduate School of AI, KAIST, Daejeon, Republic of Korea · KT Corporation, Seoul, Republic of Korea

  2. DeferKV: Rethinking Eviction Timing for One-Shot KV Cache Compression

    Oct 5, 2026Zhe Wang, Jiakai Li, Yujia Sun +2Key-Value Cache Compression

    University of Electronic Science and Technology of China, Chengdu, China · Ubiquitous Intelligence and Trusted Services Key Laboratory of Sichuan Province, Chengdu, China

  3. Spend Bytes on Breadth: Precision-Count Trade-offs for Decode-Time KV Compression in Long Chain-of-Thought Reasoning

    Oct 5, 2026Runguo LiKey-Value Cache CompressionBounded-Memory

    University of Illinois Urbana-Champaign

  4. EchoPress: Query-Agnostic KV Cache Pruning via Virtual Context Reconstruction

    Sep 30, 2026Jiawei Lin, Saibo Geng, Thomas BourgeatKey-Value Cache CompressionKeys And Value

    EPFL

  5. PatchKV: Weight-Space Compensation of KV Cache

    Sep 30, 2026Chanryeol Lee, Chanhyuk Lee, Yeonwoo Choi +2Key-Value Cache CompressionKey-Value Cache

    KAIST

  6. KV-Kaizen: Learning Context-Adaptive Cache Compression Choices

    Sep 29, 2026Joao Monteiro, Louis Béthune, Anastasiia Filippova +3Key-Value Cache CompressionLarge Language Model Compression

    Apple

  7. ARC-KV: Amortizing Anchor Search for Reconstruction-Based KV Cache Compaction

    Sep 29, 2026Zheyu Shen, Guanhua Wang, Dezhan Tu +5Key-Value Cache CompressionCache

    University of Maryland, College Park · Meta

  8. Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression

    Sep 28, 2026Xingyu Zhu, Pu, Yi +6Key-Value Cache CompressionData Compression Methods

    Luke · ByteDance Seed · University of California, Berkeley

  9. TORQUE: Optimizing What (not) to Quantize Before and After Rotation

    Sep 28, 2026Ran Ben Basat, Michael Mitzenmacher, Shay VargaftikKey-Value Cache CompressionTop-K

    University College London · Harvard University · VMware Research by Broadcom

  10. Cartridges++: KV Cache Compression without Off-Context Derailment

    Sep 28, 2026Sonia Laguna, Joao Monteiro, Marco Cuturi +2Key-Value Cache CompressionEfficient Long-Context Inference

    Apple

  11. When to Evict, Not What to Keep: Draft-Guided Eviction for Training-Free KV-Cache Compression

    Sep 27, 2026Haeyong Kang, Chang D. YooEvictionKey-Value Cache Compression

    Duksung Women’s University · KAIST

  12. MILO: Efficient Many-shot In-Context Learning with Block-wise Low-rank Compression

    Sep 24, 2026Youpeng Zhao, Tian Tan, Liqian Peng +2Large Language Model CompressionKey-Value Cache Compression

    Google · University of Central Florida

  13. Tensor Decomposition of Transformer Key-Value Caches: Spectral Structure and Format Comparison

    Sep 23, 2026Rahul Krishnan, Volker SchulzKey-Value Cache CompressionTensor Decomposition

    Universität Trier · Fachbereich IV, Mathematik, Universität Trier

  14. Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference

    Sep 22, 2026Md Mostafizer Rahman, Md Faizul Ibne Amin, Md Shahajada Mia +2LLM Inference OptimizationLarge Language Model Memory

    Lucy Family Institute for Data & Society, University of Notre Dame, IN, USA · The University of Aizu, Aizuwakamatsu, Japan · Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, IN, USA

  15. KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation

    Sep 21, 2026Sihyeon Ha, Jaeho Lee, Yo-Seb JeonKey-Value Cache CompressionCache

    Pohang University of Science and Technology (POSTECH)

  16. DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

    Sep 17, 2026DeepSeek-AI, :, Anyi Xu +590Key-Value Cache CompressionKey-Value Cache

  17. LLM Inference in a Flash!

    Sep 14, 2026Sebastian Zhao, Minseo Kim, Coleman Hooper +5LLM Inference OptimizationInference-Time

    UC Berkeley · ICSI · LBNL

  18. MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference

    Sep 7, 2026Michael Wang, Keith Li, Roozbeh BostandoostKey-Value Cache CompressionLLM Inference Optimization

    Lake Washington School District · XSchool · UMass Amherst

  19. What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservation

    Sep 3, 2026Bo Zeng, Yu Zhao, Yefeng Liu +3Key-Value Cache CompressionKey-Value Cache Eviction

    Ant Group · Alibaba Group · Universität Hamburg

  20. Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning

    Sep 3, 2026Heng Wang, Jielin Qiu, Wenting Zhao +8Key-Value Cache EvictionKey-Value Cache

    Salesforce AI Research · University of Illinois Urbana-Champaign

  21. SGD-KV: Summarization Guided KV Cache Compression

    Sep 3, 2026Zeyu Liu, Woomin Song, Xuandi Fu +5Key-Value Cache CompressionEfficient Long-Context Inference

    USC, USA · KAIST, Korea · Amazon AGI, USA

  22. Budget-Aware Compression Pipeline for Single-GPU LLM Inference: Methods, Trade-offs, and Coupling Effects

    Aug 30, 2026Hongyu Yu, Yifei ShenLarge Language Model CompressionLLM Inference Optimization

    Lenovo Research · University of Washington

  23. StepKV: Step-Aware KV Cache Compression for LLM Agents

    Aug 26, 2026Boyu Feng, Jiahong Liu, Yifan Li +7Key-Value Cache CompressionCache

    The Chinese University of Hong Kong · Huawei Technologies Co., Ltd

  24. KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Models

    Aug 10, 2026Chen Qiu, Ziwu Liu, Chao Fei +2Key-Value Cache CompressionDiagnostic Benchmark

    KAUST Thuwal, Saudi Arabia

  25. RippleKV: Cross-Layer KV Cache Allocation via Perturbation Propagation

    Aug 9, 2026Dongjie Xu, Kai Qian, Julius +6Key-Value Cache CompressionKey-Value Cache

    1Soochow University · 2Tencent Inc. · 3The Hong Kong University of Science and Technology +2

  26. VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference

    Aug 9, 2026Wenxu Jia, Dongjie Fu, Xize Cheng +6Key-Value Cache CompressionAudio Tokenizers

    1Zhejiang University · 2Meituan

  27. RotaryQuant: Fitting 120B MoE Models on Consumer Hardware via Fused Compressed-Space Attention

    Aug 8, 2026Anthony. Lui, Mohamed. Elsaied, N. P. SavaniMixture-Of-ExpertsKey-Value Cache Compression

    Cognizant AI & Analytics, London, UB8 3PH, UK · Southern Methodist University (SMU), Engineering Management, Lyle School of Engineering, Dallas, TX, USA · Cognizant AI Lab, Responsible AI Office, San Francisco, USA +1

  28. CommitKV: Lifecycle-Aware KV Cache Compression via Commit Transitions for Multi-Turn Agents

    Aug 8, 2026Weizhong Huang, Jinchao Zhang, Xiawu ZhengKey-Value Cache CompressionDepthweave-Kv

    Xiamen University · WeChat AI, Tencent Inc., China

  29. Every Cache Entry Earns Its Place: Global Allocation of Resolution and Coverage for KV Cache Compression

    Aug 7, 2026Haolin Tian, Yuzhe Liu, Tonghan WangKey-Value Cache CompressionCache

    Tsinghua University Beijing, China

  30. Fewer Tokens, Smaller Cache: Reward-Coordinated Efficient Reasoning

    Aug 5, 2026Qiyuan Zhu, Dezhi Li, Pengyu Cheng +8Large Reasoning ModelsKey-Value Cache Compression

    1HKUST · 2XJTU · 3TSE +1

  31. TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning

    Aug 4, 2026Wonpyo Park, Seung-won HwangKey-Value Cache CompressionKey-Value Cache Eviction

    Seoul National University · Google

  32. AnchorKV: Anchor-Residual KV Cache Compression

    Aug 3, 2026Malik Khalaf, Yara Shamshoum, Nitzan Hodos +2Key-Value Cache CompressionKey-Value Cache

    Department of Computer Science, Technion - Israel Institute of Technology

  33. Does Accuracy Equal Evidence? Reasoning Faithfulness under KV Cache Compression

    Aug 3, 2026Mengting Ai, Jingrui He, Yue GuoKey-Value Cache CompressionExplanation Faithfulness

    University of Illinois Urbana-Champaign

  34. RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction

    Aug 2, 2026Changwoo Baek, Seungjun Shin, Kyeongbo KongKey-Value Cache CompressionKey-Value Cache Eviction

    Pusan National University · Sookmyung Women’s University

  35. Practical Online KV Cache Compaction for LLM Agents: An Empirical Study

    Aug 2, 2026Yujian Liu, Jiabao Ji, Li An +4Key-Value Cache CompressionCompaction

    UC Santa Barbara · LinkedIn

  36. S4^4R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Caching

    Aug 1, 2026Jialong Han, You Wu, Kewei TuKey-Value Cache CompressionKey-Value Cache

    School of Information Science and Technology, ShanghaiTech University · Shanghai Engineering Research Center of Intelligent Vision and Imaging

  37. ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression

    Jul 31, 2026Yuhang Zhan, Lisi Chen, Shuo ShangKey-Value Cache CompressionCache

    University of Electronic Science and Technology of China

  38. LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decoding

    Jul 27, 2026Junsung HwangEfficient Long-Context InferenceKey-Value Cache Compression

  39. DynaCalKV: Key-Value Cache Compression via Head Grouping and Adaptive Rank Allocation

    Jul 27, 2026Tan T. Nguyen, Quan V. DangKey-Value Cache CompressionCache

    ∗Full Stack Data Science · †Department of Computer Science, University College London

  40. Cache-Aware Prompt Compression:A Two-Tier Cost Model for LLM API Caching

    Jul 17, 2026Yan SongKey-Value Cache CompressionLarge Language Model Compression

    PayPal Inc.

  41. A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocation

    Jul 14, 2026Rahul Krishnan, Volker SchulzKey-Value Cache CompressionKey-Value Cache

    Universität Trier · Fachbereich IV, Mathematik, Universität Trier

  42. How Query Visibility Changes KV-Cache Compression Rankings: A Matched-Budget Audit

    Jul 11, 2026Daming Luo, Christy Liang, Junyu XuanKey-Value Cache CompressionData Compression Methods

    University of Technology Sydney

  43. DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression

    Jul 7, 2026Anna Cordoba, Adam Puente Tercero, Nerea Angulo Hijo +4Key-Value Cache CompressionDepthweave-Kv

    Instituto de Investigación en Visión Artificial

  44. FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference

    Jul 7, 2026Anna Córdoba, Adam Puente Tercero, Nerea Angulo Hijo +4Key-Value Cache CompressionDepthweave-Kv

    Instituto de Investigación en Visión Artificial

  45. KVpop -- Key-Value Cache Compression with Predictive Online Pruning

    Jul 6, 2026Lukas Hauzenberger, Niklas Schmidinger, Anamaria-Roberta Hartl +5Key-Value Cache EvictionKey-Value Cache Compression

    1NXAI · 2Johannes Kepler University Linz, Austria

  46. The risk of KV cache compression

    Jul 1, 2026Lukas Haverbeck, Carmen Amo Alonso, Andres Felipe Posada-Moreno +2Key-Value Cache CompressionCache

    RWTH Aachen University · Stanford University

  47. MosaicKV: Serving Long-Context LLM with Dynamic Two-D KV Cache Compression

    Jul 1, 2026Sheng Qiang, Ruiwei Chen, Yinpeng Wu +5Key-Value Cache CompressionLarge Language Model Compression

    Institute of Parallel and Distributed Systems, Shanghai Jiao Tong University, Shanghai, China

  48. Towards Memory-Efficient Autoregressive Video Generation via Instance-Specific Parametric Absorption

    Jul 1, 2026Xiaomeng Fu, Jia Li, Yiming Hu +5Autoregressive Video GenerationKey-Value Cache Compression

    Institute of Information Engineering, Chinese Academy of Sciences · AMAP, Alibaba Group · The University of Hong Kong

  49. SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference

    Jun 30, 2026Amirhossein Abaskohi, Giuseppe Carenini, Peter West +1Key-Value Cache CompressionKey-Value Cache

    University of British Columbia · Microsoft Research

  50. HARD-KV: Head-Adaptive Regularization for Decoding-time KV Compression

    Jun 27, 2026Yuxuan Yang, Feiyang Ren, Bowen Zeng +4Key-Value Cache CompressionLLM Inference Optimization

    The State Key Laboratory of Blockchain and Data Security, Zhejiang University

  51. Information-Aware KV Cache Compression for Long Reasoning

    Jun 25, 2026Jushi Kai, Zhuiri Xiao, Alexandra Birch +1Key-Value Cache CompressionEfficient Long-Context Inference

    LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University · Shanghai Jiao Tong University · School of Informatics, University of Edinburgh

  52. CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

    Jun 23, 2026Xiaolin Lin, Jingcun Wang, Olga Kondrateva +3Key-Value Cache CompressionLLM Inference Optimization

    Technical University of Darmstadt, Darmstadt, Germany · University of Notre Dame, Notre Dame, IN, USA · Technical University of Ilmenau, Ilmenau, Germany

  53. ViCoStream: Streaming VideoLLMs Can Run Beyond 100 FPS with Stage-Wise Coordinated Inference

    Jun 18, 2026Yang Tan, Junlong Tong, Linan Yue +3Streaming VideoStreaming

    Southeast University · Eastern Institute of Technology, Ningbo · Shanghai Jiao Tong University

  54. AnchorKV: Safety-Aware KV Cache Compression via Soft Penalty with a Refusal Anchor

    Jun 16, 2026Ning Ni, Yingjie LaoKey-Value Cache CompressionLarge Language Model Safety

    Department of Computer Science Tufts University · Department of Electrical and Computer Engineering Tufts University

  55. Ablation, Statistical Inference, and Validation for KV-Cache Compression

    Jun 14, 2026Paolo D'Alberto, Ashish Siarasao, Elliott Delaye +1Key-Value Cache CompressionResidual Vector Quantization

    Advanced Micro Devices, Inc.

  56. Exploring a Layer-Wise Design Space for KV Cache Eviction

    Jun 13, 2026Chao Fei, Kaihua Liang, Hanzhi Hu +4Key-Value Cache CompressionKey-Value Cache

    King Abdullah University of Science and Technology · Fudan University

  57. Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compression

    Jun 10, 2026Tianhao Chen, Yuheng Wu, Kelu Yao +3Key-Value Cache CompressionMultimodal Large Language Models

    KAIST · Zhejiang Laboratory · The Chinese University of Hong Kong +1

  58. From Rigid to Dynamic: Entropy-Guided Adaptive Inference for Long-Context LLMs

    Jun 8, 2026Zhanchao Xu, Haoyang Li, Qingfa Xiao +4LLM Inference OptimizationDynamic Sparse Attention

    Department of Computing, PolyU · DSA, HKUST(GZ) · CSE, HKUST

  59. EinSort: Sorting is All We Need for Tensorizing LLM

    Jun 7, 2026Toshiaki Koike-Akino, Jing Liu, Ye WangTensor NetworksLow-Rank Structure

    Mitsubishi Electric Research Laboratories (MERL), 20 Broadway, Cambridge, MA 02139, USA.

  60. STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control

    Jun 7, 2026Priyansh Bhatnagar, Ashkan Moradifirouzabadi, Se-Hyun Yang +3Key-Value Cache CompressionKey-Value Cache

    University of California San Diego, San Diego, CA, USA · Dnotitia Inc., Seoul, Republic of Korea · Hanyang University, Seoul, Republic of Korea