KV-Cache Compression

Follow this topic Ask about this topic

Momentum

24 papers in the last four weeks, up 71% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 166

All topics

Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
Sep 8, 2026Vishesh Tripathi, Abhay Kumar, Ramsha KhanEfficient Transformer Inference Grouped-Query Attention
FrontiersMind
Jacap: Robust KV Cache Eviction via Jacobian-Based Nonlinear Information Capacity Preservation
Sep 8, 2026Jiaming Yang, Chenwei Tang, Liangli Zhen +2KV Caching KV-Cache Eviction
College of Computer Science, Sichuan University, Chengdu, China · 2Engineering Research Center of Machine Learning and Industry Intelligence, Ministry of Education, Chengdu, China · Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore
MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference
Sep 7, 2026Michael Wang, Keith Li, Roozbeh BostandoostLLM Inference Efficiency LLM Inference
Lake Washington School District · XSchool · UMass Amherst
VestigeKV: The NoPE-MLA KV Cache Carries Its Own Sparse-Attention Signal in a Vestigial Branch
Sep 3, 2026WenJie FanKV Caching KV-Cache Management
Yotta Labs
What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservation
Sep 3, 2026Bo Zeng, Yu Zhao, Yefeng Liu +3KV Caching KV-Cache Eviction
Ant Group · Alibaba Group · Universität Hamburg
GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving
Sep 3, 2026Qiankun Ma, Yanjiang Zhou, Zinan Xiong +5LLM Inference KV Caching
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · Pengcheng Laboratory · University of Chinese Academy of Sciences
Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
Sep 3, 2026Heng Wang, Jielin Qiu, Wenting Zhao +8KV Caching Memory-Efficient Optimization
Salesforce AI Research · University of Illinois Urbana-Champaign
SGD-KV: Summarization Guided KV Cache Compression
Sep 3, 2026Zeyu Liu, Woomin Song, Xuandi Fu +5KV Caching Long-Context Language Model Inference
USC, USA · KAIST, Korea · Amazon AGI, USA
HeadWiseKV: Budgeted Per-Head Cache Residency for Hybrid Long-Context Language Models
Sep 2, 2026Renjie Xie, Juncheng Yang, Aoting Hu +4KV Caching Long-Context Language Model Inference
Nanjing University of Posts and Telecommunications · Tylogi AI Lab / TAIL · Anhui University of Technology +2
CateKV: On Sequential Consistency for Long-Context LLM Inference Acceleration
Aug 31, 2026Haoyun Jiang, Haolin Li, Jianwei Zhang +7Long-Context Language Model Inference LLM Inference Acceleration
1CMIC, Shanghai Jiao Tong University · 2Alibaba Group · 3Fudan University
Strong Drafts Need Compact Memories: Long-Context Speculative Decoding with Compressed KV Cache
Aug 31, 2026Tong Yuan, Chengxi Liao, Zeyi WenMemory-Augmented Language Models KV Caching
Data Science and Analytics Thrust, Information Hub, The Hong Kong University of Science and Technology (Guangzhou)
Budget-Aware Compression Pipeline for Single-GPU LLM Inference: Methods, Trade-offs, and Coupling Effects
Aug 30, 2026Hongyu Yu, Yifei ShenLLM Quantization LLM Pruning
Lenovo Research · University of Washington
Compression-Aware Abstention: Teaching LLMs to Refuse When KV-Compression Masks Remove Answer Evidence
Aug 30, 2026Mohammadali Khodabandehlou, Bhaskar KrishnamachariLLM Hallucination Mitigation KV Caching
University of Southern California
Small Frequency Corrections Can Change What Survives KV Cache Compression
Aug 27, 2026Hong Chen, Yudong Zeng, Yongwei Huang +5KV Caching KV-Cache Eviction
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Bosum Institute of Management Science, Shenzhen, China
StepKV: Step-Aware KV Cache Compression for LLM Agents
Aug 26, 2026Boyu Feng, Jiahong Liu, Yifan Li +7KV Caching KV-Cache Compression
The Chinese University of Hong Kong · Huawei Technologies Co., Ltd
Learning how to Forget: Fine-tuning for Long-Context Sparse Attention
Aug 20, 2026Matthias Seeger, Zeyu Zhang, Vihang Patil +2Fine-Tuning KV Caching
Amazon Web Services · University of Amsterdam · Amazon +1
KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Models
Aug 10, 2026Chen Qiu, Ziwu Liu, Chao Fei +2Long-Context Language Model Evaluation KV Caching
KAUST Thuwal, Saudi Arabia
Deferred Audio Pruning with Local Audio-Visual Dynamics for Omni-LLMs
Aug 9, 2026Kyeongyoon Lee, Hongyeob Kim, Youngeun Kim +1Efficient Multimodal Inference Audio Token Compression
Sungkyunkwan University · AWS AI Labs
RippleKV: Cross-Layer KV Cache Allocation via Perturbation Propagation
Aug 9, 2026Dongjie Xu, Kai Qian, Julius +6KV Caching KV-Cache Management
1Soochow University · 2Tencent Inc. · 3The Hong Kong University of Science and Technology +2
VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference
Aug 9, 2026Wenxu Jia, Dongjie Fu, Xize Cheng +6Audio Token Compression LLM Inference Acceleration
1Zhejiang University · 2Meituan
SPECTRA: Pushing the KV Cache Beyond the 2-Bit Cliff via Spectral Transform Coding
Aug 8, 2026Jiamu Zhang, Liang Wu, Kelly Wan +2Mixed-Precision Quantization KV Caching
Nokia · Rice University
CommitKV: Lifecycle-Aware KV Cache Compression via Commit Transitions for Multi-Turn Agents
Aug 8, 2026Weizhong Huang, Jinchao Zhang, Xiawu ZhengKV Caching LLM Agent Memory
Xiamen University · WeChat AI, Tencent Inc., China
Every Cache Entry Earns Its Place: Global Allocation of Resolution and Coverage for KV Cache Compression
Aug 7, 2026Haolin Tian, Yuzhe Liu, Tonghan WangKV Caching KV-Cache Management
Tsinghua University Beijing, China
Runtime Observability for Heterogeneous Attention Memory
Aug 6, 2026Fanzhe Wei, Li Liu, Ziyang Wang +1KV Caching LLM Auditing
Metask Lab
Fewer Tokens, Smaller Cache: Reward-Coordinated Efficient Reasoning
Aug 5, 2026Qiyuan Zhu, Dezhi Li, Pengyu Cheng +8Overthinking in Language Models KV Caching
1HKUST · 2XJTU · 3TSE +1
TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning
Aug 4, 2026Wonpyo Park, Seung-won HwangLLM Pruning KV Caching
Seoul National University · Google
SAKI: Score-Aware Low-Rank Key Indexing with Random-Matrix Noise Correction for KV Retrieval
Aug 4, 2026Lin ZhangLow-Rank Attention KV Caching
Data Science and Platform, Cotality
AnchorKV: Anchor-Residual KV Cache Compression
Aug 3, 2026Malik Khalaf, Yara Shamshoum, Nitzan Hodos +2KV Caching Long-Context Language Model Inference
Department of Computer Science, Technion - Israel Institute of Technology
Does Accuracy Equal Evidence? Reasoning Faithfulness under KV Cache Compression
Aug 3, 2026Mengting Ai, Jingrui He, Yue GuoCoT Faithfulness Reasoning Evaluation
University of Illinois Urbana-Champaign
RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction
Aug 2, 2026Changwoo Baek, Seungjun Shin, Kyeongbo KongKV Caching Memory-Efficient Optimization
Pusan National University · Sookmyung Women’s University

Sep 8, 2026, cs.ARJ/K move · Enter open · S save

Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction

Vishesh Tripathi, Abhay Kumar, Ramsha Khan

FrontiersMind

Efficient Transformer Inference Grouped-Query Attention KV Caching Efficient Attention KV-Cache Compression

Open paper AskPDF arXiv

The KV cache is a primary bottleneck for Transformer decoding: its memory footprint and cache-read traffic grow with sequence length. Grouped-query attention (GQA) reduces this cost by sharing key-value heads, but still stores both a key and a value at every step. We introduce Grouped Value Attention (GVA), which stores grouped values and reconstructs content keys with a learned linear map. At inference, the map can be absorbed into the query, eliminating the need to materialize content keys in the intended decode path. A small shared decoupled RoPE channel retains positional information through a separately cached positional key. For the configurations studied, this representation reduces persistent cache scalars by approximately 45-47% relative to matched GQA. At the 350M-parameter scale with 30B FineWeb-Edu tokens, the 16-dimensional positional variant reaches 44.18 average accuracy across five tasks, compared with 44.36 for GQA and 43.88 for MLA. These results demonstrate near-GQA benchmark accuracy with a more compact cache representation. To translate this compact representation into faster autoregressive inference, we have developed custom decoding kernels and are currently evaluating their end-to-end inference performance with an open-source release planned soon.

Momentum

Related topics

Latest papers 166