KV Caching

KV: Key-Value

Follow this topic Ask about this topic

Momentum

29 papers in the last four weeks, up 16% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 276

All topics

Moment-KV: Momentum-Based Decode-Time KV Cache Compression for Long Generation
May 28, 2026Soumyadeep Jana, Sagar Nishad, Sanasam Ranbir SinghKV Caching Long-Context Language Modeling
Indian Institute of Technology Guwahati, India
Augmenting Attention with Exponentially Decaying Memory Improves Query-Aware KV Sparsity
May 27, 2026Xiuying Wei, Caglar GulcehreMemory-Augmented Language Models Self-Attention
EPFL, CLAIRE Lausanne, Switzerland
Hurwitz Quaternion Multiplicative Quantization for KV Cache Compression
May 26, 2026Kabir Swain, Sijie Han, Daniel Karl I. Weidele +3LLM Quantization KV Caching
Massachusetts Institute of Technology, Cambridge, MA, USA · University of Toronto, Toronto, Canada · IBM Research, Cambridge, MA, USA.
NestedKV: Nested Memory Routing for Long-Context KV Cache Compression
May 26, 2026Hong Chen, Xiang Liu, Yubo Gao +5KV Caching Long-Context Language Modeling
The Hong Kong University of Science and Technology (Guangzhou) · Jimei University
Enabling KV Caching of Shared Prefix for Diffusion Language Models
May 26, 2026Younghun Go, Jaehoon Han, Changyong Shin +2KV Caching KV-Cache Management
Korea University, South Korea
Quantized Keys Steal Attention: Bias Correction for KV-Cache Compression in Video Diffusion
May 25, 2026Tuna Tuncer, Felix Becker, Thomas PfeilVideo Diffusion Models Self-Attention
1Technical University of Munich · 2Tensordyne
IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference
May 25, 2026Xintong Yang, Hao Gu, Binxing Xu +6Memory-Augmented Language Models Self-Attention
The Hong Kong University of Science and Technology · Zhejiang University
Polynomial Context-Truncation Sensitivity in Autoregressive Language Models: Sequential Wyner-Ziv Bounds for KV Cache Compression
May 24, 2026Munsik KimKV Caching Rate-Distortion Theory
Independent Researcher.
CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM
May 24, 2026Yubo Li, Yidi MiaoKV Caching Long-Context Language Model Inference
Carnegie Mellon University
A Simple Plug-in for Improving Eviction-Based KV Cache Compression
May 22, 2026Yuping Lin, Jiayuan Ding, Yue Xing +3KV Caching Memory-Efficient Optimization
1Michigan State University · 2Hippocratic AI
Adaptive Mass-Segmented KV Compression for Long-Context Reasoning
May 22, 2026Junzhe Yang, Xiaoyu ShenKV Caching Long-Context Language Model Inference
Shanghai Jiao Tong University · Institute of Digital Twin, Eastern Institute of Technology, Ningbo
WorldKV: Efficient World Memory with World Retrieval and Compression
May 21, 2026Jung Yi, Minjae Kim, Paul Hyunbin Cho +3KV Caching Memory-Augmented Video Generation
1KAIST AI · 2Naver AI Lab
Asymmetric Virtual Memory Paging for Hybrid Mamba-Transformer Inference
May 21, 2026An Xuan NguyenKV Caching Memory-Efficient Optimization
Codepawl Ho Chi Minh City, Vietnam
Meta-Soft: Leveraging Composable Meta-Tokens for Context-Preserving KV Cache Compression
May 21, 2026Wei Luo, Yi Huang, Songchen Ma +3KV Caching KV-Cache Eviction
Guangdong Institute of Intelligence Science and Technology · University of Macau · Chinese University of Hong Kong, Shenzhen +1
MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering
May 21, 2026Junbin Xiao, Jiajun Chen, Tianxiang Sun +2KV Caching Video QA
University of Science and Technology of China · National University of Singapore
ArborKV: Structure-Aware KV Cache Management for Scaling Tree-based LLM Reasoning
May 21, 2026Yeqiu Chen, Ziyan Liu, Zhenxin Huang +3Inference-Time Search Tree of Thoughts
University of Science and Technology of China
Tensor Cache: Eviction-conditioned Associative Memory for Transformers
May 21, 2026Kabir Swain, Sijie Han, Daniel Karl I. Weidele +2Memory-Augmented Neural Networks Self-Attention
Massachusetts Institute of Technology, Cambridge, MA, USA · University of Toronto, Toronto, Canada · IBM Research, Cambridge, MA, USA
EntmaxKV: Support-Aware Decoding for Entmax Attention
May 20, 2026Gonçalo Duarte, Miguel Couceiro, Marcos V. TrevisoSelf-Attention KV Caching
Instituto Superior Técnico, Universidade de Lisboa. · ELLIS Unit Lisbon. · INESC-ID. +1
OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization
May 20, 2026Mark Boss, Vikram Voleti, Simon Donné +1KV Caching Memory-Efficient Optimization
Stability AI
Runtime-Certified Bounded-Error Quantized Attention
May 20, 2026Dean CalverLLM Inference KV Caching
Independent Researcher
Head-Aware Key-Value Compression for Efficient Autoregressive Image Generation
May 20, 2026Guotao Liang, Baoquan Zhang, Zhiyuan Wen +1KV Caching Autoregressive Image Generation
Harbin Institute of Technology, Shenzhen · Peng Cheng Laboratory
Latent Cache Flow: Model-to-Model Communication Without Text
May 19, 2026Maximillian Rossi, Prajwal Raghunath, Eugene WuKV Caching KV-Cache Compression
Columbia University
OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond
May 19, 2026Zunhai Su, Rui Yang, Chao Zhang +11KV Caching Rotation-Based Quantization
Tsinghua University · Meituan LongCat Team · The University of Hong Kong +3
Focused Forcing: Content-Aware Per-Frame KV Selection for Efficient Autoregressive Video Diffusion
May 18, 2026Peiliang Cai, Evelyn Zhang, Jiacheng Liu +8Video Diffusion Models Autoregressive Diffusion
1SJTU · 2SDU · 3HUST +4
KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference
May 18, 2026Jian Lin, Jiazhi Mi, Zicong Hong +5KV-Cache Offloading KV Caching
Hong Kong University of Science and Technology, China · Xi’an Jiaotong University, China
Protection Is (Nearly) All You Need: Structural Protection Dominates Scoring in Globally Capped KV Eviction
May 18, 2026Gabriel GarciaKV Caching KV-Cache Eviction
Independent Researcher
VeriCache: Turning Lossy KV Cache into Lossless LLM Inference
May 17, 2026Jiayi Yao, Samuel Shen, Kuntai Du +7KV-Cache Offloading KV Caching
University of Chicago · Tensormesh Inc. · Samsung Semiconductor +1
ObjectCache: Layerwise Object-Storage Retrieval for KV Cache Reuse
May 16, 2026Yu Zhu, Aditya Dhakal, Yunming Xiao +2LLM Inference Efficiency KV-Cache Offloading
ETH Zurich Switzerland · HPE Labs United States of America · The Chinese University of Hong Kong, Shenzhen China
GHOST: Geometry-Hierarchical Online Streaming Token Eviction for Efficient 3D Reconstruction
May 15, 2026Leyang Chen, Junyi Wu, Zhiteng Li +1Streaming 3D Reconstruction KV Caching
Shanghai Jiao Tong University
HeatKV: Head-tuned KV-cache Compression for Visual Autoregressive Modeling
May 14, 2026Jonathan Cederlund, Axel Berg, William Isaksson +3KV Caching Visual Autoregressive Models
Dept. of Automatic Control, Lund University · Arm

May 28, 2026, cs.AIJ/K move · Enter open · S save

Moment-KV: Momentum-Based Decode-Time KV Cache Compression for Long Generation

Soumyadeep Jana, Sagar Nishad, Sanasam Ranbir Singh

Indian Institute of Technology Guwahati, India

KV Caching Long-Context Language Modeling KV-Cache Eviction KV-Cache Compression

Open paper AskPDF arXiv

Key-Value (KV) cache remains a major bottleneck for deploying Large Language Models (LLMs) in long-generation tasks. Prior work often applies uniform compression across both prefill and decoding caches, but compressing the prefill cache degrades performance by corrupting critical context. While preserving the prefill cache is essential, decoding-phase compression remains underexplored, with existing methods relying on rigid recency windows or instantaneous attention. Our analysis of attention dynamics reveals strong temporal patterns: critical tokens receive sustained attention over long horizons, while local reasoning involves short-lived bursts. Static heuristics fail to capture this behavior, leading to premature eviction of important tokens or retention of stale ones. We propose Moment-KV, a decoding-time KV cache compression method based on momentum-driven temporal attention aggregation. Our method models token importance as a continuously evolving state, where attention is aggregated with decay, capturing both long-term influence and recent relevance. Experiments show that Moment-KV significantly improves generation fidelity in long-generation tasks (2.3-3.2 %) while maintaining decoding latency.

Momentum

Related topics

Latest papers 276