KV Caching

KV: Key-Value

Follow this topic Ask about this topic

Momentum

29 papers in the last four weeks, up 16% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 276

All topics

CoRDS: Coreset-based Representative and Diverse Selection for Streaming Video Understanding
May 14, 2026Ailar Mahdizadeh, Puria Azadi, Muchen Li +2Streaming Video Understanding KV Caching
University of British Columbia · Vector Institute
Minimal-Intervention KV Retention via Set-Conditioned Diversity
May 14, 2026Libo Sun, Po-wei Harn, Peixiong He +1KV Caching KV-Cache Eviction
Department of Computer Science and Software Engineering Auburn University, Auburn, AL, USA · Department of Information Management National Central University, Taoyuan, Taiwan
Self-Pruned Key-Value Attention: Learning When to Write by Predicting Future Utility
May 13, 2026Gergely Szilvasy, Manuel Faysse, Maria Lomeli +5Self-Attention KV Caching
Meta FAIR · MICS, CentraleSupélec
SPHERICAL KV: Angle-Domain Attention and Rate-Distortion Retention for Efficient Long-Context Inference
May 13, 2026Anay Chauhan, Gurucharan Marthi Krishna Kumar, Arion Das +4KV Caching Long-Context Language Model Inference
1Synopsys, India · 2McGill University, Canada · 3IIIT Ranchi, India +4
Attention Once Is All You Need: Efficient Streaming Inference with Stateful Transformers
May 13, 2026Victor NorgrenTransformer Inference LLM Inference
LayerScale, Inc.
KVServe: Service-Aware KV Cache Compression for Communication-Efficient Disaggregated LLM Serving
May 13, 2026Zedong Liu, Xinyang Ma, Dejun Luo +9LLM Serving KV Caching
University of Chinese Academy of Sciences · Institute of Computing Technology, Chinese Academy of Sciences · Shanghai Jiao Tong University
Not All Tokens Are Worth Caching: Learning Semantic-Aware Eviction for LLM Prefix Caches
May 12, 2026Shaoke Fang, Ziang Li, Wenfei Wu +3LLM Inference Efficiency KV Caching
Peking University · FirestAI · Tsinghua University +2
KV-Fold: One-Step KV-Cache Recurrence for Long-Context Inference
May 12, 2026Alireza Nadali, Patrick Cooper, Ashutosh Trivedi +1Transformer Inference KV Caching
Department of Computer Science University of Colorado Boulder Boulder, Colorado, 80301
FibQuant: Universal Vector Quantization for Random-Access KV-Cache Compression
May 12, 2026Namyoon Lee, Yongjune KimKV Caching Rotation-Based Quantization
POSTECH
Key-Value Means: Transformers with Expandable Block-Recurrent Compressed Memory
May 11, 2026Daniel Goldstein, Eugene CheahTransformer Inference Self-Attention
Featherless AI · Eleuther AI
Nectar: Neural Estimation of Cached-Token Attention via Regression
May 10, 2026João Monteiro, Michal Klein, Pierre Ablin +1Softmax Attention Self-Attention
Apple
KV-RM: Regularizing KV-Cache Movement for Static-Graph LLM Serving
May 10, 2026Zhiqing Zhong, Zhijing Ye, Jian Zhang +3KV Caching KV-Cache Management
Stevens Institute of Technology Hoboken, New Jersey, USA · The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, Guangdong, China · Argonne National Laboratory Lemont, Illinois, USA +1
Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction
May 10, 2026Ngoc Bui, Hieu Trung Nguyen, Arman Cohan +1Self-Attention KV Caching
Department of Computer Science, Yale University · The Chinese University of Hong Kong
Not All Thoughts Need HBM: Semantics-Aware Memory Hierarchy for LLM Reasoning
May 10, 2026Aojie Yuan, Tianqi Shen, Dajun ZhangKV-Cache Offloading KV Caching
University of Southern California · †University of Wisconsin–Madison
PEEK: Predictive Queue-Informed KV Cache Management for LLM Serving
May 10, 2026Bing Xie, Zhipeng Wang, Masahiro Tanaka +1LLM Serving KV Caching
†Corresponding author.
ProxyKV: Cross-Model Proxy Pruning for Efficient Long-Context LLM Inference
May 9, 2026Junjie Li, Jiong Lou, Jie LiLLM Pruning KV Caching
Shanghai Jiao Tong University
Non-Monotonic Latency in Apple MPS Decoding: KV Cache Interactions and Execution Regimes
May 9, 2026Willy Fitra HendriaLLM Inference KV Caching
1Independent Researcher, Seoul, South Korea.
ReST-KV: Robust KV Cache Eviction with Layer-wise Output Reconstruction and Spatial-Temporal Smoothing
May 9, 2026Yongqi An, Chang Lu, Kuan Zhu +5KV Caching Long-Context Language Model Inference
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · University of Electronic Science and Technology of China, Chengdu, China +2
PRISM: Fast Online LLM Serving via Scheduling-Memory Co-design
May 9, 2026Xingyu Qu, Tianhao Lin, Yiqi Li +2LLM Serving KV Caching
School of Computer Science, Wuhan University · Amazon
RDKV: Rate-Distortion Bit Allocation for Joint Eviction and Quantization of the KV Cache
May 8, 2026Junkai Zhang, Hang Guo, Luca Benini +1KV Caching Long-Context Language Model Inference
ETH Zurich · Tsinghua University
Reformulating KV Cache Eviction Problem for Long-Context LLM Inference
May 8, 2026Tho Mai, Joo-Young KimKV Caching Long-Context Language Model Inference
KAIST Daejeon, South Korea
WiCER: Wiki-memory Compile, Evaluate, Refine Iterative Knowledge Compilation for LLM Wiki Systems
May 8, 2026Juan M. HuertaKV Caching Counterexample-Guided Refinement
Zinnia Tech Solutions 600 Steamboat Road Greenwich, CT 06830, USA
Sparse Attention as a Range Searching Problem: Towards an Inference-Efficient Index for KV Cache
May 7, 2026Mohsen Dehghankar, Abolfazl AsudehSelf-Attention KV Caching
Department of Computer Science University of Illinois Chicago
Efficient Serving for Dynamic Agent Workflows with Prediction-based KV-Cache Management
May 7, 2026Haoyu Zheng, Fangcheng Fu, Jia Wu +6LLM Inference Efficiency KV Caching
Wuhan University · Shanghai Jiao Tong University · Macquarie University +2
Towards Distributed Inference of LLMs on a P2P Network
May 7, 2026Shabari S Nair, Krishanu SainiLLM Inference KV Caching
Department of Computer Science, The University of Texas at Austin.
Memory Inception: Latent-Space KV Cache Manipulation for Steering LLMs
May 7, 2026Andy Zeyi Liu, Michael Zhang, Ilana Greenberg +3Language Model Steering Memory-Augmented Language Models
Yale University · Princeton University · Jump Trading
When Quantization Is Free: An int4 KV Cache That Outruns fp16 on Apple Silicon
May 7, 2026Mohamed Amine BergachGPU Kernel Optimization KV Caching
Illumina, San Diego, CA, USA
Irminsul: MLA-Native Position-Independent Caching for Agentic LLM Serving
May 7, 2026Bole Ma, Jan Eitzinger, Harald KöstlerKV Caching LLM Inference Serving
Erlangen National High Performance Computing Center · Erlangen, Germany
When Does Value-Aware KV Eviction Help? A Fixed-Contract Diagnostic for Non-Monotone Cache Compression
May 7, 2026Ruijie Zhang, Haozhe Liang, Da Chang +4KV Caching Long-Context Language Model Inference
One Pool, Two Caches: Adaptive HBM Partitioning for Accelerating Generative Recommender Serving
May 6, 2026Wenjun Yu, Shuguang Han, Amelie Chi ZhouLLM Inference Efficiency GPU Acceleration
Hong Kong Baptist University · Hong Kong, China · Alibaba Inc +1

May 14, 2026, cs.CVJ/K move · Enter open · S save

CoRDS: Coreset-based Representative and Diverse Selection for Streaming Video Understanding

Ailar Mahdizadeh, Puria Azadi, Muchen Li, Xiangteng He+1

University of British Columbia · Vector Institute

Streaming Video Understanding KV Caching Large Vision-Language Models Coreset Selection KV-Cache Compression Feature Selection

Open paper AskPDF arXiv

Streaming video understanding with large vision-language models (VLMs) requires a compact memory that can support future reasoning over an ever-growing visual history. A common solution is to compress the key-value (KV) cache, but existing streaming methods typically rely on local token-wise heuristics, such as recency, temporal redundancy, or saliency, which do not explicitly optimize whether the retained cache is representative of the accumulated history. We propose to view KV-cache compression as a coreset selection problem: rather than scoring tokens independently for retention, we select a small subset that covers the geometry of the accumulated visual cache. Our method operates in a joint KV representation and introduces a bicriteria objective that balances coverage in key and value spaces, preserving both retrieval structure and output-relevant information. To encourage a more diverse retained subset, we further introduce an orthogonality-driven diversity criterion that favors candidates contributing new directions beyond the current selection, and connect this criterion to log-determinant subset selection. Across four open-source VLMs and five long-video and streaming-video benchmarks, our method improves over heuristic streaming compression baselines under a fixed cache budget. These results highlight that representative coreset selection offers a more effective principle, than token-wise pruning, for memory-constrained streaming video understanding.

Momentum

Related topics

Latest papers 276