KV Caching

KV: Key-Value

Follow this topic Ask about this topic

Momentum

29 papers in the last four weeks, up 16% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 276

All topics

FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention
Jun 8, 2026Yan Wang, Qifan Zhang, Jiachen Yu +12KV Caching Long-Context Language Model Inference
Independent Researchers · Tencent · The Hong Kong University of Science and Technology (Guangzhou) +1
RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference
Jun 7, 2026Anirudh SekarLLM Inference Efficiency KV Caching
Accepted to the ICML 2026 Workshop on Statistical Frameworks for Uncertainty in Agentic Systems, Seoul, South Korea, 2026.
SpectrumKV: Per-Token Mixed-Precision KV Cache Transfer for Prefill-Decode Disaggregated LLM Serving
Jun 7, 2026Yang PengjuMixed-Precision Quantization KV Caching
STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control
Jun 7, 2026Priyansh Bhatnagar, Ashkan Moradifirouzabadi, Se-Hyun Yang +3Mixed-Precision Quantization KV Caching
University of California San Diego, San Diego, CA, USA · Dnotitia Inc., Seoul, Republic of Korea · Hanyang University, Seoul, Republic of Korea
HACK++: Towards More Effective Head-Aware Key-Value Compression for Efficient Visual Autoregressive Modeling
Jun 6, 2026Ziran Qin, Yuchen Jiang, Mingbao Lin +4KV Caching Visual Autoregressive Models
Shanghai Jiao Tong University, Shanghai, China · Rakuten, Singapore · Tsinghua University, Beijing, China
IntentKV: Cross-Turn Intent-Aware KV Cache Pruning for Agent Inference
Jun 6, 2026Junjie Li, Jiong Lou, Jie LiKV Caching KV-Cache Management
Shanghai Jiao Tong University
Still: Amortized KV Cache Compaction in a Single Forward Pass
Jun 5, 2026Charles O'Neill, Alex Sandomirsky, Harry Partridge +2KV Caching Long-Context Language Model Inference
Baseten San Francisco · Baseten London
Semantic Cache Distillation: Efficient State Transfer via Reuse and Selective Patching
Jun 5, 2026Qianli Ma, Zhiqing Tang, Hanshuai Cui +2LLM Serving KV Caching
Faculty of Arts and Science, Beijing Normal University, Zhuhai 519087, China · Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai 519087, Guangdong, China · School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China +1
Tangram: Unlocking Non-Uniform KV Cache Compression for Efficient Multi-turn LLM Serving
Jun 4, 2026Hyungmin Kim, Minsoo Kim, Hongseok Kim +1LLM Serving KV Caching
RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention
Jun 4, 2026Yang Liu, ZhaoKai Luo, HuaYi Jin +5LLM Serving KV Caching
†Xiaohongshu Inc., China · ¶Huawei Cloud · §Peking University
QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving
Jun 4, 2026Jianxin Yan, Wangze Ni, Zhenxin Li +8Retrieval-Augmented Generation KV Caching
Zhejiang University, Hangzhou, China · Ant Group, Shanghai, China · The Hong Kong Polytechnic University, Hong Kong, China +5
Rethinking LoRA Memory Through the Lens of KV Cache Compression
Jun 4, 2026Chunsheng Zuo, Liaoyaqi Wang, William Jurayj +2Retrieval-Augmented Generation Memory-Augmented Language Models
Johns Hopkins University
Cartridges at Scale: Training Modular KV Caches over Large Document Collections
Jun 3, 2026Momchil Hardalov, Gonzalo Iglesias, Adrià de GispertMemory-Augmented Language Models KV Caching
Amazon AGI
Streaming Knowledge Compilation: Proactive Materiality-Scored Pinning for Time-Evolving LLM Wikis
Jun 3, 2026Juan M. HuertaMemory-Augmented Language Models Knowledge Augmentation for Language Models
Zinnia Tech Solutions 600 Steamboat Road Greenwich, CT 06830, USA
LazyAttention: Efficient Retrieval-Augmented Generation with Deferred Positional Encoding
Jun 3, 2026Haocheng Xia, Mihir Pamnani, Hanxi Fang +2Retrieval-Augmented Generation KV Caching
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign · Nexla · Amazon +1
Value-Aware Stochastic KV Cache Eviction for Reasoning Models
Jun 2, 2026Ting-Yun Chang, Harvey Yiyun Fu, Deqing Fu +3LLM Inference Efficiency KV Caching
University of Southern California · University of Chicago
NetKV: Network-Aware Decode Instance Selection for Disaggregated LLM Inference
Jun 2, 2026Mubarak Adetunji OjewaleLLM Inference KV Caching
Cloud Competency Centre, National College of Ireland
KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks
Jun 2, 2026Lorenz K. Muller, Philippe Bich, Chiara Boretti +3LLM Quantization KV Caching
Huawei
Multi-Segment Attention: Enabling Efficient KV-Cache Management for Faster Large Language Model Serving
Jun 1, 2026Chunan Shi, Yilei Chen, Yilin Chen +2KV-Cache Offloading LLM Inference
Peking University Beijing, China
STaR-KV: Spatio-Temporal Adaptive Re-weighting for KV Cache Compression in GUI Vision-Language Models
Jun 1, 2026Yuhang Han, Wenzheng Yang, Yujie Chen +4Vision-Language Models KV Caching
1EPIC Lab, SJTU · 2HKUST (GZ) · 3The University of Sydney +3
MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference
Jun 1, 2026Yu Li, Binxu Li, Tian LanKV Caching Long-Context Language Model Inference
George Washington University · Princeton University
Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
Jun 1, 2026Bruce Changlong Xu, Adarsh Kumarappan, Mu ZhouLLM Quantization KV Caching
†Stanford University · ‡California Institute of Technology
Move the Query, Not the Cache: Characterizing Cross-Instance Latent Attention Redistribution Across GPU Fabrics
May 31, 2026Bole Ma, Jan Eitzinger, Harald Köstler +1LLM Inference KV Caching
Leyline: KV Cache Directives for Agentic Inference
May 31, 2026Bole Ma, Jan Eitzinger, Harald KoestlerLLM Inference Efficiency KV Caching
Erlangen National High Performance Computing Center, Erlangen, Germany
GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMs
May 29, 2026Junjie Peng, You Wu, Haoyi Wu +4KV Caching Long-Context Language Modeling
1Sun Yat-sen University · 2ShanghaiTech University · 3Guangdong Province Key Laboratory of Information Security Technology
BudgetDraft: Acceptance-Aware Multi-View Training for Sparse-KV Speculative Decoding
May 29, 2026Liang He, Jingbo Wen, Qishi Zhan +4KV Caching Long-Context Language Model Inference
Shanghai Institute of Optics and Fine Mechanics · The University of Sydney · Marquette University +4
CacheProbe: Auditing Prompt Cache Isolation in Gateway APIs
May 28, 2026Ryan FaheyKV Caching LLM Auditing
Khoury College of Computer Sciences Northeastern University Boston, MA, USA
Probing the Prompt KV Cache: Where It Becomes Dispensable
May 28, 2026Vinayshekhar Bannihatti Kumar, Manoj Ghuhan Arivazhagan, Disha Makhija +1KV Caching KV-Cache Management
AWS AI Labs
OmniMem: Scalable and Adaptive Memory Retrieval for Long Video Generation
May 28, 2026Lin Zhao, Yushu Wu, Yifan Gong +2KV Caching Long Video Generation
Northeastern University · Adobe Research
Future Forcing: Future-aware Training-free KV Cache Policy for Autoregressive Video Generation
May 28, 2026Jiayi Luo, Qiyan Liu, Tengyang Wang +8KV Caching Long-Horizon Video Generation
BUAA · ZGCA · PKU +3

Jun 8, 2026, cs.LGJ/K move · Enter open · S save

FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention

Yan Wang, Qifan Zhang, Jiachen Yu, Tian Liang+11

Independent Researchers · Tencent · The Hong Kong University of Science and Technology (Guangzhou)+1

KV Caching Long-Context Language Model Inference Memory-Efficient Inference Sparse Attention KV-Cache Compression

Open paper AskPDF arXiv

Conventional LLMs keep the full KV cache loaded during decoding, causing a severe GPU memory bottleneck for ultra-long context serving. In this report, we propose Lookahead Sparse Attention (LSA), a novel inference paradigm powered by a Neural Memory Indexer built upon the DeepSeek-V4 architecture. Rather than passively attending to all historical tokens, LSA proactively predicts future context demands and preserves only the query-critical KV chunks in the GPU memory. Crucially, we instantiate this architecture via a backbone-free decoupled training strategy. By formulating the indexer as a standard dual-encoder architecture, we train it independently using standard retrieval training frameworks without ever loading the massive backbone model into GPU memory. We demonstrate that this "less is more" paradigm significantly maximizes serving efficiency while acting as an effective attention denoiser in tasks that rely on long-term global memory. Across primary long-context evaluation suites (e.g., LongBench-v2, LongMemEval, and RULER), FM-DS-V4 compresses the average physical KV cache footprint down to merely 13.5% of the full-context baseline, while consistently preserving or slightly elevating downstream accuracy (+0.6% absolute margin on average). Crucially, at extreme 500K scales, FlashMemory suppresses the physical KV cache overhead by over 90% without destabilizing the backbone's core reasoning capacities.

Momentum

Related topics

Latest papers 276