Eviction

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 20

All topics

When to Evict, Not What to Keep: Draft-Guided Eviction for Training-Free KV-Cache Compression
Sep 27, 2026Haeyong Kang, Chang D. YooEviction Key-Value Cache Compression
Duksung Women’s University · KAIST
When Fancy Eviction Fails: Rethinking Cache Replacement For LLM Prefix Reuse
Sep 24, 2026Yiyu Liu, Minlan Yu, Juncheng YangEviction Cache
Harvard University
Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targets
Sep 23, 2026Beomgu Kang, SoJin Yun, Hojoon Kim +1Eviction Retention
Department of Artificial Intelligence, Korea University, Republic of Korea
What Eviction Destroys: A Restore-Counterfactual Audit of Forgetting in Agent Memory
Sep 8, 2026Chen ShenEviction Forgetting
Megagon Labs
What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservation
Sep 3, 2026Bo Zeng, Yu Zhao, Yefeng Liu +3Key-Value Cache Compression Key-Value Cache Eviction
Ant Group · Alibaba Group · Universität Hamburg
Invalidation Contracts for Cross-Episode Agent Memory
Aug 31, 2026Michael Wu, Arquimedes CanedoGradient Staleness Contracts
Siemens Digital Industries Software Princeton, NJ, USA
PAGE: Partition-Aware Gated KV-Cache Eviction
Aug 26, 2026Pankaj Kumar, Subhankar MishraEviction Retrievers
School of Computer Sciences National Institute of Science Education and Research An OCC of Homi Bhabha National Institute, India
Back from the Future: Key-Value Cache Management by Counter-Causal Surprise
Jul 30, 2026Stephen Gould, Anton van den HengelKey-Value Cache Cache
Metacognition AI · Australian National University · Adelaide University
Eviction as Estimation: A Fixed-Lag Smoothing View of Test-Time Memory, and When Measuring Beats Accumulating
Jul 27, 2026Maruthi Vemula, Neeraj Praneeth GajulaWorking Memory Eviction
University of North Carolina at Chapel Hill · New York University
Error Certificates for KV-Cache Eviction via Randomized Design
Jul 23, 2026Peng XieKey-Value Cache Eviction Eviction
Technical University of Munich
CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference
Jun 23, 2026Xiaolin Lin, Jingcun Wang, Olga Kondrateva +3Key-Value Cache Compression LLM Inference Optimization
Technical University of Darmstadt, Darmstadt, Germany · University of Notre Dame, Notre Dame, IN, USA · Technical University of Ilmenau, Ilmenau, Germany
Value-Aware Stochastic KV Cache Eviction for Reasoning Models
Jun 2, 2026Ting-Yun Chang, Harvey Yiyun Fu, Deqing Fu +3Key-Value Cache Eviction Eviction
University of Southern California · University of Chicago
Leyline: KV Cache Directives for Agentic Inference
May 31, 2026Bole Ma, Jan Eitzinger, Harald KoestlerCache Key-Value Cache
Erlangen National High Performance Computing Center, Erlangen, Germany
LearnedCache: An eBPF-Integrated Perceptron-Based Eviction Policy for the Linux Page Cache
May 25, 2026Zejia QiEviction Cache
A Simple Plug-in for Improving Eviction-Based KV Cache Compression
May 22, 2026Yuping Lin, Jiayuan Ding, Yue Xing +3Key-Value Cache Compression Eviction
1Michigan State University · 2Hippocratic AI
Not All Tokens Are Worth Caching: Learning Semantic-Aware Eviction for LLM Prefix Caches
May 12, 2026Shaoke Fang, Ziang Li, Wenfei Wu +3Eviction Cache
Peking University · FirestAI · Tsinghua University +2
PEEK: Predictive Queue-Informed KV Cache Management for LLM Serving
May 10, 2026Bing Xie, Zhipeng Wang, Masahiro Tanaka +1Kv-Cache Management Eviction
†Corresponding author.
Beyond Compaction: Structured Context Eviction for Long-Horizon Agents
May 1, 2026Andrew Semenov, Svyatoslav DorofeevLong-Horizon Agents Compaction
Kiz8
Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective
Apr 28, 2026Jiaming Yang, Chenwei Tang, Liangli Zhen +1Key-Value Cache Eviction Eviction
College of Computer Science, Sichuan University, Chengdu, China · Institute of High Performance Computing, A*STAR, Singapore
MoE-nD: Per-Layer Mixture-of-Experts Routing for Multi-Axis KV Cache Compression
Apr 20, 2026Libo Sun, Peixiong He, Po-Wei Harn +1Key-Value Cache Compression Eviction
Department of Computer Science and Software Engineering, Auburn University, Auburn, AL, USA · Department of Information Management, National Central University, Taoyuan, Taiwan

Sep 27, 2026, cs.LGJ/K move · Enter open · S save

When to Evict, Not What to Keep: Draft-Guided Eviction for Training-Free KV-Cache Compression

Haeyong Kang, Chang D. Yoo

Duksung Women’s University · KAIST

Eviction Key-Value Cache Compression Bounded-Memory Data Compression Methods

Open paper AskPDF arXiv

Training-free KV-cache compression methods such as SnapKV, H2O, and PyramidKV evict tokens at the end of prefill, aiming to preserve the attention mass that future queries are expected to use -optimizing what to keep. We show that this objective fails in two distinct ways. (1) Compensation: restoring the evicted attention mass can recover the attention-level target without recovering task quality. (2) Selection: covering more of the true decode-query mass can hurt quality when the recovered mass is fragmented rather than concentrated in coherent spans. These failures share a common cause: eviction occurs before the queries that determine the answer trajectory exist. We propose Draft-Guided Eviction (DGE), which defers eviction until after drafting the first k=2 answer tokens using the full cache - just one decode step beyond prefill. Because the draft is generated from the answer's own prefix, no cache entries are discarded before this trajectory signal becomes available. The per-head cache budget remains unchanged, and DGE can be applied directly to SnapKV, PyramidKV, H2O, and StreamingLLM without modifying their eviction scores. Unlike extra-pass methods, DGE changes when eviction occurs rather than what cache entries are selected. Extensive experiments demonstrate that DGE outperforms prior methods at every evaluated budget on five of six instruct-tuned backbones, achieving 44.2 on LongBench, nearly matching FullKV at 44.3. The timing-only control DGE-W achieves the same score, demonstrating that the gain comes from when eviction occurs rather than what is selected - an effect we term trajectory anchoring.

Momentum

Related topics

Latest papers 20