WakeKV: Reactive, Reversible KV Residency for Heads That Change Their Minds

Utkarsh Ranjan

cs.CLOct 2, 2026

WakeKV: Reactive, Reversible KV Residency for Heads That Change Their Minds

Authors: Utkarsh Ranjan

Organizations: UC San Diego

Abstract

Most KV-cache compression methods classify attention heads once, either offline or during prefill, and keep this classification fixed throughout generation. Across three models (1.5B-8B) and three regimes (needle retrieval, long chain-of-thought, and multi-turn recall), we measure head behavior on four model-regime combinations and find that most heads change their reading behavior at least once during generation. We introduce WakeKV, a reactive residency policy that moves cooling heads to a recoverable CPU reservoir rather than freezing or permanently evicting their state. At matched memory or budget, WakeKV consistently improves miss rate over frozen classification and destructive eviction, evaluated across five model-regime combinations and over three cited baselines (SnapKV, uniform R-KV, and ReasonAlloc) across four eligible combinations. A FlexiCache/vLLM implementation on Mistral-7B confirms the benefit on real hardware, improving throughput while retaining LongBench quality.

Figures & tables

Panel of Figure 1 — Figure 1: (a) Prior methods split along two axes—when role is decided, and whether demotion is reversible—leaving the dynamic-and-reversible quadrant empty until WakeKV. (b) A demoted head’s state moves to a recoverable CPU reservoir instead of being destroyed; the evict foil (dashed) shares WakeKV’s exact LRU schedule but destroys state instead of reserving it.

Comparison (matched memory or budget)	Coverage	WakeKV wins
vs. Frozen, equal memory	5 model/regime	17/17
vs. Evict, equal budget	5 model/regime	19/20
vs. SnapKV/R-KV/ReasonAlloc, equal memory	4 model/regime †	32/32

Table 1: Headline results: how often reactive’s miss rate was

\leq

the comparator’s, at every point where memory or budget was matched exactly rather than compared at a shared nominal label (full breakdown in Table 3 ; real-system result in Table 2 ). † Mistral-7B/NIAH excluded: its R-KV/ReasonAlloc needed a non-default pruning interval.

Figure 2: Memory-vs-miss-rate Pareto fronts across NIAH, CoT, and multi-turn. Reactive sits at or below frozen’s at all 17 matched-memory points and below evict’s at 19 of 20 matched budgets (Table 1 ).

Config	Rerank int.	Gen tok/s	vs stock	LongBench %
Stock FlexiCache	16 (native)	114.4	1.00 $\times$	100
Reactive (WakeKV)	1	153.6	1.34 $\times$	97.4
Reactive	2	193.3	1.69 $\times$	97.1
Reactive	4	226.3	1.98 $\times$	99.1
Reactive	8	254.2	2.22 $\times$	97.9
Reactive	$16^{\star}$	266.6	2.33 $\times$	95.0

Table 2: Real-system throughput

\times

quality sweep (A30, Mistral-7B, LEval 8k

\to

1000; LongBench, 4

\times

30 samples). Bold marks speedup over stock;

R{=}1

is WakeKV’s design point, ⋆

R{=}16

its Pareto knee (quality stays above

95\%

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Model/regime	$B$	Frozen	Reactive	Evict	SnapKV	R-KV	ReasonAlloc	Reactive mem
Qwen-3B/multiturn	8	0.491	0.631	0.651	0.428	0.163	0.163	4,608
	16	0.294	0.300	0.369	0.247	0.093	0.090	9,187
	32	0.116	0.010	0.025	0.201	0.010	0.009	17,373
	64	0.110	0.000	0.000	0.201	0.000	0.000	18,621
Qwen-3B/NIAH	8	0.403	0.655	0.760	0.499	0.250	0.250	4,608
	16	0.276	0.409	0.617	0.429	0.179	0.177	9,209

Appendix

Table 3: Raw simulator sweep: miss rate per policy at each nominal budget

B

(page units), plus reactive’s realized memory (“Reactive mem”). Reactive vs. Evict, the one same-row-comparable pair, favors reactive in 19 of 20 rows (the exception is 8B/CoT,

B{=}8

); see § 5 for why the other four columns require Table 1 ’s matched-memory correction instead. † Non-default pruning interval (see text above); excluded from Table 1 ’s win-counts.

Regime	Model	Adj. Jaccard	Persistently unstable	Ever drifted ( $\geq$ 1 event)
NIAH	Qwen2.5-3B	0.53 (cont.)	0.064	0.63
CoT	R1-Distill-1.5B	0.82 (bin.)	0.023	0.85
Multi-turn	Qwen2.5-3B	0.79 (bin.)	0.085	0.76
CoT	R1-Distill-8B	0.93 (bin.)	0.042	0.79

Appendix

Table 4: Head-shift metrics by regime. Adjacent Jaccard: continuous score (NIAH), binary score (CoT/multi-turn). At 8B (bottom row), binary Jaccard tightens as copy behavior stabilizes on math CoT, but the page-level drift metric (what a residency controller reacts to) stays high at 0.79, so the case is undiminished at scale.

KV-Cache Management

Key-value (KV) cache management reduces and coordinates Transformer attention state to make long-context inference faster and more memory-efficient.

See all 201 papers

ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression
Jul 31, 2026Yuhang Zhan, Lisi Chen, Shuo ShangSoftmax Attention KV Caching
University of Electronic Science and Technology of China
DeferKV: Rethinking Eviction Timing for One-Shot KV Cache Compression
Oct 5, 2026Zhe Wang, Jiakai Li, Yujia Sun +2Long-Context Language Model Inference KV-Cache Eviction
University of Electronic Science and Technology of China, Chengdu, China · Ubiquitous Intelligence and Trusted Services Key Laboratory of Sichuan Province, Chengdu, China
ChampionKV: Head-Wise Adaptive Budget Allocation for KV Cache Compression
Oct 13, 2026Yuechuan Chen, Yanda Guo, Xiaoxi Liu +4KV-Cache Compression KV-Cache Management
Sun Yat-sen University · The Chinese University of Hong Kong · University of Electronic Science and Technology of China +2

Jul 31, 2026J/K move · Enter open · S save

ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression

Yuhang Zhan, Lisi Chen, Shuo Shang

University of Electronic Science and Technology of China

Softmax Attention KV Caching KV-Cache Eviction KV-Cache Compression Attention Mechanisms Long-Context Inference

Open paper AskPDF arXiv

KV cache compression is essential for efficient long-context inference. Existing eviction methods permanently discard unselected tokens and consequently remove their aggregate contribution to attention. Merging-based alternatives preserve more information but can perturb retained keys and values that should remain exact. We observe that the information omitted by cache eviction can be formulated as residual statistics in both the numerator and denominator of softmax attention. Based on this observation, we propose ResKV, which divides a fixed KV budget into an exact main cache and a compact residual cache that reconstructs the contribution of omitted tokens. ResKV lets main-cache tokens and residual entries participate in the same softmax normalization, so residual entries restore both attention numerator and denominator mass rather than acting as a post-hoc correction. A construction-time validation proxy determines residual allocation for each layer and KV head, while a decode-time dynamic gate adjusts residual contributions for individual queries. Comprehensive evaluations on LongBench and RULER, covering query-aware and query-agnostic settings, multiple backbones, cache budgets, and representative compression baselines, demonstrate broad improvements under the same retained KV budget while preserving the practical efficiency of compressed decoding, including peak memory usage and long-context decode throughput.

Abstract

KV-Cache Management

Explore similar work

ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression

DeferKV: Rethinking Eviction Timing for One-Shot KV Cache Compression

ChampionKV: Head-Wise Adaptive Budget Allocation for KV Cache Compression