Page-EntroKV: Hardware-Aligned, Entropy-Weighted KV-Cache Eviction under Grouped-Query Attention

Inbasekaran S

cs.LGOct 2, 2026

Page-EntroKV: Hardware-Aligned, Entropy-Weighted KV-Cache Eviction under Grouped-Query Attention

Authors: Inbasekaran S

Organizations: Department of Computer Science and Engineering SRM Institute of Science and Technology, India

Abstract

Serving long-context autoregressive language models is constrained by the key-value (KV) cache. Most dynamic eviction methods score token importance per query head and choose tokens independently. This fits poorly with grouped-query attention (GQA), where several query heads share one physical KV buffer: divergent per-head selections force the serving engine to retain the union of their choices - inflating the cache by up to the group ratio r - while arithmetic mean pooling dilutes the specialized retrieval heads that carry factual recall. We introduce Page-EntroKV, a formal framework for KV-cache eviction operating at the granularity GQA serving actually allocates. Heads within each physical group are pooled by weights derived from sink-isolated collision (Renyi-2) entropy - one inner product per head, computed once at prefill with no calibration - so sink heads cannot masquerade as retrieval heads. Pooled scores are projected onto PagedAttention page frames, and eviction executes at the hardware tuple (layer, group, page). We formalize the union overhead ratio (UOR) and intra-group disagreement, prove an exact identity linking them for two-head groups alongside two-sided bounds at every group ratio, prove strict budget preservation and a finite-context needle-retention bound that arithmetic mean pooling provably violates, and give exact per-layer page accounting. On a pilot architecture (Qwen2.5-1.5B-Instruct, r=6), head-independent replay over 2,240 group measurements yields union overhead up to 4.75x at a 2% budget, while Page-EntroKV holds UOR exactly 1.000; sink isolation removes a 13x sink masquerade; needle recall is 100% versus 0% for mean pooling at a 20% budget; retained cardinality is exact for every page size; and QA and code tasks remain solvable at 20% retention.

Figures & tables

Panel of Figure 1 — Figure 1: Page-EntroKV pipeline overview. Sink-isolated entropy-weighted pooling at the physical GQA group level, followed by block-max mapping onto PagedAttention page frames, aligns the eviction decision with the engine’s physical memory layout.

Method	Importance	GQA-union	Eviction	Strict budget	Calibration
	granularity	aware?	granularity	guarantee?	-free?
H2O [ 10 ]	$(l,h,t)$	$\times$	token	$\times$ (union $\leq r\times$ )	✓
StreamingLLM [ 13 ]	position	✓ (uniform)	token	✓ (static)	✓
Scissorhands [ 12 ]	$(l,h,t)$	$\times$	token	$\times$	✓
SnapKV [ 11 ]	$(l,h)$	$\times$	token	$\times$	✓
FastGen [ 14 ]	per-head policy	$\times$	token	$\times$	$\times$ (profiling)

Table 1: Qualitative structural comparison of KV-cache eviction approaches under physically shared GQA buffers. ✓ = supported;

\sim

= partially;

\times

= not supported; n/a = not applicable: the method is not a token-eviction policy (channel-dimension pruning or quantization) or is a context-routing architecture.

	Bonferroni	Sandwich
	( Corollary 1 )	( Theorem 1 )
$2\times$ threshold ( $r{=}4$ )	$\bar{J}\leq\frac{1}{6}\approx 0.167$	$\bar{J}\leq\frac{1}{5}=0.200$
$2\times$ threshold ( $r{=}8$ )	$\bar{J}\leq\frac{3}{28}\approx 0.107$	$\bar{J}\leq\frac{3}{11}\approx 0.273$
At $\bar{J}\to 0$ (disjoint)	$r$ (exact, tight)	$r$ (exact, tight)
At $\bar{J}\to 1$ (consensus)	clamp to $1$	$1$ (naturally tight)

Table 2: The two disagreement certificates compared. The sandwich certificate of Theorem 1 strictly widens the certified-inflation regime; it is also naturally tight at consensus, where the Bonferroni form needs its clamp.

	Shannon ( $\alpha{\to}1$ )	Collision ( $\alpha{=}2$ )	Min-entropy ( $\alpha{\to}\infty$ )
Definition	$-\sum_{t}A\ln A$	$-\ln\\|A\\|_{2}^{2}$	$-\ln\max_{t}A$
On-chip primitive	$T$ ln-calls	1 dot product $+\,1$ ln	$T$ -way max $+\,1$ ln
FP16 stability	cancellation as $A(t)\to 0$	stable ; argument in $(0,1]$	plateau ties
Spike sensitivity	logarithmic scaling	quadratic amplification of spikes	ignores shape below peak

Table 3: Why

\alpha=2

: on-chip cost and numerical behavior of the three entropy orders for one head with

T

positions. Collision entropy is the only order that reduces to a single dot product with a bounded logarithm argument.

Panel of Figure 2 — Figure 2: Page-EntroKV mechanism for an $r=4$ architecture such as Llama-3.1-8B: entropy-weighted pooling with sink-isolated entropy amplifies the retrieval head $Q_{1}$ , suppresses diffuse heads $Q_{0}$ , $Q_{2}$ , and the attention-sink head $Q_{3}$ , and maps the result onto physical PagedAttention blocks.

Panel of Figure 3 — Figure 3: Hardware-level execution flow of the fused Page-EntroKV kernels within the vLLM decoding loop.

Figure 4: Empirical union overhead ratio versus intra-group Jaccard disagreement ( $r{=}6$ , Qwen2.5-1.5B pilot, Section 6.2 ). Blue: observed head-independent selections (SnapKV-style replay). Red dashed: the Cauchy–Schwarz lower envelope of Theorem 1 . Green: Page-EntroKV’s pooled selection, $\mathrm{UOR}\equiv 1$ by construction ( Theorem 3 ). Dotted: the worst-case orthogonal ceiling $r$ . The observed tail approaches the envelope at high disagreement.

(a) By scoring method
Scorer	$k/T$	$\overline{\mathrm{UOR}}$	$\mathrm{UOR}_{\max}$	$\mathrm{frac}(\mathrm{UOR}{>}1)$	$\mathrm{cert}_{1}$	$\mathrm{cert}_{T}$	$\geq 2$ -frac
H2O (accum.)	0.02	2.849	5.100	1.000	62	290	0.879
H2O (accum.)	0.05	2.442	4.471	1.000	22	112	0.820
H2O (accum.)	0.10	2.292	3.971	1.000	6	65	0.769
Last-row	0.02	2.982	4.600	1.000	21	329	0.960
Last-row	0.05	2.703	3.961	1.000	6	196	0.933

Table 4: Head-independent replay vs. pooled selection (Qwen2.5-1.5B-Instruct,

r{=}6

L{=}28

B{=}16

). (a) Per-scorer replay:

\mathrm{cert}_{1}

counts cells where the Bonferroni certificate ( Corollary 1 ) certifies

\mathrm{UOR}\geq 2

;

\mathrm{cert}_{T}

where the sandwich certificate ( Theorem 1 ) does;

\geq 2

-frac is the fraction of cells with measured

\mathrm{UOR}\geq 2

. (b) Coarse cell-level sweep (2,240 logged group measurements; NaN rows zeroed and counted per Section 6.1 ). Pooled Page-EntroKV row: 1,008 measurements, zero layers skipped.

(a) $\tau_{g}$ sweep			(b) $\alpha$ sweep ( $\tau_{g}{=}0.5$ )
$\tau_{g}$	$\max_{h}w_{h}$	$w$ std	$\alpha$ : $\max_{h}w_{h}$ ; $[\mathcal{H}_{\min},\mathcal{H}_{\max}]$
0.1	0.9632	0.3922	0.5	0.620; $[3.530,\,7.112]$
0.2	0.9105	0.3700	1.0	0.762; $[1.820,\,6.638]$
0.5	0.7502	0.2994	2.0	0.750; $[1.091,\,5.910]$
1.0	0.5501	0.2076	999	0.641; $[0.624,\,4.260]$
2.0	0.3656	0.1179	violations of $[0,\ln T]$ : 0

Table 5: Certified mechanism ablations (Qwen2.5-1.5B pilot; coverage certification: 56/56 layer–group pairs ,

100\%

; zero NaN-skipped layers). (a) Temperature sweep:

\max_{h}w_{h}

is monotone in

\tau_{g}

toward

1/r=0.167

( Proposition 2 ). (b) Rényi order sweep at

\tau_{g}{=}0.5

: every entropy lies in

[0,\ln T]=[0,7.625]

, zero violations. (c) Cardinality exactness ( Theorem 3 ). (d) Page-union ratio ( Proposition 4 ).

Figure 5: Ablation panels ( Section 6.3 ; certified run: 56/56 layer–group pairs, zero layers skipped). (a) Temperature limits: maximum within-group weight $\max_{h}w_{h}$ decays monotonically toward the mean-pooling limit $1/r=0.167$ as $\tau_{g}$ grows, the empirical form of Proposition 2 . (b) Rényi order sensitivity at $\tau_{g}{=}0.5$ : weight sharpness peaks near $\alpha\in\{1,2\}$ ; all entropies lie within $[0,\ln T]$ with zero violations.

Figure 6: Pooled attention mass across token offsets within retained page frames ( Section 6.3 ; $B{=}16$ , $r{=}6$ pilot). Slot 0 concentrates $\approx 14.4\%$ of mass against the $6.25\%$ uniform-slot baseline, with secondary concentrations at slots 8 and 10: in-page skew that block-max aggregation is designed to capture, and a descriptive motivation for page-granular eviction.

Figure 7: Sink isolation separates head populations ( Section 6.3 ; Qwen2.5-1.5B pilot; three probes, 1,008 layer–head observations: $n{=}152$ sink, $n{=}47$ retrieval, $n{=}809$ contextual). (a) Raw collision entropy: the sink population sits at low entropy and overlaps the left flank of the retrieval population—the masquerade that Theorem 2 predicts raw entropy cannot break. (b) Sink-isolated entropy: the sink mass shifts right while retrieval heads stay low. The dashed line marks the uniform-residual ceiling $\ln(T-|S_{\mathrm{sink}}|)=6.73$ of Theorem 2 ; measured populations sit below it because real sink residuals are not uniform. Isolation inverts the sink-to-context weight ordering rather than driving any single weight to zero ( Fig. 8 ).

Figure 8: Operator-level pooling weights under raw versus sink-isolated entropy, from the same 1,008-observation census as Fig. 7 ( Section 6.3 ). Sink isolation lowers the sink head’s weight from $0.303$ to $0.116$ and raises the contextual head’s weight from $0.144$ to $0.176$ , inverting the sink-to-context weight ratio from $2.10\times$ (raw: sink heads outweigh contextual heads without earning it) to $0.66\times$ (isolated: contextual heads dominate)—the empirical form of Theorem 2 .

(a) Pooled-selection criterion
Context $T$	959	1469	1989	2499
Arithmetic mean	0.0%	0.0%	0.0%	0.0%
Page-EntroKV	100.0%	100.0%	100.0%	100.0%
(b) Strict all-layer criterion (recall by needle depth)
Method	0%	25%	50%	75% / 100%
Uncompressed	1.00	1.00	1.00	1.00 / 1.00

Table 6: Needle retrieval under two success criteria (Qwen2.5-1.5B pilot, matched

20\%

physical budget). Pooled-selection criterion: retention requires the needle’s page to survive the pooled eviction decision (no cross-layer requirement). Strict all-layer criterion: the needle’s page must survive in every layer. Asterisk: weighted recall at depth 0 spans

0

–

33\%

across strict-variant runs. All zero cells verified as genuine evictions (30/30).

LongBench subset	Page-EntroKV score (%)
qasper	38.60
multifieldqa_en	44.20
lcc	56.10
repobench-p	44.81

Table 7: Downstream feasibility (LongBench subsets that completed,

20\%

retention, 20 samples each; Qwen2.5-1.5B pilot). Scores are not comparable to published LongBench numbers (sample size, retention, and the absence of an uncompressed same-harness baseline). The remaining LongBench subsets are pending re-run after an evaluator fault; unscored subsets are excluded, not reported as zero.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Definition
$L$	Number of transformer layers, $l\in\{1,\dots,L\}$ .
$H_{Q},\ H_{\mathrm{KV}}$	Logical query heads; physical KV heads.
$r=H_{Q}/H_{\mathrm{KV}}$	GQA group ratio (query heads per physical KV buffer).
$D_{\mathrm{model}},\ D_{\mathrm{head}}$	Hidden dimension; head dimension ( $D_{\mathrm{model}}=H_{Q}\cdot D_{\mathrm{head}}$ ).
$T$	Current sequence length ( $T\leq T_{\mathrm{ctx}}$ ).
$B$	PagedAttention block size (tokens per page, e.g. $B=16$ ).

Appendix

Table 8: Symbols and notation used throughout the paper.

Guarantee	Mathematical form	Value / bound	Systems impact	Validated by
Arithmetic intensity	$W_{\mathrm{FLOPs}}/D_{\mathrm{bytes}}$ ( Eq. 1 )	$2r/b_{\mathrm{el}}=4.0$ FLOP/Byte	Decode phase memory-bound	Architectural constant
H100 ridge point	Eq. 2	$\approx 295.4$ FLOP/Byte	Decode at $<1.5\%$ of peak	Architectural constant
Cache footprint	Eq. 3	$4B_{\mathrm{batch}}LH_{\mathrm{KV}}D_{\mathrm{head}}T$ Bytes	Linear HBM capacity	Architectural constant
$r=2$ identity	$\mathrm{UOR}=2/(2-D)$ , monotone and convex in $D$ ( Remark 1 )	Exact	Disagreement implies inflation	Table 4 certificates
Bonferroni certificate	$\mathrm{UOR}\geq\max\{1,\,r-r(r-1)\bar{J}\}$	$2\times$ at $\bar{J}\leq\frac{r-2}{r(r-1)}$	Elementary certificate	Table 4 ( $\mathrm{cert}_{1}$ )
Sandwich bound	$\frac{r}{1+2(r-1)\bar{J}/(1+\bar{J})}\leq\mathrm{UOR}\leq\min(r,\,r-\frac{2J_{\max}}{1+J_{\max}})$	Tight at extremes	Two-sided inflation bound	Table 4 ( $\mathrm{cert}_{T}$ ); Fig. 4

Appendix

Table 9: Master verification summary of Page-EntroKV theoretical guarantees and hardware geometry (Tables 4 – 7 ; Figs. 4 – 8 ).

Figure 9: Corollary 2 two-regime mechanism check ( $r{=}6$ , single $\varepsilon=0.05$ used throughout, $\tau_{g}=0.5$ ). (a) Synthetic premise-satisfied regime : an engine-path construction with a needle head at mass $1-\varepsilon$ and near-maximal-entropy siblings; the measured pooled needle mass coincides with the bound $(1-\varepsilon)-\Delta(T)$ at every context length (raw gap $\leq 3\times 10^{-7}$ ; markers offset by $+0.01$ vertically for visibility), verifying the mechanism end-to-end up to 128k. (b) Real-text pilot regime (premise unsatisfied) : Qwen2.5-1.5B-Instruct eager-attention captures at $T\leq 2048$ ; the measured best-head needle mass is $\approx 0.16$ rather than $0.95$ , so the premise of Theorem 4 is not reached and entropy-weighted pooling degenerates toward arithmetic-mean pooling ( Proposition 2 ii), the pooled and mean masses agreeing to within a factor $1.06$ . The two panels together delimit where the guarantee binds; neither panel alone is interpretable.

$T$	Non-sink $N$	$D(T)$	$\Delta(T)$	$\mathbf{A}^{\mathrm{pooled}}_{l,g}(t^{*})$	$\mathbf{A}^{\mathrm{mean}}_{l,g}(t^{*})$	Ratio
4k	4,092	$4.18\times 10^{-7}$	$1.59\times 10^{-6}$	0.949998	0.118750	$8.00\times$
8k	8,188	$1.04\times 10^{-7}$	$3.97\times 10^{-7}$	0.949999	0.118750	$8.00\times$
16k	16,380	$2.61\times 10^{-8}$	$9.92\times 10^{-8}$	0.949999	0.118750	$8.00\times$
32k	32,764	$6.52\times 10^{-9}$	$2.48\times 10^{-8}$	0.950000	0.118750	$8.00\times$
64k	65,532	$1.63\times 10^{-9}$	$6.20\times 10^{-9}$	0.950000	0.118750	$8.00\times$
128k	131,068	$4.08\times 10^{-10}$	$1.63\times 10^{-9}$	0.950000	0.118750	$8.00\times$

Appendix

Table 10: Non-asymptotic scaling of the finite-context bound ( Corollary 2 ) across deployment contexts (

r{=}8

\tau_{g}{=}0.5

c_{j}{=}0

\eta_{\mathrm{ret}}=\ln 2

1-\epsilon=0.95

). All entries are arithmetic instantiations of the closed-form bound—not measurements.

KV-Cache Eviction

Key-value (KV) cache eviction reduces language-model memory use by removing or compressing cached tokens while preserving useful context.

See all 99 papers

Aug 26, 2026

PAGE: Partition-Aware Gated KV-Cache Eviction

KV-cache eviction can do more than compress. In long-context LLMs, keeping only some cached tokens sometimes matches or exceeds full-cache accuracy, because many redundant prefill tokens otherwise dilute attention away from the tokens that carry the answer. This benefit is not uniform, and evicting the wrong tokens can drop accuracy to zero on tasks that require precise retrieval, so the useful question is not only which tokens to keep but also whether to evict this input at all. We show that one label-free number computed from the prefill attention, the drop between early and late layers in how much attention heads agree on which tokens to read, predicts per input, before any decoding, which of the two cases an input falls under. We build this into PAGE (Partition-Aware Gated Eviction), a wrapper that runs any SnapKV-style evictor when the drop is large and keeps the full cache when it is small, with no training, labels, or fine-tuning. PAGE is a safety mechanism rather than a compressor, so we measure it by the failures it prevents. It cuts the harm rate on capacity-bound inputs from 0.75 to 0.026, and on multi-key retrieval with Mistral-7B plain SnapKV falls from 99% to 0% as the budget shrinks, while PAGE holds it at 89%. Elsewhere, it passes the base evictor through unchanged, which is the intended behaviour and is what we observe in 8 of 16 cells. Code is available at https://anonymous.4open.science/r/PAGE-018239.

Pankaj Kumar, Subhankar Mishra

School of Computer Sciences National Institute of Science Education and Research An OCC of Homi Bhabha National Institute, India

Long-Context Retrieval KV Caching KV-Cache Eviction Cache Replacement

Ask PDF

Aug 5, 2026

QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding

Autoregressive large language model inference is increasingly constrained by the memory footprint of the Key-Value (KV) cache. A dominant line of work reduces this footprint by evicting tokens that appear unimportant under attention-derived scores. However, such policies make an implicit irreversible decision: once a token is evicted, it cannot become useful again. We show that this assumption is brittle during decoding. Token and window importance drift as generated queries evolve, causing standard eviction policies to permanently discard states that later receive substantial attention under the full-cache model. To characterize this behaviour, we introduce Future Missed Mass and Global LIR, two diagnostics that measure future attention assigned to discarded states and the reactivation of historically inactive regions. We propose QEvict, a three-tier KV-cache management scheme that replaces binary retain-or-delete eviction with recoverable eviction. QEvict maintains high-confidence windows in full precision, stores intermediate windows in a quantized recoverable tier, and deletes only the lowest-confidence windows. During decoding, cumulative attention scores update window importance and when a quantized window becomes important again, it is dequantized and promoted to the full-precision. Under a fixed memory budget, this design preserves broader historical context while retaining exact full precision for the most important regions. Across long-context understanding, retrieval, and reasoning benchmarks, QEvict consistently improves over representative eviction and quantization baselines, reducing missed attention and improving information retention

Ayushman Garg, Akshita Gupta, Shaswata Bhattacharya +3

Mehta Family School of Data Science and Artificial Intelligence, Indian Institute of Technology Roorkee · Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi · Department of Electrical Engineering, Indian Institute of Technology Delhi

KV Caching Long-Context Language Model Inference KV-Cache Eviction KV-Cache Management KV-Cache Quantization KV-Cache Pruning

Ask PDF

May 10, 2026

Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction

The key-value (KV) cache is a major bottleneck in long-context inference, where memory and computation grow with sequence length. Existing KV eviction methods reduce this cost but typically degrade performance relative to full-cache inference. Our key insight is that full-cache attention is not always optimal: in long contexts, irrelevant tokens can dilute attention away from useful evidence, so selective, learnable eviction can improve generation rather than merely approximate the full cache. We introduce a global retention-based KV eviction method that learns each token's future utility under a unified memory budget. Lightweight retention gates assign utility scores to cached KV entries, and a shared final scoring projection calibrates these scores across all layers and heads. This enables a single global eviction policy in which tokens from different layers, heads, and modalities compete directly for cache capacity. We further provide theoretical analysis showing that preferentially retaining useful tokens reduces attention dilution, and we justify geometric retention as a query-agnostic proxy for future utility. Across diverse long-context language and vision-language reasoning, and multi-turn dialogue benchmarks, our method substantially reduces KV memory while matching or surpassing full-cache inference. These results suggest that learned, globally calibrated KV eviction is not only a compression technique, but also a mechanism for improving long-context reasoning.

Ngoc Bui, Hieu Trung Nguyen, Arman Cohan +1

Department of Computer Science, Yale University · The Chinese University of Hong Kong

Self-Attention KV Caching KV-Cache Eviction Long-Context Inference KV-Cache Pruning

Ask PDF

Abstract

KV-Cache Eviction

Explore similar work

PAGE: Partition-Aware Gated KV-Cache Eviction

QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding

Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction