ChampionKV: Head-Wise Adaptive Budget Allocation for KV Cache Compression

Yuechuan Chen; Yanda Guo; Xiaoxi Liu; Weilin Ruan; Huanbo Xu; Ruyi Wang; Wei Zhu

cs.LGOct 13, 2026

ChampionKV: Head-Wise Adaptive Budget Allocation for KV Cache Compression

Authors: Yuechuan Chen, Yanda Guo, Xiaoxi Liu, Weilin Ruan, Huanbo Xu, Ruyi Wang, Wei Zhu

Organizations: Sun Yat-sen University · The Chinese University of Hong Kong · University of Electronic Science and Technology of China · Tongji University ·

Abstract

Some observation-window-based KV cache compressors use prompt attention to identify important tokens but allocate uniform or predefined budgets across heads, overlooking input-dependent differences in budget demand. We propose ChampionKV, an input-adaptive head-wise budget allocator for long-context KV cache compression. It uses the maximum aggregated observation-window attention score as a lightweight empirical proxy for attention concentration, assigning larger budgets to heads with dispersed attention and smaller budgets to concentrated heads. An abnormal-head mask and a floor-budget mechanism improve robustness under weak attention responses. ChampionKV plugs into compressors such as SnapKV and PyramidKV without changing their token-selection procedures. On LongBench and RULER, both Champion variants consistently improve over their SnapKV and PyramidKV base compressors; the improvements are largest and statistically significant on Llama3.1-70B-Instruct, especially under tight budgets, whereas results on the tested 7B/8B models remain competitive but mixed. ChampionKV does not uniformly outperform Ada-KV, whose global cross-head Top-K selection retains an advantage in several settings. Controlled analysis further shows improved retained attention mass over the corresponding base compressors at negligible overhead.

Figures & tables

Figure 1: Overview of ChampionKV . Given observation-window attention, ChampionKV estimates head-wise attention concentration using the maximum aggregated attention score and adaptively reallocates KV cache budget across heads. Dispersed heads receive larger budgets, while concentrated heads receive smaller budgets, improving retained attention mass without modifying the underlying token-selection procedure.

Figure 2: Standard deviation of aggregated observation-window attention scores across layers and heads in Llama3.1-8B-Instruct on NarrativeQA. The layer–head variation suggests heterogeneous attention concentration.

KV Budget = Full
	Single-Doc QA			Multi-Doc QA			Summarization			Few-shot Learning			Synthetic		Code
Method	NrtvQA	Qasper	MF-en	HotpotQA	2WikiMQA	Musique	GovReport	QMSum	MultiNews	TREC	TriviaQA	SAMSum	PCount	PRe	LCC	RB-P	Avg.
FullKV	27.41	46.02	53.84	53.96	59.35	33.42	34.48	24.79	27.89	68.00	91.33	47.18	14.00	66.00	64.08	54.92	47.92
KV Budget = 64
Ada-KV	27.17	42.69	52.08	44.56	52.41	24.65	23.01	22.53	22.37	67.00	91.27	44.25	14.00	66.00	32.98	56.83	42.74
SnapKV	25.48	33.80	47.02	42.36	44.46	22.83	20.70	21.35	21.09	60.00	88.26	43.15	14.00	66.00	36.98	53.45	40.06

Table 1: LongBench results on Llama3.1-70B-Instruct. Avg. is the mean over all 16 tasks. FullKV is the uncompressed reference; bold marks the best compressed result within each budget.

Panel of Figure 3 — Figure 3: RULER performance on Llama3.1-70B-Instruct under different KV cache ratios.

Panel of Figure 4 — Figure 4: Retained attention mass on randomly sampled LongBench prompts with KV cache budget 512 and observation window size $W=32$ . Baseline denotes the original SnapKV or PyramidKV allocation, while Ours denotes the same compressor after applying Champion head-wise reallocation.

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Concentrated aggregated observation-window attention distribution of Layer 0, Head 31 in Llama3.1-8B-Instruct. The maximum score occurs at token 0 and most remaining historical tokens receive much lower scores.

Panel of Figure 6 — Figure 6: Dispersed aggregated observation-window attention distribution of Layer 0, Head 22 in Llama3.1-8B-Instruct. Although the maximum score occurs at token 0, many other historical tokens receive relatively higher scores across the sequence.

Panel of Figure 7 — Figure 7: Correlation analysis on NarrativeQA with Llama3.1-8B-Instruct with $W=8$ .

Panel of Figure 8 — Figure 8: Correlation analysis on Mistral-7B-Instruct-v0.3 with $W=8$ .

Panel of Figure 9 — Figure 9: Correlation analysis on Llama3.1-70B-Instruct with $W=8$ .

Panel of Figure 10 — Figure 10: Correlation analysis on Llama3.1-8B-Instruct with $W=4$ .

Panel of Figure 11 — Figure 11: Correlation analysis on Llama3.1-8B-Instruct with $W=16$ .

Panel of Figure 12 — Figure 12: Correlation analysis on Llama3.1-8B-Instruct with $W=32$ .

Panel of Figure 13 — Figure 13: Correlation analysis on Llama3.1-8B-Instruct with $W=64$ .

Len.	Slope	$L-W$	$1/\sqrt{L-W}$	Err. (%)
2k	0.022322	1934	0.022739	1.8682
4k	0.015649	3934	0.015943	1.8817
6k	0.012753	5934	0.012982	1.7921
8k	0.011034	7934	0.011227	1.7468
10k	0.009863	9934	0.010033	1.7253
12k	0.009004	11934	0.009154	1.6650

Appendix

Table 2: Scaling relation between the fitted slope and

1/\sqrt{L-W}

on NarrativeQA with Llama3.1-8B-Instruct.

Dataset	$L-W$	Slope	$1/\sqrt{L-W}$	Err. (%)
2WikiMQA	7063	0.011524	0.011899	3.2529
GovReport	7942	0.010821	0.011221	3.6974
HotpotQA	7941	0.010768	0.011222	4.2143
LCC	7918	0.011030	0.011238	1.8865
MultiNews	2656	0.018709	0.019404	3.7135
MF-en	1522	0.024590	0.025633	4.2399

Appendix

Table 3: Slope validation across LongBench datasets on Llama3.1-8B-Instruct with

W=8

Dataset	$L-W$	Slope	$1/\sqrt{L-W}$	Err. (%)
GovReport	7942	0.011058	0.011221	1.4749
HotpotQA	7941	0.010997	0.011222	2.0442
LCC	7918	0.011153	0.011238	0.7629
NrtvQA	7942	0.011121	0.011221	0.9000
PCount	7942	0.011094	0.011221	1.1456
RB-P	7938	0.011169	0.011224	0.4917

Appendix

Table 4: Slope validation across LongBench datasets on Llama3.1-70B-Instruct with

W=8

Dataset	$L-W$	Slope	$1/\sqrt{L-W}$	Err. (%)
2WikiMQA	8093	0.010713	0.011116	3.7610
GovReport	11927	0.008692	0.009157	5.3452
HotpotQA	13132	0.008392	0.008726	3.9846
LCC	17798	0.007337	0.007496	2.1635
MF-en	1877	0.022153	0.023082	4.1922
MultiNews	3104	0.016902	0.017949	6.1943

Appendix

Table 5: Slope validation across LongBench datasets on Mistral-7B-Instruct-v0.3 with

W=8

Method	Mem.	Prefill	Dec.	Total
	(GB)	(ms)	(ms/tok)	(ms)
SnapKV	38.605	3795.82	29.46	4237.64
Ada-KV	38.605	3887.61	44.99	4562.39
Champion-Snap	38.230	3885.97	44.78	4557.73
Champion-Pyramid	38.240	3875.88	42.21	4509.08

Appendix

Table 8: Hardware profiling on Llama3.1-8B-Instruct with a 32,100-token input, 15 generated tokens, and KV cache budget 1024. Bold marks the lowest value in each column.

Dataset	Max len	Dataset	Max len
NrtvQA	128	MultiNews	512
Qasper	128	PCount	32
MF-en	64	PRe	32
HotpotQA	32	Lcc	64
2WikiMQA	32	TriviaQA	32
Musique	32	SAMSum	128

Appendix

Table 9: Maximum decoding lengths for the 16 LongBench datasets used in evaluation.

Method	Input len.	Peak Mem.	Prefill time	Decode Lat.	Total Gen. time
		(GB)	(ms)	(ms/token)	(ms)
Ada-KV	8,100	21.221	1097.12	45.64	1781.68
Champion-Snap	8,100	20.845	800.56	49.43	1542.04
Champion-Pyramid	8,100	20.855	922.82	46.50	1620.34
Ada-KV	16,100	27.014	1677.00	45.79	2363.83
Champion-Snap	16,100	26.640	1652.47	46.33	2347.44

Appendix

Table 10: Hardware profiling of Llama3.1-8B-Instruct across input lengths with 15 generated tokens and KV cache budget 1024.

$\rho$	$\tau$	NrtvQA	HotpotQA	GovReport	TREC	PCount	Lcc	Avg.
0.1	0.01	26.26	45.37	24.97	71.00	7.00	57.88	38.75
0.2	0.0001	26.09	46.39	24.78	70.00	7.00	57.70	38.66
0.2	0.001	26.09	46.39	25.21	70.00	7.00	57.74	38.74
0.2	0.01	27.81	45.87	25.15	70.00	7.00	57.73	38.93
0.5	0.01	25.96	46.95	24.87	71.00	5.00	58.51	38.72

Appendix

Table 11: Hyperparameter analysis of Champion-Snap on Mistral-7B-Instruct-v0.3 over six representative LongBench tasks. Bold marks the best result in each column among the tested configurations.

Model	Dataset	Abnormal instances	Total instances	Aggregate rate / Prompt-level range
Llama3.1-8B-Instruct	GovReport	1,343	102,400	1.31% / 0.59–1.56%
Llama3.1-8B-Instruct	HotpotQA	1,643	102,400	1.60% / 0.39–2.64%
Llama3.1-8B-Instruct	Lcc	1,757	307,200	0.57% / 0.10–1.27%
Llama3.1-8B-Instruct	NarrativeQA	1,700	102,400	1.66% / 1.07–3.42%
Llama3.1-8B-Instruct	PassageCount	1,394	102,400	1.36% / 1.17–1.46%
Llama3.1-8B-Instruct	TREC	1,069	102,400	1.04% / 0.29–1.76%

Appendix

Table 12: Prevalence of abnormal heads under

\tau=0.01

. Total instances count layer–head pairs over all evaluated prompts; the final column reports the aggregate rate followed by the minimum–maximum prompt-level rate.

Model	Input length	NrtvQA	HotpotQA	GovReport	TREC	PCount	Lcc	Avg.
Llama3.1-8B-Instruct	8k	20.32	15.95	25.69	66.00	8.42	60.11	32.75
Llama3.1-8B-Instruct	16k	24.28	14.88	25.51	65.00	8.06	59.42	32.86
Llama3.1-8B-Instruct	32k	28.01	14.36	25.24	66.00	6.64	59.53	33.30
Mistral-7B-Instruct-v0.3	8k	27.81	45.87	25.15	70.00	7.00	57.73	38.93
Mistral-7B-Instruct-v0.3	16k	23.80	47.59	25.54	70.00	7.00	57.71	38.61
Mistral-7B-Instruct-v0.3	32k	26.36	45.87	24.93	70.00	7.00	57.50	38.61

Appendix

Table 13: Champion-Snap results across input lengths on six representative LongBench tasks. Avg. is the mean over the six tasks.

Panel of Figure 14 — Figure 14: RULER performance on Llama3.1-8B-Instruct under different KV cache ratios.

KV Budget = Full
	Single-Doc QA			Multi-Doc QA			Summarization			Few-shot Learning			Synthetic		Code
Method	NrtvQA	Qasper	MF-en	HotpotQA	2WikiMQA	Musique	GovReport	QMSum	MultiNews	TREC	TriviaQA	SAMSum	PCount	PRe	LCC	RB-P	Avg.
FullKV	21.77	43.07	27.82	52.29	43.86	27.05	34.20	22.31	26.86	67.00	88.26	44.07	7.50	65.67	64.66	55.32	43.23
KV Budget = 64
SnapKV	15.04	23.29	21.19	40.06	33.41	20.02	20.64	22.01	19.85	56.00	83.27	40.67	8.17	61.70	57.85	44.69	35.49
Ada-KV	17.78	27.16	22.56	41.80	34.03	20.34	21.75	22.23	21.01	54.00	80.58	39.92	9.03	67.86	56.15	45.63	36.36

Appendix

Table 14: LongBench results on Llama3.1-8B-Instruct. Avg. is the mean over all 16 tasks. FullKV is the uncompressed reference; bold marks the best compressed result within each budget.

KV Budget = Full
	Single-Doc QA			Multi-Doc QA			Summarization			Few-shot Learning			Synthetic		Code
Method	NrtvQA	Qasper	MF-en	HotpotQA	2WikiMQA	Musique	GovReport	QMSum	MultiNews	TREC	TriviaQA	SAMSum	PCount	PRe	LCC	RB-P	Avg.
FullKV	29.21	41.24	54.06	46.08	37.94	24.66	34.71	26.22	28.22	73.00	85.26	47.16	7.00	99.00	62.61	61.65	47.38
KV Budget = 64
SnapKV	24.89	27.46	45.20	40.94	34.85	22.31	19.67	21.37	18.64	53.00	83.79	40.15	8.00	92.00	56.15	50.51	39.93
Ada-KV	25.71	29.03	46.99	45.33	33.06	23.90	21.05	20.87	20.18	67.50	88.95	42.47	4.50	85.00	51.79	50.58	41.06

Appendix

Table 15: LongBench results on Mistral-7B-Instruct-v0.3. Avg. is the mean over all 16 tasks. FullKV is the uncompressed reference; bold marks the best compressed result within each budget.

Method	Ratio	S1	S2	S3	MK1	MK2	MK3	MQ	MV	CWE	FWE	VT	Avg.
FullKV	100%	100.00	100.00	100.00	100.00	100.00	99.22	100.00	95.82	99.52	97.40	99.81	99.25
SnapKV	20%	100.00	100.00	60.21	100.00	100.00	37.88	99.02	96.32	73.99	96.07	100.00	87.59
Ada-KV	20%	100.00	100.00	44.16	100.00	99.42	32.20	99.51	97.21	43.92	95.39	99.85	82.88
Champion-Snap	20%	100.00	100.00	62.36	100.00	99.04	38.56	99.76	96.80	77.26	96.09	99.81	88.15
SnapKV	40%	100.00	100.00	87.38	100.00	100.00	41.41	100.00	95.05	97.69	97.39	99.81	92.61
Ada-KV	40%	100.00	100.00	80.84	100.00	99.81	37.43	99.90	96.81	85.85	96.36	99.96	90.63

Appendix

Table 16: Detailed RULER subtask results on Llama3.1-8B-Instruct under different KV cache retention ratios, including Ada-KV.

Method	Ratio	S1	S2	S3	MK1	MK2	MK3	MQ	MV	CWE	FWE	VT	Avg.
FullKV	100%	100.00	100.00	100.00	100.00	100.00	97.55	81.08	94.23	98.81	94.28	99.43	96.85
SnapKV	20%	100.00	100.00	80.63	100.00	98.25	31.49	74.24	78.74	86.59	90.21	97.70	85.26
Ada-KV	20%	100.00	99.76	84.51	99.65	97.20	26.10	68.89	74.90	32.07	86.82	97.23	78.83
Champion-Snap	20%	100.00	100.00	84.19	100.00	96.61	35.75	74.02	80.58	84.41	90.96	98.05	85.87
SnapKV	40%	100.00	100.00	95.99	100.00	98.48	49.70	81.12	92.98	96.81	93.05	99.24	91.58
Ada-KV	40%	100.00	100.00	96.93	99.66	97.94	32.82	80.50	91.21	62.85	92.14	98.96	86.64

Appendix

Table 17: Detailed RULER subtask results on Mistral-7B-Instruct-v0.3 under different KV cache retention ratios, including Ada-KV.

Method	Ratio	S1	S2	S3	MK1	MK2	MK3	MQ	MV	CWE	FWE	VT	Avg.
FullKV	100%	100.00	100.00	100.00	100.00	100.00	100.00	100.00	100.00	100.00	99.70	100.00	99.97
SnapKV	20%	100.00	100.00	100.00	100.00	100.00	44.15	100.00	100.00	99.52	97.78	100.00	94.68
Ada-KV	20%	100.00	100.00	98.02	100.00	100.00	46.57	99.96	99.96	99.83	97.29	99.90	94.68
Champion-Snap	20%	100.00	100.00	100.00	100.00	100.00	57.96	100.00	100.00	100.00	98.15	100.00	96.01
SnapKV	40%	100.00	100.00	100.00	100.00	100.00	68.27	100.00	100.00	100.00	97.81	100.00	96.92
Ada-KV	40%	100.00	100.00	96.34	100.00	100.00	76.43	100.00	99.92	99.91	98.81	99.97	97.40

Appendix

Table 18: Detailed RULER subtask results on Llama3.1-70B-Instruct under different KV cache retention ratios, including Ada-KV.

Model	Plug-in	Benchmark	Pairs	W	T	L	Win%	Mean $\Delta$	Median $\Delta$	Wilcoxon $p$
Llama3.1-70B	Champion-Snap	LongBench	64	44	12	8	68.8	+0.763	+0.440	$<0.001$
Llama3.1-70B	Champion-Pyramid	LongBench	64	39	12	13	60.9	+0.648	+0.265	$<0.001$
Llama3.1-8B	Champion-Snap	LongBench	64	28	0	36	43.8	$-0.031$	$-0.115$	0.55
Llama3.1-8B	Champion-Pyramid	LongBench	64	27	5	32	42.2	+0.010	$-0.050$	0.48
Mistral-7B	Champion-Snap	LongBench	64	25	6	33	39.1	$-0.350$	$-0.055$	0.18
Mistral-7B	Champion-Pyramid	LongBench	64	25	2	37	39.1	$-0.177$	$-0.250$	0.11

Appendix

Table 19: Paired delta statistics for ChampionKV plug-ins. Paired per-unit deltas are Champion minus baseline at (task, budget) granularity for LongBench (16 tasks

\times

4 budgets

=64

) and at (subtask, ratio) granularity for RULER (

11\times 4=44

). W/T/L count pairs with positive/zero/negative deltas; the two-sided Wilcoxon signed-rank test is over nonzero deltas. Because many RULER subtasks are saturated (ties), the win rate alone understates positive gains; the test uses the sign and magnitude of nonzero deltas.

KV-Cache Compression

Key-value cache compression reduces the memory and bandwidth needed to retain attention states during long-context inference.

See all 257 papers

KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation
Sep 21, 2026Sihyeon Ha, Jaeho Lee, Yo-Seb JeonLow-Rank Compression KV-Cache Compression
Pohang University of Science and Technology (POSTECH)
KVSculpt: KV Cache Compression as Distillation
Mar 29, 2026Bo Jiang, Sian JinKV-Cache Management KV-Cache Compression
Temple University
LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction
Apr 22, 2026Enshuai Zhou, Yifan Hao, Chao Wang +7KV Caching Long-Context Language Model Inference
University of Science and Technology of China, Hefei, China · State Key Lab of Processors, Institute of Computing Technology, CAS, Beijing, China · University of Chinese Academy of Sciences, Beijing, China

Sep 21, 2026J/K move · Enter open · S save

KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation

Sihyeon Ha, Jaeho Lee, Yo-Seb Jeon

Pohang University of Science and Technology (POSTECH)

Low-Rank Compression KV-Cache Compression KV-Cache Quantization Memory Compression

Open paper AskPDF arXiv

What limits KV-cache compression at extreme bit-rates? We argue that it is not the choice of compression scheme, but how its budget is allocated across attention heads. Existing methods apply rank and bit-width uniformly, ignoring that each head has a different optimal mix of rank truncation and quantization. We show that co-optimizing rank and bit-width per head, using only standard low-rank projection and scalar quantization, dominates uniform allocation, with the largest gains at low bit-rates. Our method, KV-COBRA (Co-Optimized Bit-Rank Allocation), formalizes this as a resource-allocation problem: it balances rank-truncation loss against quantization loss within each head, then redistributes budget across heads to minimize total distortion. A fused Hadamard rotation equalizes per-channel variance, and reordering the SVD basis by attention-KL importance makes the solver query-aware. The same allocator extends to joint

K{+}V

compression. On perplexity, zero-shot, and long-context benchmarks from

0.5

4

bits per dimension (bpd), KV-COBRA shows the smallest accuracy degradation among evaluated methods at low bpd, with no per-token overhead.

Abstract

KV-Cache Compression

Explore similar work

KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation

KVSculpt: KV Cache Compression as Distillation

LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction