QK-Wanda: Coupling Queries and Keys for Unstructured Pruning

Ivan Ilin; Peter Richtárik

cs.LGOct 1, 2026

QK-Wanda: Coupling Queries and Keys for Unstructured Pruning

Authors: Ivan Ilin, Peter Richtárik

Organizations: KAUST

Abstract

Wanda (Sun et al., 2024) prunes large language models by scoring weights independently within each linear projection, although queries and keys interact through dot products. We introduce QK-Wanda, which scores query and key weights by their individual deletion costs under an unmasked pre-RoPE reconstruction objective. It augments Wanda scores with information from the opposite projection (keys for query weights, and queries for key weights), allowing both projections to share a pruning budget. Its closed-form scores require no gradients or weight updates; full pruning takes 1.3% longer than Wanda on A100 and 3.1% longer on H200 with the calibration used in our main experiments. We evaluate QK-only pruning across 15 models from TinyLlama, Llama 2, Llama 3, and Qwen2.5, spanning 0.5B-72B parameters. Relative to Wanda, QK-Wanda reduces QK reconstruction error by an average of 60% at 50% sparsity and 45% at 80%. Downstream gains depend on the model. At 80% sparsity on Llama 2 70B, WikiText-2 and C4 perplexity decrease by 20.3% and 13.5%, while mean zero-shot accuracy rises by 5.94 percentage points. Qwen2.5-72B also improves, but Llama-3.1-70B has substantially higher perplexity despite lower reconstruction error. These results show both the promise of coupled pruning criteria and the limits of local reconstruction as a predictor of model quality.

Figures & tables

Figure 1: Wanda versus QK-Wanda for one token,

X=(1,2)^{\top}

(

d=2,T=1

). Orange/green factors are input/opposite-projection norms;

\times

denotes elementwise multiplication with row/column broadcasting. Dashed boxes mark budget pools. Both prune 50% overall: Wanda removes 50% per row; QK-Wanda removes 75% of Q and 25% of K. Hats denote pruned weights.

Method	Complexity	Llama-3.2 1B	Llama-3.2 3B	Llama-3.1 8B	Llama-3.1 70B	Avg. ratio
Wanda	$O(NTd(D_{Q}+D_{K}))$	6.59 s	17.02 s	34.57 s	284.37 s	1.00
QK-Wanda	$O(NTd(D_{Q}+D_{K}))$	7.12 s	17.45 s	34.98 s	285.98 s	1.03

Table 1: Full QK pruning time on one H200:

8192\times 64

calibration tokens, batch 32. Includes calibration, scoring, mask selection, zeroing and pruned propagation; excludes loading and setup transfers. Medians of five runs; complexity covers QK projections and scoring per block.

Figure 2: QK-only pruning of Llama-2-70B. Left and center: WikiText-2 PPL at low and high sparsity. Right: mean relative QK reconstruction error ( Equation 49 ). Matrix-budget Wanda and separate-budget QK-Wanda remove the same number of weights from each projection. C4 results: Figure 34 ; additional model curves: Appendices I and J .

		Llama-2-70B				Llama-3.1-70B				Qwen2.5-72B
Method	Sparsity	Wiki2 PPL	C4 PPL	Zero-shot acc.	Rel. QK error	Wiki2 PPL	C4 PPL	Zero-shot acc.	Rel. QK error	Wiki2 PPL	C4 PPL	Zero-shot acc.	Rel. QK error
Dense	0%	3.32	5.71	71.50	0.00	2.81	7.11	74.21	0.00	3.87	9.26	74.75	0.00
Wanda	30%	3.32	5.71	71.50	0.04	2.82	7.12	74.16	0.04	3.88	9.26	74.68	0.04
Wanda (matrix)	30%	3.32	5.71	71.49	0.04	2.86	7.15	73.95	0.18	3.89	9.26	75.02	0.05
QK-Wanda	30%	3.32	5.71	71.54	0.00	2.82	7.12	74.23	0.02	3.88	9.26	74.66	0.01
QK-Wanda-MRC	30%	3.32	5.71	71.65	0.01	2.83	7.14	74.07	0.08	3.88	9.26	74.64	0.02

Table 2: QK-only pruning of the largest checkpoints: WikiText-2/C4 PPL (

\downarrow

), seven-task mean zero-shot accuracy (

\uparrow

), and QK reconstruction error (

\downarrow

). Accuracy and error are percentages. Wanda uses row budgets, Wanda (matrix) separate projection budgets, and both QK-Wanda variants shared budgets. Bold marks the best result at each sparsity, using unrounded values. Full model and task results appear in Appendices G , H , and N .

Figure 3: QK-Wanda redistributes a shared 50% pruning budget between query and key weights across transformer blocks. The left panel shows Llama-2-7B/13B (MHA); the right shows Llama-2-70B and Qwen2.5-72B (GQA). Solid and dashed lines show the percentages of query and key weights removed, respectively. Profiles for all 15 checkpoints appear in Appendix E.6 .

		TinyLlama-1.1B			Llama-2-7B			Llama-2-13B
Method	Sparsity	Wiki2 PPL	C4 PPL	Zero-shot acc. (%)	Wiki2 PPL	C4 PPL	Zero-shot acc. (%)	Wiki2 PPL	C4 PPL	Zero-shot acc. (%)
Dense	0%	7.77	9.94	49.93	5.47	7.26	64.38	4.88	6.73	67.05
Wanda	30%	8.33	10.46	50.22	5.66	7.50	63.88	5.06	6.90	66.14
Wanda + QK-Wanda	30%	8.31	10.44	49.96	5.66	7.49	64.02	5.05	6.90	66.46
Wanda	50%	11.52	13.85	48.22	6.89	9.06	60.23	5.90	8.00	64.61
Wanda + QK-Wanda	50%	11.22	13.62	48.56	6.84	9.05	61.02	5.86	7.98	64.55

Table 3: Pruning all seven transformer-block projections at 30% and 50% sparsity. Wanda + QK-Wanda combines shared-budget QK pruning with row-wise Wanda on the remaining projections. Accuracy is the seven-task zero-shot mean (%). Bold marks better unrounded values within each checkpoint and sparsity.

Appendix figures & tables84 assets

Supplementary material from the paper’s appendix.

Appendix

Dimensions and indices
$d,\ d_{\mathrm{out}},\ m$	–	Projection input width, output width, and attention-head width, respectively.
$N,\ s,\ T,\ T_{s}$	–	Calibration sequence count/index; common length and length of sequence $s$ .
$i,j;\ t,u$	–	Output/input weight coordinates; query/key token positions.
$D,\ell;\ f$	–	Number/index of transformer blocks; MLP intermediate width.
$H_{Q},H_{K};\ h,g$	–	Numbers of query/key heads and their respective indices.
$\mathcal{H}_{g}$	–	Query heads sharing key head $g$ ; these groups partition all query heads.

Appendix

Table 4: Notation used in the main text and appendices. Derivations fix one block;

h,g

index its heads and

s

its input sequences. The block index

\ell

appears in cross-block metrics. Indices are suppressed only when their objects are fixed.

		WikiText-2 PPL		C4 PPL
Model	Sparsity	QK-Wanda-M	QK-Wanda	QK-Wanda-M	QK-Wanda
Llama-2-7B	30%	5.478	5.478	7.272	7.272
	50%	5.530	5.529	7.358	7.357
	80%	8.328	8.313	11.213	11.197
Llama-2-13B	30%	4.888	4.888	6.733	6.733
	50%	4.933	4.933	6.792	6.792

Appendix

Table B.1: Causal QK-Wanda-M versus unmasked QK-Wanda with shared QK budgets. Both variants use the same checkpoint and calibration protocol. Lower perplexity is better; bold marks the better value in each pair, including ties at the displayed precision.

Model	Sparsity	Score	WikiText-2 PPL	C4 PPL	Post-RoPE QK error (%)
Llama-3.2-1B	50%	M	$\mathbf{11.1933\,\pm\,0.0196}$	$\mathbf{17.8518\,\pm\,0.0687}$	$0.20$
		MR	$12.2233\,\pm\,0.0195$	$21.0389\,\pm\,0.0728$	$\mathbf{0.18}$
Llama-3.2-1B	80%	M	$\mathbf{192.6108\,\pm\,0.4938}$	$\mathbf{261.1174\,\pm\,2.7507}$	$\mathbf{2.88}$
		MR	$232.3745\,\pm\,0.8535$	$269.6802\,\pm\,1.5493$	$3.04$
Llama-3.2-3B	50%	M	$\mathbf{8.0829\,\pm\,0.0012}$	$\mathbf{11.8228\,\pm\,0.0010}$	$0.12$
		MR	$8.1256\,\pm\,0.0014$	$11.8965\,\pm\,0.0009$	$\mathbf{0.11}$

Appendix

Table 5: Causal pre-RoPE (M) and causal-plus-RoPE (MR) with separate matrix budgets and

256\times 32

calibration tokens. PPL is the mean and sample standard deviation over three calibration seeds. Post-RoPE QK error is the pooled relative squared error (%), averaged over seeds. Bold marks lower displayed means within each checkpoint and sparsity, including ties.

Checkpoint	Total (B)	QK (M)	QK / total (%)	QK / block linear (%)
TinyLlama-1.1B	1.100	103.8	9.44	10.71
Llama-2-7B	6.738	1073.7	15.93	16.58
Llama-2-13B	13.016	2097.2	16.11	16.53
Llama-2-70B	68.977	6039.8	8.76	8.82
Llama-3.2-1B	1.236	83.9	6.79	8.62
Llama-3.2-3B	3.213	352.3	10.97	12.50

Appendix

Table 6: Scope of the QK-only experiments. B and M denote billions and millions of parameters. The last two columns use different denominators: the complete model and its seven block projection weight matrices.

meta-llama/Llama-3.1-8B
Revision	d04e592bb4f6aa9cfee91e2e20afa771667e1d4b
meta-llama/Llama-3.2-1B
Revision	4e20de362430cd3b72f300e6b0f18e50e7166e08
meta-llama/Llama-3.2-3B
Revision	13afe5124825b4f3751f836b40dafda64c1ed062
TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T

Appendix

Table 7: Hugging Face repository identifiers and pinned revisions used for both model and tokenizer.

Study bundle	Source-manifest SHA256
qk_budget	91360064ecb2be26de3ffe3450262f6f
	3f5961c4569c460920b14e3c1b6b788f
qk_budget_1b	10987a5ad0b19d3fbdea2dc1f83c8d6a
	339c0460ab8e001d4e152e20d62fdc5b
qk_budget_3b	291a3cdcb13802ca1a7f9a8e5770eaf8
	90d80f8de1d1b03a54e352da645d43a0

Appendix

Table 8: Frozen source-manifest SHA256 identifiers recorded in study plans; each hash continues on the following line. Retained runs also record mask and result hashes.

Figure 4: QK-Wanda allocation at a shared 50% budget on Llama 3. Solid lines show query-weight removal and dashed lines key-weight removal; colors and markers identify checkpoints. The dotted line marks 50% removal in each projection.

Figure 5: QK-Wanda allocation at a shared 50% budget on TinyLlama and Llama 2. Solid lines show query-weight removal and dashed lines key-weight removal; colors and markers identify checkpoints. The dotted line marks 50% removal in each projection.

Figure 6: QK-Wanda allocation with a shared 50% budget on Qwen2.5, with 0.5B–3B models in the upper panel and 7B–72B models in the lower panel. Color and marker shape identify the checkpoint within each panel. Solid lines denote

W^{Q}

, dashed lines denote

W^{K}

, and the dotted line marks separate 50% budgets. Each curve ends at its checkpoint’s final block; 72B extends to block 80.

		TinyLlama-1.1B			Llama-2-7B			Llama-2-13B
Method	Sparsity	Wiki2 PPL	C4 PPL	Zero-shot acc. (%)	Wiki2 PPL	C4 PPL	Zero-shot acc. (%)	Wiki2 PPL	C4 PPL	Zero-shot acc. (%)
Dense	0%	7.77	9.94	49.93	5.47	7.26	64.38	4.88	6.73	67.05
Wanda	30%	8.33	10.46	50.22	5.66	7.50	63.88	5.06	6.90	66.14
Wanda + QK-Wanda-M	30%	8.31	10.44	49.94	5.66	7.49	64.03	5.05	6.89	66.39
Wanda	50%	11.52	13.85	48.22	6.89	9.06	60.23	5.90	8.00	64.61
Wanda + QK-Wanda-M	50%	11.22	13.62	48.42	6.85	9.06	61.09	5.86	7.98	64.76

Appendix

Table 9: Pruning all seven transformer-block projections (QK/V/O and MLP gate/up/down), with embeddings and the output head excluded from the sparsity denominator. Wanda uses row budgets throughout; Wanda + QK-Wanda-M uses shared QK-Wanda-M only for QK and row-wise Wanda elsewhere. Accuracy is the seven-task mean (%); seed 0. Bold marks lower displayed PPL or higher accuracy at each sparsity, including ties.

		Wanda PPL		QK-Wanda-M PPL
Samples	Tokens	WikiText-2	C4	WikiText-2	C4
32	65,536	6.4165	9.8760	6.4585	9.9249
64	131,072	6.4163	9.8757	6.4573	9.9199
128	262,144	6.4161	9.8753	6.4583	9.9210
256 $\dagger$	524,288	6.4172	9.8781	6.4580	9.9186
512	1,048,576	6.4162	9.8759	6.4569	9.9194

Appendix

Table 10: Sample-count sensitivity on Llama-3.1-8B at 50% QK sparsity, length 2048, and seed 0. Wanda uses row budgets and QK-Wanda-M separate matrix budgets. Evaluation context is 2048. Bold marks within-method, within-dataset minima at the displayed precision;

\dagger

marks the selected common default, independently of boldface.

		Wanda		QK-Wanda-M		QK-Wanda
Length	Sequences	WikiText-2	C4	WikiText-2	C4	WikiText-2	C4
4	131,072	6.4076	9.8574	6.4722	9.9447	6.4728	9.9415
8	65,536	6.4036	9.8523	6.4599	9.9185	6.4596	9.9169
16	32,768	6.4029	9.8506	6.4522	9.9028	6.4515	9.9016
32	16,384	6.4043	9.8494	6.4458	9.8970	6.4464	9.8966
64 $\dagger$	8,192	6.4038	9.8527	6.4437	9.8923	6.4425	9.8925

Appendix

Table 11: Context-length sensitivity on Llama-3.1-8B at 50% QK sparsity and seed 0, with exactly 524,288 calibration tokens. QK-Wanda-M and QK-Wanda use separate matrix-wide budgets for Q and K; M uses causal pre-RoPE scores, while the unsuffixed method uses unmasked scores. Evaluation context is 2048. Bold marks within-method, within-dataset minima at displayed precision;

\dagger

marks the selected common default independently of boldface.

		QK-Wanda-M		QK-Wanda
Length	Sequences	WikiText-2	C4	WikiText-2	C4
4	131,072	5.5459	7.3785	5.5469	7.3743
8	65,536	5.5391	7.3700	5.5388	7.3672
16	32,768	5.5356	7.3641	5.5355	7.3621
32	16,384	5.5345	7.3616	5.5334	7.3606
64 $\dagger$	8,192	5.5319	7.3596	5.5326	7.3588

Appendix

Table 12: Context-length sensitivity on Llama-2-7B at 50% QK sparsity and seed 0, with exactly 524,288 calibration tokens. Both QK-Wanda variants use separate matrix-wide budgets for Q and K; M uses causal pre-RoPE scores and the unsuffixed method uses unmasked scores. Evaluation context is 2048. Bold marks within-method, within-dataset minima at displayed precision;

\dagger

marks the common default chosen from the Llama-3.1-8B sweep.

		Llama-3.2-1B				Llama-3.2-3B
Method	Sparsity	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)
Dense	0%	9.75	14.01	54.14	0.00	7.81	11.33	61.74	0.00
Wanda	30%	9.83	14.15	54.78	0.23	7.85	11.41	61.35	0.15
QK-Wanda-M	30%	9.81	14.14	54.28	0.09	7.84	11.38	61.55	0.07
QK-Wanda (shared)	30%	9.81	14.14	54.18	0.08	7.84	11.38	61.57	0.07
QK-Wanda (sep.)	30%	9.82	14.17	54.17	0.11	7.84	11.38	61.64	0.08

Appendix

Table 13: QK-only pruning on Llama 3. Each checkpoint has WikiText-2 PPL, C4 PPL, seven-task mean accuracy (%), and relative QK error (%; Equation 49 ). QK-Wanda is shown with shared and separate (sep.) budgets; the labeled variants use shared budgets. Centered variants are defined in Appendix O . Bold marks the best displayed result within each sparsity, including ties; dashes denote variants not evaluated.

		Llama-3.1-8B				Llama-3.1-70B
Method	Sparsity	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)
Dense	0%	6.24	9.54	69.10	0.00	2.81	7.11	74.21	0.00
Wanda	30%	6.26	9.58	68.80	0.12	2.82	7.12	74.16	0.04
QK-Wanda-M	30%	6.25	9.57	69.40	0.05	2.82	7.12	74.18	0.02
QK-Wanda (shared)	30%	6.25	9.57	69.29	0.05	2.82	7.12	74.23	0.02
QK-Wanda (sep.)	30%	6.26	9.58	69.31	0.06	2.82	7.13	74.08	0.03

Appendix

Table 21

		TinyLlama-1.1B (GQA)				Llama-2-7B (MHA)
Method	Sparsity	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)
Dense	0%	7.77	9.94	49.93	0.00	5.47	7.26	64.38	0.00
Wanda	30%	7.79	9.97	49.89	0.08	5.49	7.28	64.00	0.09
QK-Wanda-M	30%	7.78	9.96	50.02	0.03	5.48	7.27	64.24	0.03
QK-Wanda (shared)	30%	7.78	9.96	49.95	0.03	5.48	7.27	64.15	0.03
QK-Wanda (sep.)	30%	7.79	9.96	49.96	0.03	5.48	7.27	64.32	0.03

Appendix

Table 14: QK-only pruning on TinyLlama and Llama 2. Columns report WikiText-2 PPL, C4 PPL, seven-task mean accuracy (%), and relative QK error (%; Equation 49 ). QK-Wanda uses shared or separate (sep.) budgets; the labeled variants use shared budgets. Bold marks the best displayed result at each sparsity, including ties.

		Llama-2-13B (MHA)				Llama-2-70B (GQA)
Method	Sparsity	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)
Dense	0%	4.88	6.73	67.05	0.00	3.32	5.71	71.50	0.00
Wanda	30%	4.90	6.74	66.77	0.06	3.32	5.71	71.50	0.04
QK-Wanda-M	30%	4.89	6.73	66.92	0.02	3.32	5.71	71.62	0.00
QK-Wanda (shared)	30%	4.89	6.73	66.84	0.02	3.32	5.71	71.54	0.00
QK-Wanda (sep.)	30%	4.89	6.73	66.86	0.02	3.32	5.71	71.59	0.01

Appendix

Table 23

		Qwen2.5-0.5B				Qwen2.5-1.5B				Qwen2.5-3B
Method	Sparsity	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)
Dense	0%	13.07	20.38	50.86	0.00	9.26	15.12	61.64	0.00	8.03	13.35	65.29	0.00
Wanda	30%	13.12	20.43	50.48	0.07	9.30	15.15	61.45	0.08	8.06	13.37	65.22	0.06
QK-Wanda-M	30%	13.09	20.40	50.97	0.02	9.27	15.13	61.69	0.02	8.05	13.37	65.31	0.02
QK-Wanda (shared)	30%	13.09	20.40	50.98	0.02	9.27	15.13	61.82	0.02	8.05	13.37	65.34	0.02
QK-Wanda (sep.)	30%	13.10	20.41	50.73	0.02	9.28	15.14	61.60	0.03	8.05	13.37	65.20	0.03

Appendix

Table 15: QK-only pruning on Qwen2.5 0.5B–32B. Columns report WikiText-2 PPL, C4 PPL, seven-task mean accuracy (%), and relative QK error (%; Equation 49 ). QK-Wanda uses shared or separate (sep.) budgets; the labeled variants use shared budgets. Biases remain dense. Bold marks the best displayed result at each sparsity, including ties; dashes denote variants not evaluated.

		Qwen2.5-7B				Qwen2.5-14B				Qwen2.5-32B
Method	Sparsity	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)	Wiki2 PPL	C4 PPL	Zero shot acc.	Rel. QK error (%)
Dense	0%	6.85	11.88	70.56	0.00	5.29	10.35	72.35	0.00	5.02	10.17	72.36	0.00
Wanda	30%	6.86	11.89	70.45	0.03	5.31	10.36	72.62	0.06	5.03	10.18	72.48	0.05
QK-Wanda-M	30%	6.85	11.89	70.44	0.01	5.30	10.35	72.59	0.02	5.02	10.17	72.32	0.02
QK-Wanda (shared)	30%	6.85	11.89	70.54	0.01	5.30	10.35	72.66	0.02	5.02	10.17	72.33	0.02
QK-Wanda (sep.)	30%	6.86	11.89	70.48	0.01	5.30	10.35	72.58	0.03	5.02	10.17	72.33	0.02

Appendix

Table 25

Method	Sparsity	Wiki2 PPL	C4 PPL	Zero-shot acc. (%)	Rel. QK error (%)
Dense	0%	3.87	9.26	74.75	0.00
Wanda	30%	3.88	9.26	74.68	0.04
QK-Wanda-M	30%	3.88	9.26	74.61	0.01
QK-Wanda (shared)	30%	3.88	9.26	74.66	0.01
QK-Wanda (sep.)	30%	3.88	9.26	74.63	0.02
QK-Wanda-MRC	30%	3.88	9.26	74.64	0.02

Appendix

Table 16: QK-only pruning on Qwen2.5-72B in BF16. QK-Wanda uses shared or separate (sep.) budgets; the labeled variants use shared budgets. Columns report WikiText-2 PPL, C4 PPL, seven-task mean accuracy (%), and relative QK error (%; Equation 49 ). Bold marks the best displayed result at each sparsity, including ties.

Subset / comparison	WikiText-2	C4	QK error	Accuracy
All sparsities	102/135	98/135	133/135	27/45
Tabulated sparsities	35/45	33/45	44/45	27/45
TinyLlama / Llama 2	35/36	35/36	36/36	11/12
Llama 3	13/36	13/36	34/36	6/12
Qwen2.5	54/63	50/63	63/63	10/21
30% sparsity	14/15	14/15	15/15	10/15

Appendix

Table 17: Favorable comparisons / evaluated configurations. Shared QK-Wanda-M is compared with row-wise Wanda unless indicated. Accuracy uses the unweighted seven-task mean.

Comparison	WikiText-2	C4	Full QK error	Accuracy
All sparsities	103/135	101/135	133/135	30/45
Tabulated sparsities	35/45	35/45	44/45	30/45
Shared vs. separate QK	112/135	119/135	122/135	—
Separate QK vs. row Wanda	95/135	85/135	124/135	—

Appendix

Table 18: Favorable comparisons for QK-Wanda. Shared QK budgets are compared with row-wise Wanda unless indicated; accuracy is the unweighted seven-task mean. Historical masked counts above remain separately labeled.

Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
TinyLlama-1.1B
Dense	0%	57.77	51.99	59.16	59.04	55.35	30.20	36.00	49.93
Wanda	30%	57.89	51.99	58.96	58.56	55.26	30.38	36.20	49.89
QK-Wanda-M	30%	58.50	52.35	59.11	58.41	55.30	30.46	36.00	50.02
QK-Wanda (shared)	30%	58.53	51.62	59.12	58.48	55.43	30.46	36.00	49.95
QK-Wanda (sep.)	30%	58.01	51.99	59.06	58.64	55.30	30.72	36.00	49.96

Appendix

Table 19: Seven-task zero-shot accuracy (%) for TinyLlama, Llama 2, and Llama 3 with QK-only pruning. Each model includes its dense reference. QK-Wanda uses shared or separate (sep.) budgets; other QK variants use shared budgets. Centered variants are described in Appendix O . Metrics follow Appendix E.2 ; Avg. is their unweighted mean. Bold marks the best accuracy within each model and sparsity, including ties. Dashes mark variants not evaluated in this comparison.

Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Llama-2-7B
Dense	0%	77.74	62.82	75.98	69.06	74.54	46.33	44.20	64.38
Wanda	30%	76.73	61.73	75.76	68.75	74.79	45.82	44.40	64.00
QK-Wanda-M	30%	77.46	62.45	75.99	68.67	74.75	46.16	44.20	64.24
QK-Wanda (shared)	30%	77.43	61.73	76.01	68.82	74.79	46.25	44.00	64.15
QK-Wanda (sep.)	30%	77.49	62.82	75.92	68.75	74.75	46.33	44.20	64.32

Appendix

Table 30

Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Llama-2-13B
Dense	0%	80.61	65.34	79.38	72.14	77.53	49.15	45.20	67.05
Wanda	30%	80.37	63.18	79.33	71.98	77.48	49.49	45.60	66.77
QK-Wanda-M	30%	80.73	63.90	79.42	72.22	77.57	48.98	45.60	66.92
QK-Wanda (shared)	30%	80.64	63.90	79.41	71.90	77.48	48.98	45.60	66.84
QK-Wanda (sep.)	30%	80.73	63.54	79.43	72.14	77.57	49.23	45.40	66.86

Appendix

Table 31

Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Llama-3.2-1B
Dense	0%	64.01	56.68	63.67	60.62	60.44	36.35	37.20	54.14
Wanda	30%	64.40	60.29	63.49	61.09	60.31	36.86	37.00	54.78
QK-Wanda-M	30%	64.37	57.40	63.65	60.30	60.40	36.43	37.40	54.28
QK-Wanda (shared)	30%	64.22	56.68	63.66	60.14	60.65	36.52	37.40	54.18
QK-Wanda (sep.)	30%	64.10	57.40	63.49	59.83	60.61	36.77	37.00	54.17

Appendix

Table 32

Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Llama-3.2-3B
Dense	0%	73.24	54.51	73.59	70.01	71.72	46.08	43.00	61.74
Wanda	30%	73.49	53.43	73.40	69.22	71.63	45.65	42.60	61.35
QK-Wanda-M	30%	73.21	54.15	73.56	69.30	71.63	46.42	42.60	61.55
QK-Wanda (shared)	30%	73.30	54.15	73.59	69.38	71.63	46.16	42.80	61.57
QK-Wanda (sep.)	30%	73.21	54.51	73.49	69.14	71.80	46.50	42.80	61.64

Appendix

Table 33

Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Llama-3.1-8B
Dense	0%	82.11	69.68	78.88	73.64	81.10	53.50	44.80	69.10
Wanda	30%	81.80	68.23	78.60	73.09	81.10	53.58	45.20	68.80
QK-Wanda-M	30%	82.32	70.76	79.05	73.64	81.19	53.67	45.20	69.40
QK-Wanda (shared)	30%	82.29	70.04	79.02	73.80	81.10	53.58	45.20	69.29
QK-Wanda (sep.)	30%	82.48	70.40	79.03	73.64	80.93	53.50	45.20	69.31

Appendix

Table 34

Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Qwen2.5-0.5B
Dense	0%	62.42	58.84	52.15	56.27	58.71	32.42	35.20	50.86
Wanda	30%	61.10	58.48	51.95	56.04	58.29	32.51	35.00	50.48
QK-Wanda-M	30%	61.71	59.21	52.16	57.46	58.50	32.34	35.40	50.97
QK-Wanda (shared)	30%	62.11	59.57	52.13	56.99	58.59	32.25	35.20	50.98
QK-Wanda (sep.)	30%	61.65	58.12	52.10	57.54	58.42	31.91	35.40	50.73

Appendix

Table 20: QK-Wanda is shown with shared and separate (sep.) budgets; dashes mark variants not evaluated in this comparison. Zero-shot accuracy (%) on Qwen2.5 base checkpoints with QK-only unstructured pruning. Each model section contains its dense reference and original row-wise Wanda versus shared-budget QK-Wanda-M . Centered QK-Wanda-MC is additionally reported for 0.5B/1.5B/3B/7B (Appendix O ). Task metrics follow Appendix E.2 ; Avg. is their unweighted mean. Bold marks the highest displayed accuracy within each model and sparsity, including ties.

Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Qwen2.5-1.5B
Dense	0%	72.94	70.04	67.77	63.30	71.51	45.14	40.80	61.64
Wanda	30%	73.09	68.95	67.82	63.54	71.13	44.62	41.00	61.45
QK-Wanda-M	30%	73.09	70.76	67.75	63.85	70.62	44.97	40.80	61.69
QK-Wanda (shared)	30%	72.94	71.48	67.72	64.17	70.88	44.80	40.80	61.82
QK-Wanda (sep.)	30%	72.48	70.40	67.86	63.85	70.79	44.80	41.00	61.60

Appendix

Table 36

Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Qwen2.5-3B
Dense	0%	77.22	75.45	73.53	68.35	73.11	47.35	42.00	65.29
Wanda	30%	77.13	73.65	73.46	69.38	72.90	47.61	42.40	65.22
QK-Wanda-M	30%	77.58	74.01	73.60	68.59	73.82	47.35	42.20	65.31
QK-Wanda (shared)	30%	77.37	74.73	73.57	68.27	73.78	47.44	42.20	65.34
QK-Wanda (sep.)	30%	77.49	74.01	73.57	68.67	73.06	47.01	42.60	65.20

Appendix

Table 37

Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Qwen2.5-7B
Dense	0%	84.65	81.59	78.94	73.09	77.36	51.11	47.20	70.56
Wanda	30%	84.53	81.95	78.90	73.16	77.27	50.77	46.60	70.45
QK-Wanda-M	30%	84.46	81.59	78.99	72.93	77.31	50.77	47.00	70.44
QK-Wanda (shared)	30%	84.53	81.59	79.00	72.93	77.40	51.11	47.20	70.54
QK-Wanda (sep.)	30%	84.37	81.59	78.96	72.85	77.36	51.02	47.20	70.48

Appendix

Table 38

Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Qwen2.5-14B
Dense	0%	85.26	80.14	82.88	74.98	79.12	58.87	45.20	72.35
Wanda	30%	85.41	80.14	82.88	76.40	79.21	58.70	45.60	72.62
QK-Wanda-M	30%	85.54	80.14	82.96	75.06	79.59	59.47	45.40	72.59
QK-Wanda (shared)	30%	85.44	80.14	82.93	75.61	79.50	59.56	45.40	72.66
QK-Wanda (sep.)	30%	85.72	80.51	82.90	75.61	79.25	58.70	45.40	72.58

Appendix

Table 39

Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Qwen2.5-32B
Dense	0%	87.16	81.59	84.07	75.30	78.28	55.72	44.40	72.36
Wanda	30%	87.31	81.23	84.06	75.61	78.28	56.48	44.40	72.48
QK-Wanda-M	30%	87.40	80.87	84.02	75.30	78.20	56.23	44.20	72.32
QK-Wanda (shared)	30%	87.43	80.87	84.03	75.22	78.16	56.23	44.40	72.33
QK-Wanda (sep.)	30%	87.34	80.87	84.05	75.45	78.03	56.14	44.40	72.33

Appendix

Table 40

Llama-2-70B
Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Dense	0%	83.76	67.87	83.81	77.98	80.98	57.34	48.80	71.50
Wanda	30%	83.43	68.95	83.72	77.82	80.72	57.42	48.40	71.50
QK-Wanda-M	30%	83.64	68.95	83.79	77.90	81.02	57.42	48.60	71.62
QK-Wanda (shared)	30%	83.64	68.59	83.81	77.90	81.02	57.42	48.40	71.54
QK-Wanda (sep.)	30%	83.73	68.95	83.78	77.82	81.02	57.42	48.40	71.59

Appendix

Table 21: Unmasked QK-Wanda is shown with shared and separate (sep.) budgets; dashes mark variants not evaluated in this comparison. Zero-shot accuracy (%) on Llama 2 and Llama 3.1 70B with QK-only unstructured pruning. Each model section contains its dense reference and original row-wise Wanda versus shared-budget QK-Wanda-M . Task metrics follow Appendix E.2 ; Avg. is their unweighted mean. Bold marks the highest displayed accuracy within each model and sparsity, including ties.

Method	Sparsity	BoolQ	RTE	Hella Swag	Wino Grande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Dense	0%	89.17	77.26	86.06	78.14	83.38	62.63	46.60	74.75
Wanda	30%	89.27	77.26	86.13	77.51	83.33	62.46	46.80	74.68
QK-Wanda-M	30%	89.17	77.26	86.04	77.58	83.16	62.46	46.60	74.61
QK-Wanda (shared)	30%	89.05	77.62	86.05	77.58	83.04	62.46	46.80	74.66
QK-Wanda (sep.)	30%	89.14	77.62	86.04	77.19	83.21	62.63	46.60	74.63
QK-Wanda-MRC	30%	89.05	77.26	86.08	77.66	83.29	62.12	47.00	74.64

Appendix

Table 22: Unmasked QK-Wanda is shown with shared and separate (sep.) budgets; dashes mark variants not evaluated in this comparison. Zero-shot accuracy (%) on Qwen2.5-72B with QK-only unstructured pruning. Dense is the unpruned reference; QK-Wanda-M uses shared QK budgets. We report accuracy for BoolQ, RTE, and WinoGrande, and length-normalized accuracy for HellaSwag, ARC-Easy/Challenge, and OpenBookQA. Avg. is the unweighted mean of the seven unrounded scores. Higher is better; bold marks the best displayed result at each sparsity, including ties.

Figure 7: Llama-2-13B: WikiText-2 PPL across QK sparsity.

Figure 8: Llama-2-13B: C4 PPL across QK sparsity.

Figure 9: Llama-2-7B: WikiText-2 PPL across QK sparsity.

Figure 10: Llama-2-7B: C4 PPL across QK sparsity.

Method	Sparsity	WikiText-2 PPL	C4 PPL	Avg. acc. (%)	Causal QK error (%)
Dense	0%	5.47	7.26	64.38	0.00
Wanda	30%	5.49	7.28	64.00	0.10
QK-Wanda-M (row-wise)	30%	5.49	7.28	63.95	0.10
QK-Wanda-M (separate)	30%	5.48	7.27	64.32	0.03
QK-Wanda-M (shared)	30%	5.48	7.27	64.24	0.03
QK-Wanda-MR (separate)	30%	5.48	7.27	64.15	0.04

Appendix

Table 23: Scoring and budget ablation on Llama-2-7B with QK-only unstructured pruning. QK-Wanda-M uses causal pre-RoPE scores with row-wise, separate matrix, or shared QK budgets. QK-Wanda-MR additionally includes RoPE, with separate or shared budgets. Wanda uses row-wise budgets. This specialized ablation retains the historical causal pre-RoPE error: the block-relative squared norm uses only causally allowed pairs in both numerator and denominator. It is distinct from the full-product diagnostic in Equation 49 . Bold selects the best displayed result among six methods at each sparsity, including ties.

Figure 11: TinyLlama-1.1B: WikiText-2 PPL across QK sparsity.

Figure 12: TinyLlama-1.1B: C4 PPL across QK sparsity.

Figure 13: Llama-3.2-1B: WikiText-2 PPL across QK sparsity.

Figure 14: Llama-3.2-1B: C4 PPL across QK sparsity.

Figure 15: Llama-3.2-3B: WikiText-2 PPL across QK sparsity.

Figure 16: Llama-3.2-3B: C4 PPL across QK sparsity.

Figure 17: Llama-3.1-8B: WikiText-2 PPL across QK sparsity.

Figure 18: Llama-3.1-8B: C4 PPL across QK sparsity.

Figure 19: Qwen2.5-0.5B: WikiText-2 PPL across QK sparsity.

Figure 20: Qwen2.5-0.5B: C4 PPL across QK sparsity.

Figure 21: Qwen2.5-1.5B: WikiText-2 PPL across QK sparsity.

Figure 22: Qwen2.5-1.5B: C4 PPL across QK sparsity.

Figure 23: Qwen2.5-3B: WikiText-2 PPL across QK sparsity.

Figure 24: Qwen2.5-3B: C4 PPL across QK sparsity.

Figure 25: Qwen2.5-7B: WikiText-2 PPL across QK sparsity.

Figure 26: Qwen2.5-7B: C4 PPL across QK sparsity.

Figure 27: Qwen2.5-14B: WikiText-2 PPL across QK sparsity.

Figure 28: Qwen2.5-14B: C4 PPL across QK sparsity.

Figure 29: Qwen2.5-32B: WikiText-2 PPL across QK sparsity.

Figure 30: Qwen2.5-32B: C4 PPL across QK sparsity.

Figure 31: Qwen2.5-72B: WikiText-2 PPL across QK sparsity.

Figure 32: Qwen2.5-72B: C4 PPL across QK sparsity.

Figure 33: Llama 2 70B: WikiText-2 PPL across QK sparsity.

Figure 34: Llama 2 70B: C4 PPL across QK sparsity.

Figure 35: Llama 3.1 70B: WikiText-2 PPL across QK sparsity.

Figure 36: Llama 3.1 70B: C4 PPL across QK sparsity.

Figure 37: TinyLlama 1.1B (left) and Llama 2 7B (right): QK reconstruction error under the common protocol in Equation 49 .

Figure 38: Llama 2 13B (left) and Llama 2 70B (right): QK reconstruction error under the common protocol in Equation 49 .

Figure 39: Llama 3.2 1B (left) and Llama 3.2 3B (right): QK reconstruction error under the common protocol in Equation 49 .

Figure 40: Llama 3.1 8B (left) and Llama 3.1 70B (right): QK reconstruction error under the common protocol in Equation 49 .

Figure 41: Qwen2.5 0.5B (left) and Qwen2.5 1.5B (right): QK reconstruction error under the common protocol in Equation 49 .

Figure 42: Qwen2.5 3B (left) and Qwen2.5 7B (right): QK reconstruction error under the common protocol in Equation 49 .

Figure 43: Qwen2.5 14B (left) and Qwen2.5 32B (right): QK reconstruction error under the common protocol in Equation 49 .

Figure 44: Qwen2.5 72B: QK reconstruction error under the common protocol in Equation 49 .

Calib. $N\times T$	Score	Steps	WikiText-2 PPL	C4 PPL	Time (s)
128 $\times$ 2048	Wanda	–	10.6474	15.7481	22.2
128 $\times$ 2048	QK-Wanda-M	1	10.5334	$\mathbf{15.5175}$	26.2
128 $\times$ 2048	QK-Wanda-M	10	$\mathbf{10.5332}$	15.5178	94.9
256 $\times$ 32	Wanda	–	$\mathbf{10.3054}$	15.0255	15.7
256 $\times$ 32	QK-Wanda-M	1	10.3072	$\mathbf{15.0187}$	17.2
256 $\times$ 32	QK-Wanda-M	2	10.3077	15.0193	21.9

Appendix

Table 24: Row-wise progressive pruning on Llama-3.2-1B. The final budget is 50% in every output row. Each row is one seed-0 run; bold selects the lowest unrounded PPL for each calibration setting.

Calib. $N\times T$	Score	Steps	WikiText-2 PPL	C4 PPL	Time (s)
128 $\times$ 2048	QK-Wanda-M	1	$\mathbf{11.4888}$	$\mathbf{18.2040}$	25.9
128 $\times$ 2048	QK-Wanda-M	10	12.5458	20.8593	94.9
256 $\times$ 32	QK-Wanda-M	1	$\mathbf{11.1735}$	$\mathbf{17.7755}$	19.8
256 $\times$ 32	QK-Wanda-M	2	11.2330	17.9338	22.4
256 $\times$ 32	QK-Wanda-M	5	11.4152	18.5233	37.0
256 $\times$ 32	QK-Wanda-M	10	11.6609	19.1525	66.3

Appendix

Table 25: Progressive QK-Wanda-M pruning with separate matrix budgets. Each Q and K matrix has its own 50% budget. All other settings match Table 24 ; this ablation used a separate single-seed calibration protocol. Bold marks the lowest unrounded PPL for each calibration setting.

Method	Complexity	Llama-3.2 1B	Llama-3.2 3B	Llama-3.1 8B	Llama-3.1 70B	Avg. ratio
Wanda	$O(NTd(D_{Q}+D_{K}))$	16.88 s	47.92 s	101.98 s	833.06 s	1.00
QK-Wanda	$O(NTd(D_{Q}+D_{K}))$	17.30 s	48.66 s	102.80 s	836.27 s	1.01

Appendix

Table 26: Full QK pruning computation on one A100 PCIe 40GB, at 50% QK sparsity with the main

8192\times 64=524{,}288

-token C4 calibration, batch 32. Times include calibration forwards, statistics, scoring, GPU stable sorting and mask selection, weight zeroing and pruned propagation. Loading, setup transfers, initial preparation, benchmark resets are excluded. Medians of five sums over all blocks; ratios are relative to Wanda. Complexity covers QK projections and scoring per block, excluding sorting.

Method	Complexity	Llama-3.2 1B	Llama-3.2 3B	Llama-3.1 8B	Llama-3.1 70B	Avg. ratio
Wanda	$O(NTd(D_{Q}+D_{K}))$	6.59 s	17.02 s	34.57 s	284.37 s	1.00
QK-Wanda	$O(NTd(D_{Q}+D_{K}))$	7.12 s	17.45 s	34.98 s	285.98 s	1.03

Appendix

Table 27: Full QK pruning computation on one H200, at 50% QK sparsity with the main

8192\times 64=524{,}288

GPU	Model	Wanda range	QK-Wanda range	Ratio
A100	Llama-3.2 1B	16.87–16.88	17.29–17.30	1.025
A100	Llama-3.2 3B	47.91–47.93	48.65–48.68	1.015
A100	Llama-3.1 8B	101.95–102.00	102.78–102.84	1.008
A100	Llama-3.1 70B	832.94–833.41	836.20–836.87	1.004
H200	Llama-3.2 1B	6.58–6.59	7.11–7.13	1.081
H200	Llama-3.2 3B	17.01–17.04	17.43–17.47	1.025

Appendix

Table 28: Variability of full QK pruning times with the main calibration protocol, in seconds. Each interval is the minimum–maximum of five full-block sums; ratios use their medians.

Sparsity	Wanda matrix	QK-Wanda-M separate	Wanda matrix	QK-Wanda-M separate	Wanda matrix	QK-Wanda-M separate
	WikiText-2 PPL		C4 PPL		Rel. QK error (%)
Llama-2-70B
10%	3.32	3.32	5.71	5.71	0.00	0.00
20%	3.32	3.32	5.71	5.71	0.01	0.00
30%	3.32	3.32	5.71	5.71	0.04	0.01
40%	3.33	3.32	5.72	5.71	0.18	0.03

Appendix

Table 29: Scoring at matched matrix budgets: Wanda and causal pre-RoPE QK-Wanda-M with separate QK budgets and identical per-projection deletion counts. Lower is better. Bold marks lower displayed values within each pair, including ties. Errors are block-mean relative QK reconstruction errors (%).

Llama-2-70B
Method	Sparsity	BoolQ	RTE	HellaSwag	WinoGrande	ARC- Easy	ARC- Challenge	OpenBook QA	Avg.
Dense	0%	83.76	67.87	83.81	77.98	80.98	57.34	48.80	71.50
Wanda	30%	83.43	68.95	83.72	77.82	80.72	57.42	48.40	71.50
Wanda (matrix)	30%	83.70	68.23	83.76	78.06	80.89	57.17	48.60	71.49
QK-Wanda-M	30%	83.64	68.95	83.79	77.90	81.02	57.42	48.60	71.62
Wanda	50%	82.63	69.31	83.38	77.98	80.89	57.00	48.40	71.37

Appendix

Table 30: Zero-shot accuracy (%) for the matrix-budget control and the original comparison. Wanda uses row budgets; Wanda (matrix) ranks weights within each projection; QK-Wanda-M uses shared QK budgets. Each section reuses the same dense reference and evaluation examples. Bold marks the highest displayed sparse score at each sparsity, including ties; Avg. is the unweighted seven-task mean.

				50% sparsity			80% sparsity
Method / reference	M	R	C	WikiText-2	C4	Avg.	WikiText-2	C4	Avg.
Llama-2-7B
Dense	—			5.472	7.264	64.38	5.472	7.264	64.38
Wanda	—			5.618	7.419	63.43	12.540	13.940	52.56
QK-Wanda	0	0	0	5.529	7.357	63.98	8.313	11.197	57.84
QK-Wanda-M	1	0	0	5.530	7.358	63.92	8.328	11.213	57.81

Appendix

Table 31: Eight scoring combinations on 7–8B models: causal mask (M), RoPE (R), and centering (C). QK variants share a QK budget; Wanda uses row-wise budgets. Dense (0%) is repeated as a reference. Avg. is the mean seven-task zero-shot accuracy (%). Bold marks the lowest available PPL among QK variants (displayed ties included). Attention itself is unchanged.

				50% sparsity			80% sparsity
Method / reference	M	R	C	WikiText-2	C4	Avg.	WikiText-2	C4	Avg.
Llama-2-70B
Dense	—			3.319	5.709	71.50	3.319	5.709	71.50
Wanda	—			3.359	5.742	71.37	4.550	6.906	65.85
QK-Wanda	0	0	0	3.332	5.716	71.60	3.625	5.971	71.79
QK-Wanda-M	1	0	0	3.332	5.716	71.62	3.629	5.972	71.76

Appendix

Table 32: Eight scoring combinations on 70–72B models: causal mask (M), RoPE (R), and centering (C). QK variants share a QK budget; Wanda uses row-wise budgets. Dense (0%) is repeated as a reference. Avg. is the mean seven-task zero-shot accuracy (%). Bold marks the lowest available PPL among QK variants (displayed ties included). Attention itself is unchanged.

Explore similar work

F-WANDA: Fisher-Reweighted Post-Training Pruning for Sustainable Deployment of Large Language Models
Aug 1, 2026Himanshu MishraLarge Language Model Compression Fluency
University of British Columbia
Relative Kinetic Utility: Calibrating Cross-Layer Credit for Global Structured LLM Pruning
May 9, 2026Tianhao Qian, Guilin Qi, Jiayu ChenChain-of-Thought Reasoning Inference Latency
School of Mathematics, Southeast University Nanjing 210096, China
LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry
Sep 11, 2026Sankar Behera, Dhruv Singh, Anshika Agnihotri +3Large Language Model Compression Unstructured Pruning
CSE, IIT Jammu · Mathematics, IIT Jammu · IT, EC Bikaner +1

Aug 1, 2026, cs.AIJ/K move · Enter open · S save

F-WANDA: Fisher-Reweighted Post-Training Pruning for Sustainable Deployment of Large Language Models

Himanshu Mishra

University of British Columbia

Large Language Model Compression Fluency Text Corpora

Open paper AskPDF arXiv

One-shot post-training pruning is the most energy-frugal compression strategy for largelanguage models (LLMs), yet existing approaches trade either quality (WANDA) or compute cost (SPARSEGPT). We introduce F-WANDA, a drop-in modification of WANDA that reallocates the per-row keep budget across output neurons in proportion to the empirical Fisher information of the pre-activation. The Fisher signal is collected in a single additional backward pass over the same calibration corpus WANDA already uses; no weights are updated. On LLAMA-2-7B at 50 % unstructured sparsity, F-WANDA attains WikiText-2 perplexity of 6.85, matches WANDA fluency, and improves 5-shot MMLU by +1.6 pp over WANDA and +1.1 pp over SPARSEGPT, while incurring only one-third of SPARSEGPT pruning wall-clock and energy. The headline trade-off is achieved without extra calibration data or fine-tuning, placing F-WANDA on the Pareto frontier of quality versus pruning cost for sustainable LLM compression.

Abstract

Explore similar work

F-WANDA: Fisher-Reweighted Post-Training Pruning for Sustainable Deployment of Large Language Models

Relative Kinetic Utility: Calibrating Cross-Layer Credit for Global Structured LLM Pruning

LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry