GPrune-LLM: Generalization-Aware Structured Pruning for Large Language Models

Xiaoyun Liu; Divya Saxena; Jiannong Cao; Yuqing Zhao; Yiying Dong; Penghui Ruan

cs.LGMar 12, 2026

GPrune-LLM: Generalization-Aware Structured Pruning for Large Language Models

Authors: Xiaoyun Liu, Divya Saxena, Jiannong Cao, Yuqing Zhao, Yiying Dong, Penghui Ruan

Organizations: Department of Computing, The Hong Kong Polytechnic University, Hong Kong · School of Artificial Intelligence and Data Science, IIT Jodhpur, Jodhpur 342030, India

Abstract

Structured pruning is widely applied to compress large language models (LLMs), but its performance depends heavily on how neuron importance is estimated. Most existing methods rely on activation statistics from a single calibration set, which introduces calibration bias and degrades downstream cross-task generalization. We observe that neurons exhibit heterogeneous distribution sensitivity, ranging from maintaining relatively stable rankings across calibration datasets to showing substantially larger cross-dataset variation. Ignoring this heterogeneity, existing methods rank all neurons in shared spaces with a uniform scoring source, so calibration-specific neurons dominate the ranking and weakly-activated neurons are scored unreliably. To address this, we propose GPrune-LLM, a structured pruning framework that reduces calibration bias by measuring and exploiting the cross-distribution behavior of neurons for fair comparison. Specifically, we restructure the neuron ranking space into behavior-consistent local spaces, adapt the scoring source where the calibration signal is unreliable, and learn per-module sparsity allocation under a global budget. Experiments across multiple models and downstream tasks show that GPrune-LLM improves the generalization of its base pruning metrics, with gains most pronounced at high sparsity, and reduces dependence on the choice of importance metric.

Figures & tables

Figure 1: Neuron importance changes with the calibration distribution, and neurons differ in their sensitivity to it. (a) Per-neuron importance rank drift

d

under a calibration swap, over all FFN layers. Re-sampling the same distribution (grey) barely shifts ranks (median rank correlation

0.98

), whereas switching WikiText2

\leftrightarrow

PTB (blue) causes large and heterogeneous drift (

0.62

), with some neurons drifting sharply while others remain nearly fixed. (b) Per-layer agreement (Adjusted Rand Index, ARI) between the neuron partitions produced before and after the same calibration swap. Modularizing neurons by cross-distribution behavior (GPrune) is more consistent than functional grouping estimated from a single distribution (FANG) (mean ARI

0.51

vs.

0.12

Figure 2: Overview of GPrune-LLM. Neurons are first partitioned into behavior-consistent modules by jointly modeling data-free parameter similarity and cross-distribution rank drift, with assignments refined differentiably. Each module is characterized by its mean rank drift (distribution sensitivity) and mean score (calibration-signal strength). Modules that are both distribution-sensitive and weakly supported are re-scored from the calibration distribution whose local ranking is more repeatable. Finally, module-specific sparsity is adaptively learned under a global budget with cross-distribution supervision, yielding a pruned LLM with improved generalization across diverse downstream tasks.

Ratio	Method	WT2 PPL $\downarrow$	PTB PPL $\downarrow$	BoolQ	PIQA	Hella.	Wino.	ARC-e	ARC-c	OBQA	Avg. Acc. $\uparrow$
100%	LLaMA2-7B	12.19	48.35	71.10	78.40	72.96	67.25	69.36	40.53	40.80	62.91
80%	LLM-Pruner	19.23	72.62	62.26	76.28	65.82	60.85	63.85	37.46	39.20	57.96
	Wanda-sp	17.50	67.01	46.79	76.50	68.13	65.43	65.95	39.76	38.20	57.25
	GPrune-LLM (Wanda-sp)	15.44 $\uparrow$	56.95 $\uparrow$	60.76 $\uparrow$	76.22	68.67 $\uparrow$	65.51 $\uparrow$	64.18	37.80	39.80 $\uparrow$	58.99 $\uparrow$
	FLAP	16.41	61.61	52.45	73.61	62.56	62.12	59.13	34.81	39.20	54.84
	GPrune-LLM (FLAP)	15.55 $\uparrow$	59.56 $\uparrow$	64.16 $\uparrow$	74.76 $\uparrow$	66.49 $\uparrow$	62.19 $\uparrow$	62.50 $\uparrow$	36.35 $\uparrow$	39.40 $\uparrow$	57.98 $\uparrow$

Table 1: Comparison on LLaMA2-7B. For each retention ratio, the best and second-best results are marked in bold and underlined , respectively.

\uparrow

denotes improvement over the corresponding base method at the same retention ratio. For language modeling perplexity, lower is better. For classification tasks accuracy, higher is better.

Ratio	Method	WikiText2 PPL $\downarrow$	PTB PPL $\downarrow$	Avg Acc $\uparrow$
100%	Vicuna1.5-7B	16.24	60.78	63.50
80%	LLM-Pruner	24.90	86.20	56.51
	Wanda-sp	23.66	73.67	59.08
	GPrune-LLM (Wanda-sp)	20.12 $\uparrow$	67.35 $\uparrow$	58.11
	FLAP	21.06	76.42	57.88
	GPrune-LLM (FLAP)	18.95 $\uparrow$	68.24 $\uparrow$	58.57 $\uparrow$

Table 2: Compact comparison on Vicuna1.5-7B.

Figure 3: Visualization of how GPrune-LLM changes FLAP’s pruning decisions on LLaMA2-7B at 50% retention. (a) GPrune-LLM shifts the retention budget away from the most unstable modules and toward the more stable ones. (b) GPrune-LLM concentrates its revision of which neurons to keep on the most unstable modules.

BCM	RASA	MSL	WT2 PPL $\downarrow$	PTB PPL $\downarrow$	Avg. Acc. $\uparrow$
			42.81	152.49	41.52
		✓	39.56	159.78	44.92
✓		✓	37.65	147.31	45.18
✓	✓	✓	37.02	125.41	46.40

Table 3: Component ablation of GPrune-LLM on FLAP at 50% retention ratio on LLaMA2-7B. BCM, RASA, and MSL denote Behavior-Consistent Modularization, Reliability-Aware Score Adaptation, and Modular Sparsity Learning, respectively.

Method	Calibration	WikiText2 PPL $\downarrow$	PTB PPL $\downarrow$	Avg Acc $\uparrow$
FLAP	WikiText2	42.81	152.49	41.52
GPrune-LLM (FLAP)	WikiText2+ PTB	37.02 $\uparrow$	125.41 $\uparrow$	46.40 $\uparrow$
GPrune-LLM (FLAP)	WikiText2+ C4	40.93 $\uparrow$	136.11 $\uparrow$	46.61 $\uparrow$
FLAP	PTB	63.87	119.61	40.55
GPrune-LLM (FLAP)	PTB+ WikiText2	40.56 $\uparrow$	93.78 $\uparrow$	45.83 $\uparrow$
GPrune-LLM (FLAP)	PTB+ C4	55.04 $\uparrow$	117.85 $\uparrow$	46.07 $\uparrow$

Table 4: Comparison on different primary and auxiliary calibration datasets on FLAP at 50% retention ratio on LLaMA2-7B.

Method	Scoring	WikiText2 PPL $\downarrow$	PTB PPL $\downarrow$	Avg Acc $\uparrow$
FANG	WT2	36.31	154.23	45.71
FANG	WT2+PTB mean score	43.01	115.92	42.18
FANG	WT2+PTB mean rank	50.29	144.71	42.77
FANG	WT2+PTB pool	42.88	114.20	42.04
GPrune-LLM (twFLAP)	WT2+PTB	36.83	129.77	47.22

Table 5: Effect of multi-dataset scoring at 50% retention ratio on LLaMA2-7B.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Ratio	Method	WikiText2 PPL $\downarrow$	PTB PPL $\downarrow$	BoolQ	PIQA	HellaSwag	WinoGrande	ARC-e	ARC-c	OBQA	Avg. Acc. $\uparrow$
100%	Vicuna1.5-7B	16.24	60.78	75.75	77.80	71.02	67.64	69.02	40.87	42.40	63.50
80%	LLM-Pruner	24.90	86.20	50.98	75.84	64.14	59.83	66.12	38.65	40.00	56.51
	Wanda-sp	23.66	73.67	67.31	74.16	65.21	61.80	66.96	38.31	39.80	59.08
	GPrune-LLM (Wanda-sp)	20.12 $\uparrow$	67.35 $\uparrow$	56.76	75.95 $\uparrow$	66.85 $\uparrow$	65.11 $\uparrow$	65.95	36.95	39.20	58.11
	FLAP	21.06	76.42	66.88	72.47	63.17	64.48	63.55	34.81	39.80	57.88
	GPrune-LLM (FLAP)	18.95 $\uparrow$	68.24 $\uparrow$	59.11	75.03 $\uparrow$	67.26 $\uparrow$	65.27 $\uparrow$	65.57 $\uparrow$	36.18 $\uparrow$	41.60 $\uparrow$	58.57 $\uparrow$

Appendix

Table 6: Full comparison on Vicuna1.5-7B.

Ratio	Method	WikiText2 PPL $\downarrow$	PTB PPL $\downarrow$	BoolQ	PIQA	HellaSwag	WinoGrande	ARC-e	ARC-c	OBQA	Avg. Acc. $\uparrow$
100%	LLaMA2-13B	10.98	54.42	68.99	79.05	76.60	69.77	73.32	45.56	42.00	65.04
50%	Wanda-sp	205.58	393.96	61.16	53.05	26.65	49.49	29.00	26.19	29.60	39.31
	GPrune-LLM (Wanda-sp)	28.30 $\uparrow$	126.67 $\uparrow$	62.17 $\uparrow$	68.17 $\uparrow$	54.20 $\uparrow$	55.09 $\uparrow$	51.35 $\uparrow$	33.62 $\uparrow$	39.00 $\uparrow$	51.94 $\uparrow$
	FLAP	27.91	156.91	62.66	65.02	48.95	57.85	47.43	31.48	37.00	50.06
	GPrune-LLM (FLAP)	26.54 $\uparrow$	105.20 $\uparrow$	61.59	67.52 $\uparrow$	53.71 $\uparrow$	56.75	52.15 $\uparrow$	32.85 $\uparrow$	38.20 $\uparrow$	51.82 $\uparrow$
	FANG	26.43	155.46	62.35	64.85	50.53	55.64	50.55	31.23	39.20	50.62

Appendix

Table 7: Full comparison on LLaMA2-13B. WikiText2 is abbreviated as WT2 elsewhere in the supplement. Throughout the supplement, bold and underlined values denote the best and second-best entries, respectively, and

\uparrow

indicates improvement over the corresponding base method. Lower perplexity and higher accuracy are better.

Component	Design dimension	Variant	WT2 PPL $\downarrow$	PTB PPL $\downarrow$	Avg. Acc. $\uparrow$
BCM	Modularization	Random	38.80	134.15	45.46
		Initialization	35.68	133.53	44.00
		Initialization + refinement	37.02	125.41	46.40
RASA	Module selection	None	37.65	147.31	45.18
		Random	36.59	136.71	45.00
		Rank drift	35.44	124.29	46.10

Appendix

Table 8: Fine-grained component ablations on LLaMA2-7B with FLAP at 50% retention ratio.

Method	Scoring	WT2 PPL $\downarrow$	PTB PPL $\downarrow$	Avg. Acc. $\uparrow$
Wanda-sp	WT2	262.32	505.51	39.37
Wanda-sp	Score avg.	111.15	309.30	40.37
Wanda-sp	Rank avg.	70.30	198.71	42.01
Wanda-sp	Pool	275.81	478.19	39.03
GPrune-LLM (Wanda-sp)	WT2+PTB	60.80	191.99	45.02
FLAP	WT2	42.81	152.49	41.52

Appendix

Table 9: Comparison of multi-dataset calibration at 50% retention ratio on LLaMA2-7B. Score avg. averages importance scores across WikiText2 and PTB. Rank avg. averages per-dataset ranks. Pool merges both calibration sets before scoring.

Dataset-pair comparison	ARI $\uparrow$	Matched overlap $\uparrow$
WT2–PTB vs. WT2–C4	$0.575\pm 0.121$	$0.707\pm 0.122$
WT2–PTB vs. PTB–C4	$0.582\pm 0.122$	$0.706\pm 0.124$
WT2–C4 vs. PTB–C4	$0.583\pm 0.124$	$0.714\pm 0.123$

Appendix

Table 10: Per-layer module agreement under calibration-pair substitution on LLaMA2-7B with FLAP. Values are mean

\pm

standard deviation across 32 FFN layers.

Retention	$\|\mathcal{D}\|$	Epoch	WT2 PPL $\downarrow$	PTB PPL $\downarrow$	Avg. Acc. $\uparrow$
50%	128	4	39.23	132.98	46.36
	256	7	39.56	145.52	45.86
	512	7	40.17	147.29	46.71
	1024	8	39.67	129.07	45.70
	2048	2	37.02	125.41	46.40

Appendix

Table 11: Effect of calibration sample number on GPrune-LLM (FLAP) at 50% retention ratio on LLaMA2-7B.

Method	Seed	WT2 PPL $\downarrow$	PTB PPL $\downarrow$	Avg. Acc. $\uparrow$
FANG	0	36.31	154.23	45.71
	1	36.85	155.82	44.88
	2	37.55	158.75	45.50
	Mean $\pm$ Std.	$36.90\pm 0.51$	$156.27\pm 1.87$	$45.36\pm 0.35$
GPrune-LLM (twFLAP)	0	36.83	129.77	47.22
	1	36.16	146.90	45.65

Appendix

Table 12: Seed repeatability on LLaMA2-7B at 50% retention. Mean and standard deviation are computed over seeds 0–2.

Retention Ratio	Method	Params	Tokens/s $\uparrow$
100%	LLaMA2-7B	6.74B	97.04
50%	LLM-Pruner	3.45B	161.69
	Wanda-sp	3.5B	167.44
	GPrune-LLM (Wanda-sp)	3.49B	160.38
	FLAP	3.5B	148.52
	GPrune-LLM (FLAP)	3.49B	154.59

Appendix

Table 13: Practical inference throughput on LLaMA2-7B at 50% retention. All methods use the same device and software stack; reported values aggregate 10 timed repetitions after one warm-up run.

Component	Parameter	Value
General	Calibration samples for each of $A$ and $B$	2048
	Calibration sequence length	128
	Primary calibration dataset $A$	WikiText2
	Auxiliary calibration dataset $B$	PTB
BCM	Candidate initial module counts $\mathcal{K}$	16, 24, 32, 40, 48
RASA	Rank-drift quantile level $\gamma_{\mathrm{drift}}$	0.90

Appendix

Table 14: Principal hyperparameters used in the reported experiments.

Ratio	Method	WT2 PPL $\downarrow$	PTB PPL $\downarrow$	BoolQ	PIQA	HellaSwag	WinoGrande	ARC-e	ARC-c	OBQA	Avg. Acc. $\uparrow$
100%	LLaMA2-7B	12.19	48.35	71.10	76.82	72.96	67.25	53.58	40.53	40.80	60.43
80%	LLM-Pruner	19.23	72.62	62.26	75.41	65.82	60.85	50.17	37.46	39.20	55.88
	Wanda-sp	17.50	67.01	46.79	75.14	68.13	65.43	51.64	38.82	38.20	54.88
	GPrune-LLM (Wanda-sp)	15.44 $\uparrow$	56.95 $\uparrow$	60.76 $\uparrow$	74.54	68.67 $\uparrow$	65.51 $\uparrow$	50.55	37.80	39.80 $\uparrow$	56.80 $\uparrow$
	FLAP	16.41	61.61	52.45	73.01	62.56	62.12	47.35	34.81	39.20	53.07
	GPrune-LLM (FLAP)	15.55 $\uparrow$	59.56 $\uparrow$	64.16 $\uparrow$	74.76 $\uparrow$	66.49 $\uparrow$	62.19 $\uparrow$	49.37 $\uparrow$	36.35 $\uparrow$	39.40 $\uparrow$	56.10 $\uparrow$

Appendix

Table 15: Full comparison on LLaMA2-7B under the fixed task-metric reporting protocol.

Ratio	Method	WT2 PPL $\downarrow$	PTB PPL $\downarrow$	BoolQ	PIQA	HellaSwag	WinoGrande	ARC-e	ARC-c	OBQA	Avg. Acc. $\uparrow$
100%	Vicuna1.5-7B	16.24	60.78	75.75	77.80	71.02	67.64	56.06	39.93	42.40	61.52
80%	LLM-Pruner	24.90	86.20	50.98	75.24	64.14	59.83	53.07	37.12	40.00	54.34
	Wanda-sp	23.66	73.67	67.31	73.45	65.21	61.80	53.11	38.31	39.80	57.00
	GPrune-LLM (Wanda-sp)	20.12 $\uparrow$	67.35 $\uparrow$	56.76	74.59 $\uparrow$	66.85 $\uparrow$	65.11 $\uparrow$	53.07	36.35	39.20	55.99
	FLAP	21.06	76.42	66.88	71.71	63.17	64.48	51.47	34.73	39.80	56.04
	GPrune-LLM (FLAP)	18.95 $\uparrow$	68.24 $\uparrow$	59.11	74.54 $\uparrow$	67.26 $\uparrow$	65.27 $\uparrow$	52.57 $\uparrow$	35.41 $\uparrow$	41.60 $\uparrow$	56.54 $\uparrow$

Appendix

Table 16: Full comparison on Vicuna1.5-7B under the fixed task-metric reporting protocol.

Ratio	Method	WT2 PPL $\downarrow$	PTB PPL $\downarrow$	BoolQ	PIQA	HellaSwag	WinoGrande	ARC-e	ARC-c	OBQA	Avg. Acc. $\uparrow$
100%	LLaMA2-13B	10.98	54.42	68.99	79.05	76.60	69.77	57.95	44.20	42.00	62.65
50%	Wanda-sp	205.58	393.96	61.16	50.87	26.65	49.49	28.66	26.19	29.60	38.95
	GPrune-LLM (Wanda-sp)	28.30 $\uparrow$	126.67 $\uparrow$	62.17 $\uparrow$	67.90 $\uparrow$	54.20 $\uparrow$	55.09 $\uparrow$	43.39 $\uparrow$	33.62 $\uparrow$	39.00 $\uparrow$	50.77 $\uparrow$
	FLAP	27.91	156.91	62.66	65.02	48.95	57.85	43.22	31.48	37.00	49.46
	GPrune-LLM (FLAP)	26.54 $\uparrow$	105.20 $\uparrow$	61.59	67.52 $\uparrow$	53.71 $\uparrow$	56.75	44.32 $\uparrow$	32.85 $\uparrow$	38.20 $\uparrow$	50.71 $\uparrow$
	FANG	26.43	155.46	62.35	64.85	50.53	55.64	43.60	31.23	39.20	49.63

Appendix

Table 17: Full comparison on LLaMA2-13B under the fixed task-metric reporting protocol.

LLM Pruning

Large language model pruning removes less useful weights or components to reduce memory and inference costs while preserving model capabilities.

See all 178 papers

LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry
Sep 10, 2026Sankar Behera, Dhruv Singh, Anshika Agnihotri +3LLM Pruning LLM Compression
CSE, IIT Jammu · Mathematics, IIT Jammu · IT, EC Bikaner +1
Diet Your LLM: Dimension-wise Global Pruning of LLMs via Merging Task-specific Importance Score
Mar 25, 2026Jimyung Hong, Jaehyung KimLLM Pruning Structured Pruning
Yonsei University
Correlation-Aware Structured Pruning for Large Language Models
Aug 23, 2026Sicheng Xu, Hao Shi, Wei Zhang +6LLM Pruning Neural Network Pruning
Department of Mathematical Sciences, Tsinghua University · Theory Lab, Central Research Institute, 2012 Labs, Huawei Tech. Co. Ltd.

Sep 10, 2026J/K move · Enter open · S save

LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry

Sankar Behera, Dhruv Singh, Anshika Agnihotri, Raj Kumar Choudhary+2

CSE, IIT Jammu · Mathematics, IIT Jammu · IT, EC Bikaner+1

LLM Pruning LLM Compression Structured Pruning Neural Network Pruning

Open paper AskPDF arXiv

Structured pruning of large language models (LLMs) offers hardware-efficient compression, yet existing methods require calibration data, gradient computation, or large auxiliary policy networks at pruning time. LILA (Latent-Informed Layer Analysis) scores neuron importance via the Kolmogorov--Smirnov (KS) distance between empirical singular value distributions of the full and neuron-ablated feed-forward network (FFN) weight matrix, providing a closed-form spectral rule requiring no training, calibration data, or auxiliary network. Without any fine-tuning, LILA surpasses PruneNet (45M-parameter RL policy) by 1.57~~pp in zero-shot accuracy on LLaMA-2-7B at 25% sparsity, and outperforms WikiText-2-calibrated SliceGPT by up to 6.0~~pp across all sparsity levels, while preserving the original architecture. After one epoch of LoRA recovery fine-tuning, LILA achieves highly competitive performance, matching the heavily calibrated SliceGPT baseline to within a 0.48~pp margin across LLaMA-2-7B and Phi-2, despite using zero calibration data. A Neural Tangent Kernel analysis confirms a 22

\times

reduction in functional distortion versus random pruning, providing theoretical grounding for the spectral importance criterion. Finally, extending LILA to dynamically allocate sparsity budgets via KS-scores yields state-of-the-art generative preservation at moderate compression, while uncovering fundamental single-layer architectural bottlenecks at higher compression regimes.

Abstract

LLM Pruning

Explore similar work

LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry

Diet Your LLM: Dimension-wise Global Pruning of LLMs via Merging Task-specific Importance Score

Correlation-Aware Structured Pruning for Large Language Models