Diet Your LLM: Dimension-wise Global Pruning of LLMs via Merging Task-specific Importance Score

Jimyung Hong; Jaehyung Kim

cs.LGMar 25, 2026

Diet Your LLM: Dimension-wise Global Pruning of LLMs via Merging Task-specific Importance Score

Authors: Jimyung Hong, Jaehyung Kim

Organizations: Yonsei University

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities, but their massive scale poses significant challenges for practical deployment. Structured pruning offers a promising solution by removing entire dimensions or layers, yet existing methods face critical trade-offs: task-agnostic approaches cannot adapt to task-specific requirements, while task-aware methods require costly training to learn task adaptability. We propose DIET (Dimension-wise global pruning of LLMs via merging Task-wise importance scores), a training-free structured pruning method that combines dimension-level granularity with task-aware selection. DIET profiles activation magnitudes across tasks using only 100 samples per task, then applies majority voting to construct a single global mask. Experiments on seven zero-shot benchmarks using Gemma-2 2B and 9B models demonstrate the effectiveness of DIET. At 20% sparsity on Gemma-2 2B, DIET improves average accuracy by 2.1 percentage points over SliceGPT and by 9.2 percentage points over PuDDing. On Gemma-2 9B, DIET remains competitive with SliceGPT while consistently outperforming PuDDing at 10% and 20% sparsity.

Figures & tables

Figure 1: Step 1. Task-specific samples are used to profile a dense LLM and compute task-wise activation vectors. Step 2. Low-importance dimensions vote to prune, and the votes are merged by majority voting into a global mask

m\in\{0,1\}^{d}

. Step 3. The mask is applied across layers, producing a pruned LLM with reduced hidden width

d^{\prime}

Method	Sparsity	Per-task Accuracies (%)							Average Acc. (%)
Method	Sparsity	BoolQ	RTE	HellaSwag*	WinoGrande	Arc-E*	Arc-C*	OBQA*	Average Acc. (%)
No pruning (Original)	0	73.5	61.4	73.0	68.4	80.2	49.7	41.8	64.0
Magnitude-Dim	20%	38.2	55.2	31.1	50.0	39.8	21.3	25.0	37.2
SliceGPT	20%	38.4	52.7	43.5	60.9	47.0	27.3	30.6	42.9
PuDDing	20%	43.4	53.8	25.8	49.8	27.3	23.6	27.0	35.8
DieT (Ours)	20%	63.3	54.9	41.8	53.2	49.8	24.1	27.6	45.0

Table 1: Zero-shot accuracy comparisons of DieT against baseline pruning algorithms on Gemma-2 2B on seven NLP benchmarks. The best performances are marked in bold , and the runner-up is marked with underline . Tasks with * are presented with acc_norm and remaining tasks with acc .

Method	Sparsity	Per-task Accuracies (%)							Average Acc. (%)
Method	Sparsity	BoolQ	RTE	HellaSwag*	WinoGrande	Arc-E*	Arc-C*	OBQA*	Average Acc. (%)
No pruning (Original)	0	84.4	67.9	80.0	74.2	87.8	66.1	47.2	72.5
Magnitude-Dim	20%	68.5	53.8	49.5	60.6	62.0	30.8	31.6	51.0
SliceGPT	20%	49.2	54.2	54.5	68.3	62.4	36.1	39.4	52.0
PuDDing	20%	62.8	57.4	49.8	59.7	51.3	32.1	29.2	48.9
DIET (Ours)	20%	74.7	54.2	48.5	59.2	61.7	32.0	31.2	51.6

Table 2: Zero-shot accuracy comparisons of DieT against baseline pruning algorithms on Gemma-2 9B on seven NLP benchmarks. The best performances are marked in bold , and the runner-up is marked with underline . Tasks with * are presented with acc_norm and remaining tasks with acc .

Model	Avg. Task-wise Peak Memory (MB)	Memory Reduction	Estimated FLOPs per Batch ( $10^{12}$ )	FLOPs Reduction
Original	7244.7	—	4.62	—
DieT (Zero-mask)	7244.7	$0.0\%$	4.62	$0.0\%$
DieT -Hard	6247.2	$13.8\%$	3.70	$20.0\%$

Table 3: Memory and computational efficiency comparison of the original, zero-masked, and hard-pruned DieT models on Gemma-2 2B at 20% sparsity.

Method	Sparsity	Per-task Accuracies (%)							Average Acc. (%)
Method	Sparsity	BoolQ	RTE	HellaSwag*	WinoGrande	Arc-E*	Arc-C*	OBQA*	Average Acc. (%)
No pruning (Original)	0	73.5	61.4	73.0	68.4	80.2	49.7	41.8	64.0
DieT (Zero-mask)	20%	63.3	54.9	41.8	53.2	49.8	24.1	27.6	45.0
DieT (Hard-prune)	20%	64.7	51.6	42.5	54.7	51.3	25.9	27.4	45.4

Table 4: Zero-shot accuracy comparisons of the standard zero-masking DieT and the hard-pruning implementation ( DieT -Hard) on Gemma-2 2B. DieT -Hard physically removes dimensions and applies variance rescaling to mitigate distribution shift. The better performance between the two pruned methods is marked in bold . Tasks with * are presented with acc_norm and remaining tasks with acc .

Figure 2: Consensus-vote histogram by dimension. Numbers show total counts (and share of

d{=}2304

Figure 3: Zero-shot accuracy by activation-profiling sample size. Benchmarks with * use acc_norm .

Method	Sparsity	Qwen2.5-7B	Phi-4-mini	Qwen3-14B-Base
No pruning	0%	70.5	58.8	72.7
Magnitude - Dim	10%	38.6	36.1	34.7
DieT (Ours)	10%	60.9	46.3	66.7
DieT (Ours)	20%	37.0	37.2	49.0

Table 5: Zero-shot average accuracy (%) across seven benchmarks on Qwen2.5-7B, Phi-4-Mini-Reasoning, and Qwen3-14B-Base at 10% and 20% sparsity.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Content
Question	do iran and afghanistan speak the same language
Context	Persian language – Persian, also known by its endonym Farsi, is one of the Western Iranian languages… It is primarily spoken in Iran, Afghanistan (officially known as Dari since 1958), and Tajikistan…
Ground Truth	Yes

Appendix

Table 6: Example data from BoolQ .

Component	Content
Premise	No Weapons of Mass Destruction Found in Iraq Yet.
Hypothesis	Weapons of Mass Destruction Found in Iraq.
Ground Truth	not entailment

Appendix

Table 7: Example data from RTE .

Component	Content
Context	Then, the man writes over the snow covering the window of a car, and a woman wearing winter clothes smiles. then
Options	A. , the man adds wax to the windshield and cuts it. B. , a person board a ski lift, while two men supporting the head of the person wearing winter clothes snow as the we girls sled. C. , the man puts on a christmas coat, knitted with netting. D. , the man continues removing the snow on his car.
Ground Truth	D

Appendix

Table 8: Example data from HellaSwag .

Component	Content
Sentence	Ian volunteered to eat Dennis’s menudo after already having a bowl because ___ despised eating intestine.
Options	1. Ian 2. Dennis
Ground Truth	2 ( Dennis )

Appendix

Table 9: Example data from WinoGrande .

Component	Content
Question	Which factor will most likely cause a person to develop a fever?
Options	A. a leg muscle relaxing after exercise B. a bacterial population in the bloodstream C. several viral particles on the skin D. carbohydrates being digested in the stomach
Ground Truth	B

Appendix

Table 10: Example data from ARC-Easy .

Component	Content
Question	George wants to warm his hands quickly by rubbing them. Which skin surface will produce the most heat?
Options	A. dry palms B. wet palms C. palms covered with oil D. palms covered with lotion
Ground Truth	A

Appendix

Table 11: Example data from ARC-Challenge .

Component	Content
Question	The sun is responsible for
Options	A. puppies learning new tricks B. children growing up and getting old C. flowers wilting in a vase D. plants sprouting, blooming and wilting
Ground Truth	D

Appendix

Table 12: Example data from OpenBookQA .

Component	Content
Premise	Yes, I sang for him and beautiful spirituals he liked.
Hypothesis	I refused to sing for him.
Options	A. Entailment B. Contradiction C. Neutral
Ground Truth	B

Appendix

Table 13: Example data from XNLI .

Component	Content
Prompt	Write a short essay about climate change. Your response must have exactly 3 paragraphs. Do not use the word "global" anywhere in the text.
Constraints	1. Number of paragraphs must be exactly 3. 2. Negative constraint: Forbidden word "global".
Ground Truth	Verified automatically via rule-based software program execution.

Appendix

Table 14: Example data from IFEval .

Component	Content
Subject	high_school_biology
Question	In a population of giraffes, an environmental change occurs that favors individuals that are tallest. As a result, more of the taller individuals are able to obtain nutrients and survive to pass along their genetic information. This is an example of
Options	A. directional selection. B. stabilizing selection. C. sexual selection. D. disruptive selection.
Ground Truth	A

Appendix

Table 15: Example data from MMLU .

Stage	Basic Prompt / Source
Activation Collection (Pruning)	No task instruction is added. Inputs are constructed from dataset fields with lightweight field delimiters where applicable.
Final Evaluation (lm-eval)	lm-eval-harness default prompts with zero-shot ( --num_fewshot 0 ). No custom instruction added.

Appendix

Table 16: Prompt policy used in our pipeline. For pruning analysis we do not use explicit templates; for final evaluation we rely on lm-eval-harness defaults.

Dataset	Split	Input Format for Activation Collection
BoolQ	train	{passage} Question: {question} Answer:
RTE	train	{premise} Question: {hypothesis} True or False?
HellaSwag	train	{context} {choice}
WinoGrande	train	{sentence}
ARC-Easy	train	Question: {question} Answer: {choice}
ARC-Challenge	train	Question: {question} Answer: {choice}

Appendix

Table 17: Input formatting templates used for aggregating activation statistics across different benchmarks.

Benchmark	Basic Prompt
BoolQ	Answer the question with Yes or No based on the context. Q: {question} Context: {passage} Final format: Answer: [Yes or No]
RTE	Determine whether the hypothesis is entailed by the premise. Premise: {premise} Hypothesis: {hypothesis} Final format: Answer: [entailment or not entailment]
HellaSwag	Choose the most plausible ending. Context: {ctx} Choices: (A) {A} (B) {B} (C) {C} (D) {D} Final format: Answer: [A/B/C/D]
WinoGrande	Fill the blank in the sentence with one candidate. Sentence: {sentence with ___ blank} Candidates: {option1}, {option2} Final format: Answer: [1 or 2]
ARC-Easy / ARC-Challenge	Select one correct option. Question: {question} Choices: (A) {A} (B) {B} (C) {C} (D) {D} Final format: Answer: [A/B/C/D]
OpenBookQA	Use basic science knowledge to answer. Question: {question_stem} Choices: (A) {A} (B) {B} (C) {C} (D) {D} Final format: Answer: [A/B/C/D]

Appendix

Table 18: Basic prompts used for our evaluation (lm-eval-harness defaults).

Method	Sparsity	Per-task Accuracies (%)							Average Acc. (%)
Method	Sparsity	BoolQ	RTE	HellaSwag*	WinoGrande	Arc-E*	Arc-C*	OBQA*	Average Acc. (%)
No pruning (Original)	0	73.5	61.4	73.0	68.4	80.2	49.7	41.8	64.0
Magnitude - Dim	30%	37.8	50.5	28.7	49.3	34.5	20.7	25.6	35.3
SliceGPT	30%	37.8	52.7	32.9	57.0	36.0	24.0	27.2	38.2
PuDDing	30%	39.6	46.2	26.3	49.6	28.6	24.5	27.2	34.6
DIET (Ours)	30%	41.3	49.1	28.8	49.3	35.9	21.5	24.2	35.7

Appendix

Table 19: Zero-shot accuracy comparisons of DieT against baseline pruning algorithms on Gemma-2 2B on seven NLP benchmarks. The best performances are marked in bold , and the runner-up is marked with underline . Tasks with * are presented with acc_norm and remaining tasks with acc .

Method	Sparsity	Per-task Accuracies (%)							Average Acc. (%)
Method	Sparsity	BoolQ	RTE	HellaSwag*	WinoGrande	Arc-E*	Arc-C*	OBQA*	Average Acc. (%)
No pruning (Original)	0	84.4	67.9	80.0	74.2	87.8	66.1	47.2	72.5
Magnitude - Dim	30%	53.9	52.7	32.6	51.9	44.1	22.6	25.0	40.4
SliceGPT	30%	37.9	53.8	40.8	63.2	46.2	27.2	31.6	43.0
PuDDing	30%	50.5	66.1	49.2	60.0	52.1	32.2	31.6	48.8
DIET (Ours)	30%	61.2	53.1	32.3	52.9	43.1	22.5	24.2	41.3

Appendix

Table 20: Zero-shot accuracy comparisons of DieT against baseline pruning algorithms on Gemma-2 9B on seven NLP benchmarks. The best performances are marked in bold , and the runner-up is marked with underline . Tasks with * are presented with acc_norm and remaining tasks with acc . All values are reported as percentages (%).

Method	Sparsity	Per-language Accuracies (%)															Average Acc. (%)
Method	Sparsity	en	fr	es	de	el	bg	ru	tr	ar	vi	th	zh	hi	sw	ur	Average Acc. (%)
No pruning (Original)	0	53.9	49.3	46.8	49.0	39.1	44.1	49.0	45.0	33.2	47.0	44.4	35.5	43.5	41.0	34.1	43.7
Magnitude-Dim	10%	37.1	37.2	34.5	34.3	34.2	34.7	36.4	33.6	34.1	33.5	33.5	34.0	34.8	34.1	34.3	34.7
DIET (Ours)	10%	49.8	41.9	45.3	42.6	35.6	39.2	38.7	34.2	33.5	34.3	37.6	32.8	37.7	34.2	33.3	38.0
Magnitude-Dim	20%	35.0	34.9	33.7	33.4	33.6	33.7	36.4	32.9	33.6	33.1	33.5	33.3	36.1	33.6	33.4	34.0
DIET (Ours)	20%	40.0	34.1	36.6	33.7	33.4	34.1	33.5	33.4	33.2	33.4	33.6	33.3	35.1	34.6	33.3	34.4

Appendix

Table 21: Zero-shot accuracy comparisons of DieT against baseline pruning algorithms on Gemma-2 2B on XNLI (15 languages). The best average performances at each sparsity level are marked in bold . All values are reported as percentages (%).

Method	Sparsity	Per-language Accuracies (%)															Average Acc. (%)
Method	Sparsity	en	fr	es	de	el	bg	ru	tr	ar	vi	th	zh	hi	sw	ur	Average Acc. (%)
No pruning (Original)	0	54.0	51.6	51.0	51.1	45.3	48.6	49.0	50.4	33.8	45.2	47.5	33.4	46.6	46.5	44.4	46.6
Magnitude-Dim	10%	52.3	44.0	49.5	49.0	42.7	38.6	41.5	42.9	33.6	47.4	38.8	33.3	40.2	40.7	34.1	41.9
DIET (Ours)	10%	52.6	50.2	47.0	48.7	41.6	37.4	47.3	44.7	34.0	42.9	40.8	33.9	44.5	38.8	38.1	42.8
Magnitude-Dim	20%	45.0	35.2	43.5	42.3	38.8	35.1	35.6	35.3	33.7	39.9	35.9	33.1	39.2	37.1	33.4	37.5
DIET (Ours)	20%	43.7	43.3	37.1	43.5	38.6	34.9	37.2	38.8	33.5	36.1	34.5	33.7	37.6	34.5	34.1	37.4

Appendix

Table 22: Zero-shot accuracy comparisons of DieT against baseline pruning algorithms on Gemma-2 9B on XNLI (15 languages). The best average performances at each sparsity level are marked in bold . All values are reported as percentages (%).

Method	Sparsity	Gemma-2 2B				Gemma-2 9B
		Strict Acc		Loose Acc		Strict Acc		Loose Acc
		Prompt	Inst	Prompt	Inst	Prompt	Inst	Prompt	Inst
Original	0	0.1368	0.2710	0.1386	0.2758	0.1497	0.2698	0.1590	0.2770
Magnitude-Dim	10%	0.1331	0.2698	0.1405	0.2758	0.1497	0.2614	0.1608	0.2710
DieT (Ours)	10%	0.1035	0.2158	0.1183	0.2278	0.1553	0.2842	0.1590	0.2902
Magnitude-Dim	20%	0.1054	0.2302	0.1054	0.2302	0.1165	0.2398	0.1220	0.2434

Appendix

Table 23: Zero-shot accuracy comparisons on IFEval for Gemma-2 2B and 9B. All values are reported as accuracy scores.

Category	Magnitude-Dim	DieT (Ours)	$\Delta$ (pp)
Humanities	23.91	28.48	$+4.57$
Other	27.71	32.57	$+4.86$
Social Sciences	26.00	31.78	$+5.78$
STEM	26.29	31.81	$+5.52$
Overall	25.74	30.86	$+5.12$

Appendix

Table 24: Category-wise zero-shot MMLU accuracy (%) on Gemma-2 2B at 20% structured pruning.

\Delta

denotes the absolute improvement of DieT over Magnitude-Dim in percentage points.

Method	Sparsity	Per-task Accuracies (%)							Average Acc. (%)
Method	Sparsity	BoolQ	RTE	HellaSwag*	WinoGrande	Arc-E*	Arc-C*	OBQA*	Average Acc. (%)
Arc-Easy Mask	20%	62.9	52.5	39.0	53.1	48.6	25.2	26.8	44.0
Arc-Challenge Mask	20%	63.4	52.4	39.0	51.5	48.0	24.5	26.8	43.6
All Tasks (Ours)	20%	63.3	54.9	41.8	53.2	49.8	24.1	27.6	45.0

Appendix

Table 25: Zero-shot accuracy on Gemma-2 2B with masks derived from arc_easy , arc_challenge , and all tasks. Tasks with * are presented with acc_norm and remaining tasks with acc .

Method	Sparsity	Per-task Accuracies (%)							Average Acc. (%)
Method	Sparsity	BoolQ	RTE	HellaSwag*	WinoGrande	Arc-E*	Arc-C*	OBQA*	Average Acc. (%)
Continuous	20%	63.3	52.7	42.1	52.6	50.4	24.5	26.8	44.6
Voting (Ours)	20%	63.3	54.9	41.8	53.2	49.8	24.1	27.6	45.0

Appendix

Table 26: Zero-shot accuracy comparisons of continuous score merging and our discrete (vote-based) mask on Gemma-2 2B at 20% sparsity. Tasks with * are presented with acc_norm and remaining tasks with acc .

LLM Pruning

Large language model pruning removes less useful weights or components to reduce memory and inference costs while preserving model capabilities.

See all 168 papers

LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry
Sep 10, 2026Sankar Behera, Dhruv Singh, Anshika Agnihotri +3LLM Pruning LLM Compression
CSE, IIT Jammu · Mathematics, IIT Jammu · IT, EC Bikaner +1
Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models
Nov 8, 2023Rocktim Jyoti Das, Mukul Ranjan, Mingjie Sun +2LVLM Pruning LLM Pruning
Mohamed bin Zayed University of AI · Carnegie Mellon University
GRASPrune: Global Gating for Budgeted Structured Pruning of Large Language Models
Apr 21, 2026Ziyang Wang, Jiangfeng Xiao, Chuan Xiao +3LLM Pruning LLM Compression
Beijing Institute of Technology, Zhuhai · Shenzhen University · Osaka University

Sep 10, 2026J/K move · Enter open · S save

LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry

Sankar Behera, Dhruv Singh, Anshika Agnihotri, Raj Kumar Choudhary+2

CSE, IIT Jammu · Mathematics, IIT Jammu · IT, EC Bikaner+1

LLM Pruning LLM Compression Structured Pruning Neural Network Pruning

Open paper AskPDF arXiv

Structured pruning of large language models (LLMs) offers hardware-efficient compression, yet existing methods require calibration data, gradient computation, or large auxiliary policy networks at pruning time. LILA (Latent-Informed Layer Analysis) scores neuron importance via the Kolmogorov--Smirnov (KS) distance between empirical singular value distributions of the full and neuron-ablated feed-forward network (FFN) weight matrix, providing a closed-form spectral rule requiring no training, calibration data, or auxiliary network. Without any fine-tuning, LILA surpasses PruneNet (45M-parameter RL policy) by 1.57~~pp in zero-shot accuracy on LLaMA-2-7B at 25% sparsity, and outperforms WikiText-2-calibrated SliceGPT by up to 6.0~~pp across all sparsity levels, while preserving the original architecture. After one epoch of LoRA recovery fine-tuning, LILA achieves highly competitive performance, matching the heavily calibrated SliceGPT baseline to within a 0.48~pp margin across LLaMA-2-7B and Phi-2, despite using zero calibration data. A Neural Tangent Kernel analysis confirms a 22

\times

reduction in functional distortion versus random pruning, providing theoretical grounding for the spectral importance criterion. Finally, extending LILA to dynamically allocate sparsity budgets via KS-scores yields state-of-the-art generative preservation at moderate compression, while uncovering fundamental single-layer architectural bottlenecks at higher compression regimes.

Abstract

LLM Pruning

Explore similar work

LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry

Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models

GRASPrune: Global Gating for Budgeted Structured Pruning of Large Language Models