OPTIMA: Optimal One-shot Pruning for LLMs via Quadratic Programming Reconstruction

Mohammad Mozaffari; Samuel Kushnir; Maryam Mehri Dehnavi; Amir Yazdanbakhsh

cs.LGDec 15, 2025

OPTIMA: Optimal One-shot Pruning for LLMs via Quadratic Programming Reconstruction

Authors: Mohammad Mozaffari, Samuel Kushnir, Maryam Mehri Dehnavi, Amir Yazdanbakhsh

Organizations: University of Toronto · Google DeepMind

Abstract

Post-training pruning cuts the inference cost of large language models, yet it faces a trade-off: simple heuristics that zero weights are fast but degrade accuracy, while principled joint optimization methods recover accuracy but are computationally infeasible at modern scale. One-shot methods such as SparseGPT sit in between and apply efficient but approximate weight updates. To close this gap, we introduce OPTIMA, a practical one-shot post-training pruning method that balances accuracy and scalability. OPTIMA casts layer-wise weight reconstruction after mask selection as independent, column-wise quadratic programs (QPs) that share one Hessian per weight matrix. Solving these QPs yields, at solver convergence, the per-column minimizer of the layer-wise reconstruction objective under the shared Hessian, and the shared Hessian makes the QPs easy to batch on accelerators. We implement an accelerator-friendly QP solver that accumulates one Hessian per weight matrix and solves many small QPs in parallel with a convergence certificate per column, enabling one-shot post-training pruning at scale on a single accelerator without fine-tuning. OPTIMA integrates with existing mask selectors and improves average zero-shot accuracy in 40 of 49 (model, mask selector, sparsity) settings across the LLaMA and Gemma families, with mean gains of +0.61, +0.56, and +1.01 points at 50% unstructured, 2:4, and 60% unstructured sparsity, and up to +3.97 points at 60% sparsity, while matching ADMM-based exact reconstruction on its own masks. On an NVIDIA H100, OPTIMA prunes an 8B-parameter transformer end-to-end in 40 hours and 60 GB of peak GPU memory. This shows that principled column-wise QP reconstruction is a practical alternative to heuristic weight updates in one-shot post-training pruning.

Figures & tables

Figure 1: Overview of OPTIMA . One Hessian $\mathbf{H}$ per weight matrix is estimated from the calibration data; after the mask selector fixes pruned entries, each column’s remaining weights are updated by a small QP with $\mathbf{H}$ , and the identically structured QPs are solved in parallel on an accelerator.

		$\displaystyle\min_{{\color[rgb]{0.5,0,0.5}\Delta\mathbf{w}_{j}}}{\color[rgb]{0.5,0,0.5}\Delta\mathbf{w}_{j}^{T}}{\color[rgb]{0.5,0,0.5}\mathbf{H}_{\mathbf{W}}}{\color[rgb]{0.5,0,0.5}\Delta\mathbf{w}_{j}}$
		$\displaystyle\text{s.t. }({\color[rgb]{0.5,0,0.5}\Delta\mathbf{w}_{j}})_{i}=-({\color[rgb]{0.5,0,0.5}\mathbf{w}_{j}})_{i},\;\forall i\in{\color[rgb]{0.5,0,0.5}\mathcal{S}_{j}}$

Algorithm 1 Layer-wise pruning with batched column-wise quadratic programming

Figure 2: Ratio of OPTIMA ’s layer-wise output error to that of the Wanda, SparseGPT, and Thanos updates on LLaMA 3.2 1B; values below 1 mean OPTIMA lowers the error, and values near 1 mark weight matrices where the fallback kept the selector’s update.

Model	Mask Selection	Weight Update	Perplexity	Metrics (%)
Model	Mask Selection	Weight Update	Perplexity	MMLU	PIQA	Arc-E	Arc-C	Wino	OpenQA	Avg.
LLaMA 3.1 8B	Dense	–	5.84	63.57	80.09	81.44	51.37	73.48	33.40	63.89
	Wanda	–	9.64	47.79	75.68	72.56	40.70	70.09	27.40	55.70
	Wanda	OPTIMA	9.37	48.85	76.71	73.82	42.32	70.32	28.20	56.70
	SparseGPT	SparseGPT	9.30	51.32	76.17	73.02	41.30	70.88	29.40	57.02
	SparseGPT	OPTIMA	9.33	49.31	76.61	74.28	42.83	70.88	28.20	57.02

Table 1: WikiText2 perplexity and zero-shot accuracy at 50% unstructured sparsity. OPTIMA improves the average accuracy in 12 of 15 (model, mask selector) pairs and ties one. Within each block, the better of the two Weight Update rows is bolded.

Model	Mask Selection	Weight Update	Perplexity	Metrics (%)
Model	Mask Selection	Weight Update	Perplexity	MMLU	PIQA	Arc-E	Arc-C	Wino	OpenQA	Avg.
LLaMA 3.1 8B	Dense	–	5.84	63.57	80.09	81.44	51.37	73.48	33.40	63.89
	Wanda	–	13.54	43.42	73.18	69.23	35.32	67.32	25.80	52.38
	Wanda	OPTIMA	12.58	45.45	73.39	69.57	36.18	68.90	25.20	53.12
	SparseGPT	SparseGPT	12.37	45.62	73.83	69.15	35.84	69.22	25.60	53.21
	SparseGPT	OPTIMA	12.54	46.04	73.72	69.95	36.77	69.61	27.00	53.85

Table 2: WikiText2 perplexity and zero-shot accuracy at 2:4 sparsity applied to the MLP layers only (38% to 41% end-to-end sparsity). OPTIMA improves the average accuracy in 17 of 19 (model, mask selector) pairs. Within each block, the better of the two Weight Update rows is bolded.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Calibration Samples	128
Tokens per Sample	2048
Dataset for Calibration	C4
Relative Tolerance (rAPDHG)	0.01
Absolute Tolerance (rAPDHG)	0.01
Maximum Iterations (rAPDHG)	100,000

Appendix

Table 3: Key hyperparameters of OPTIMA and of the ADAM baseline of subsection G.1 (last two rows).

Sparsity	Model	Weight Update	Perplexity	Metrics (%)
Sparsity	Model	Weight Update	Perplexity	MMLU	PIQA	Arc-E	Arc-C	Wino	OpenQA	Avg.
50% unstructured	LLaMA 3.2 1B	–	25.35	25.97	65.40	50.72	22.87	54.46	21.20	40.10
		ADMM	17.42	27.00	68.88	55.51	27.13	56.27	23.00	42.97
		OPTIMA	17.42	26.93	69.59	55.68	27.13	56.35	22.40	43.01
	LLaMA 3.2 3B	–	14.03	38.37	72.74	65.24	34.13	63.06	25.40	49.82
		ADMM	11.58	43.19	73.94	67.42	34.47	66.69	28.60	52.38

Appendix

Table 4: Matched-protocol comparison with ADMM-based reconstruction ( Boža, 2024 ) . ADMM selects the mask; on that same mask and the same calibration data, the rows compare no weight update (–), ADMM’s own update, and OPTIMA . Within each block, the better perplexity and average accuracy of the two updates are bolded.

Model	Mask Selection	Weight Update	Perplexity	Metrics (%)
Model	Mask Selection	Weight Update	Perplexity	MMLU	PIQA	Arc-E	Arc-C	Wino	OpenQA	Avg.
LLaMA 3.1 8B	Dense	–	5.84	63.57	80.09	81.44	51.37	73.48	33.40	63.89
	Wanda	–	21.65	31.98	69.53	61.11	27.30	61.09	21.40	45.40
	Wanda	OPTIMA	17.56	33.96	71.60	63.76	29.35	66.06	22.60	47.89
	SparseGPT	SparseGPT	15.44	35.32	71.55	62.88	31.66	68.19	24.20	48.96
	SparseGPT	OPTIMA	15.64	32.44	71.87	63.97	33.11	67.56	24.60	48.93

Appendix

Table 5: Model perplexity on WikiText2 and accuracy on zero-shot downstream tasks for 60% unstructured sparsity. OPTIMA improves the average accuracy in 11 of 15 (model, mask selector) pairs. Within each (Model, Mask Selection) block, the better of the two Weight Update rows is bolded.

Model	Mask Selection	Weight Update	Perplexity	Metrics (%)
Model	Mask Selection	Weight Update	Perplexity	MMLU	PIQA	Arc-E	Arc-C	Wino	OpenQA	Avg.
Qwen 2.5 0.5B	Dense	–	13.08	47.36	69.97	64.18	29.18	55.80	24.40	48.48
	Wanda	–	24.00	30.52	64.09	57.41	24.06	54.38	19.80	41.71
	Wanda	OPTIMA	22.70	26.14	64.58	57.79	25.26	56.04	22.00	41.97
	SparseGPT	SparseGPT	20.33	29.38	64.74	56.52	24.15	56.20	20.60	41.93
	SparseGPT	OPTIMA	19.54	27.68	65.13	56.99	24.66	55.33	20.60	41.73

Appendix

Table 6: Qwen-2.5 family perplexity on WikiText2 and accuracy on zero-shot downstream tasks for 50% unstructured sparsity. OPTIMA improves the average accuracy in 11 of 15 (model size, mask selector) pairs. Within each (Model, Mask Selection) block, the better of the two Weight Update rows is bolded.

Model	Mask Selection	Weight Update	Perplexity	Metrics (%)
Model	Mask Selection	Weight Update	Perplexity	MMLU	PIQA	Arc-E	Arc-C	Wino	OpenQA	Average
Qwen 2.5 0.5B	Dense	–	13.08	47.36	69.97	64.18	29.18	55.80	24.40	48.48
	Wanda	–	83.42	23.02	59.96	43.81	18.09	50.28	12.80	34.66
	Wanda	OPTIMA	51.97	23.16	60.72	46.25	20.14	51.78	16.40	36.41
	SparseGPT	SparseGPT	40.56	22.90	61.59	48.40	21.25	52.80	16.80	37.29
	SparseGPT	OPTIMA	36.77	23.06	62.13	48.74	21.33	53.99	17.40	37.77

Appendix

Table 7: Qwen-2.5 perplexity on WikiText2 and accuracy on zero-shot downstream tasks for 60% unstructured sparsity. OPTIMA improves the average accuracy in 11 of 15 (model size, mask selector) pairs. Within each (Model, Mask Selection) block, the better of the two Weight Update rows is bolded.

Model	Mask Selection	Weight Update	Perplexity	Metrics (%)
Model	Mask Selection	Weight Update	Perplexity	MMLU	PIQA	Arc-E	Arc-C	Wino	OpenQA	Avg.
Qwen 2.5 0.5B	Dense	–	13.08	47.36	69.97	64.18	29.18	55.80	24.40	48.48
	Wanda	–	41.30	27.88	61.75	48.99	23.38	52.72	14.00	38.12
	Wanda	OPTIMA	27.61	25.57	63.76	51.18	22.18	53.28	15.80	38.63
	SparseGPT	SparseGPT	27.15	24.83	62.79	49.41	22.35	52.33	17.20	38.15
	SparseGPT	OPTIMA	25.77	23.38	62.95	51.30	22.95	54.70	17.00	38.71

Appendix

Table 8: Qwen-2.5 perplexity on WikiText2 and accuracy on zero-shot downstream tasks for 2:4 sparsity. In this experiment, only the layers in the MLP part of the transformer are pruned, and the self-attention layers are dense, resulting in an end-to-end sparsity ratio of 38% to 41%. OPTIMA improves the average accuracy in 9 of 15 (model size, mask selector) pairs. Within each (Model, Mask Selection) block, the better of the two Weight Update rows is bolded.

Mask Selection	Weight Update	Perplexity	Metrics (%)
Mask Selection	Weight Update	Perplexity	MMLU	PIQA	Arc-E	Arc-C	Wino	OpenQA	Avg.
Wanda	–	23.47 $\pm$ 0.11	26.38 $\pm$ 0.06	64.97 $\pm$ 0.17	51.96 $\pm$ 0.02	23.74 $\pm$ 0.06	54.66 $\pm$ 0.23	18.04 $\pm$ 0.08	39.96 $\pm$ 0.05
Wanda	OPTIMA	18.77 $\pm$ 0.09	27.65 $\pm$ 0.13	67.11 $\pm$ 0.27	52.57 $\pm$ 0.05	24.74 $\pm$ 0.10	55.89 $\pm$ 0.13	20.16 $\pm$ 0.08	41.35 $\pm$ 0.18

Appendix

Table 9: Mean

\pm

standard deviation over five independent draws of 128 C4 calibration samples for LLaMA 3.2 1B at 50% unstructured sparsity.

Figure 3: Sensitivity analysis for the number of calibration samples for different pruning methods.

Model	Mask Selection	Weight Update	Perplexity	Metrics (%)
Model	Mask Selection	Weight Update	Perplexity	MMLU	PIQA	Arc-E	Arc-C	Wino	OpenQA	Avg.
Gemma 3 1B	Dense	–	14.17	24.95	74.81	71.93	35.41	58.72	28.80	49.10
	Wanda	–	32.96	22.97	67.19	61.03	26.37	55.72	20.00	42.21
	Wanda	ADAM	29.25	23.16	69.04	62.71	27.73	57.46	22.20	43.72
	Wanda	OPTIMA	28.90	23.96	69.48	62.84	28.58	56.83	22.40	44.01
	SparseGPT	SparseGPT	28.34	24.85	68.88	60.94	26.62	55.49	21.40	43.03

Appendix

Table 10: Comparison of OPTIMA with other optimizers without convergence guarantees (ADAM). ADAM can lead to suboptimal solutions (Gemma 3 1B) or divergence of the model (OPT 125M).

Mask / Weight Update	LLaMA 3.2 1B		LLaMA 3.2 3B
Mask / Weight Update	Perplexity	Avg. (%)	Perplexity	Avg. (%)
Dense	9.75	49.09	7.81	57.95
Wanda, no update	23.51	40.01	12.92	49.95
Wanda + OPTIMA	18.84	41.33	12.24	51.37
Wanda + ADAM (1)	23.50	39.95	12.92	49.96
Wanda + ADAM (2)	22.82	40.02	12.76	49.88

Appendix

Table 11: Recovery fine-tuning with ADAM on the 128 calibration samples, with the mask held fixed, for 1 to 128 iterations (in parentheses), at 50% unstructured sparsity. Avg. is the six-task average accuracy.

Model	Mask	$10^{-2}$ (default)	$10^{-3}$	$10^{-4}$
LLaMA 3.2 1B	Wanda	18.84 / 41.33	18.63 / 41.56	18.64 / 41.27
LLaMA 3.2 1B	SparseGPT	18.09 / 42.72	17.91 / 42.21	17.86 / 42.12
LLaMA 3.2 1B	Thanos	18.77 / 41.94	18.79 / 42.30	18.38 / 42.54
Gemma 3 1B	Wanda	28.90 / 44.01	28.86 / 43.60	29.07 / 43.64
Gemma 3 1B	SparseGPT	27.35 / 43.76	27.05 / 43.47	27.08 / 43.51
Gemma 3 1B	Thanos	28.14 / 44.05	27.97 / 43.66	27.55 / 43.78

Appendix

Table 12: Solver-tolerance ablation at 50% unstructured sparsity. Each cell reports WikiText2 perplexity / six-task average accuracy (%) for the given relative and absolute rAPDHG tolerance.

Sparsity	Mask Selection	Weight Update	Perplexity	Avg. (%)
Dense	–	–	2.81	70.35
50% unstructured	Wanda	–	5.70	67.04
50% unstructured	Wanda	OPTIMA	5.68	67.30
60% unstructured	Wanda	–	8.04	60.74
60% unstructured	Wanda	OPTIMA	8.26	61.37

Appendix

Table 13: LLaMA-3.1-70B pruned with Wanda masks on a single NVIDIA H100 (about 350 hours end-to-end). Avg. is the six-task average accuracy.

Model	Wanda (s)	SparseGPT (s)	OPTIMA (h)
LLaMA 3.2 1B	15	65	2.5
Gemma 2 2B	21	120	5.5
LLaMA 3.2 3B	27	155	7.0
LLaMA 3.1 8B	50	308	40.0
LLaMA 3.1 70B	–	–	$\approx$ 350

Appendix

Table 14: Mask-selection time of Wanda and SparseGPT versus OPTIMA ’s end-to-end pruning time on a single NVIDIA H100, under the identical protocol (128 calibration samples of 2048 tokens); mask selection is below 1% of OPTIMA ’s time.

LLM Pruning

Large language model pruning removes less useful weights or components to reduce memory and inference costs while preserving model capabilities.

See all 172 papers

ROSE: Reordered SparseGPT for More Accurate One-Shot Large Language Models Pruning
Mar 6, 2026Mingluo Su, Huan WangLLM Pruning Neural Network Pruning
Westlake University
MOONSHOT : A Framework for Multi-Objective Pruning of Vision and Large Language Models
Apr 14, 2026Gabriel Afriat, Xiang Meng, Shibal Ibrahim +2LLM Pruning Neural Network Pruning
Operations Research Center Massachusetts Institute of Technology · Google · OpenAI +1
SpenseGPT: Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference
Jun 9, 2026Jaeseong Lee, Seung-won Hwang, Samyam RajbhandariLLM Pruning High-Performance Computing
Snowflake AI Research* · Seoul National University†

Mar 6, 2026J/K move · Enter open · S save

ROSE: Reordered SparseGPT for More Accurate One-Shot Large Language Models Pruning

Mingluo Su, Huan Wang

Westlake University

LLM Pruning Neural Network Pruning LLM Compression Transformer Pruning

Open paper AskPDF arXiv

Pruning is widely recognized as an effective method for reducing the parameters of large language models (LLMs), potentially leading to more efficient deployment and inference. One classic and prominent path of LLM one-shot pruning is to leverage second-order gradients (i.e., Hessian), represented by the pioneering work SparseGPT. However, the predefined left-to-right pruning order in SparseGPT leads to suboptimal performance when the weights exhibit columnar patterns. This paper studies the effect of pruning order under the SparseGPT framework. The analyses lead us to propose ROSE, a reordered SparseGPT method that prioritizes weights with larger potential pruning errors to be pruned earlier. ROSE first performs pre-pruning to identify candidate weights for removal, and estimates both column and block pruning loss. Subsequently, two-level reordering is performed: columns within each block are reordered in descending order of column loss, while blocks are reordered based on block loss. We introduce the relative range of block loss as a metric to identify columnar layers, enabling adaptive reordering across the entire model. Substantial empirical results on prevalent LLMs (LLaMA2-7B/13B/70B, LLaMA3-8B, Mistral-7B) demonstrate that ROSE surpasses the original SparseGPT and other counterpart pruning methods. Our code is available at https://github.com/mingluo-su/ROSE.

Abstract

LLM Pruning

Explore similar work

ROSE: Reordered SparseGPT for More Accurate One-Shot Large Language Models Pruning

MOONSHOT : A Framework for Multi-Objective Pruning of Vision and Large Language Models

SpenseGPT: Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference