Frequency Matters: Fast Model-Agnostic Data Curation for Pruning and Quantization

Francesco Pio Monaco; Elia Cunegatti; Flavio Vella; Giovanni Iacca

cs.CLMar 17, 2026

Frequency Matters: Fast Model-Agnostic Data Curation for Pruning and Quantization

Authors: Francesco Pio Monaco, Elia Cunegatti, Flavio Vella, Giovanni Iacca

Organizations: University of Trento

Abstract

Post-training model compression is essential for enhancing the portability of Large Language Models (LLMs) while preserving their performance. While several compression approaches have been proposed, less emphasis has been placed on selecting the most suitable set of data (the so-called calibration data) for finding the compressed model configuration. The choice of calibration data is a critical step in preserving model capabilities both intra- and inter-tasks. In this work, we address the challenge of identifying high-performance calibration sets for both pruning and quantization by analyzing intrinsic data properties rather than model-specific signals. We introduce ZipCal, a model-agnostic data curation strategy that maximizes lexical diversity based on Zipfian power laws. Experiments demonstrate that our method outperforms standard uniform random sampling across various pruning benchmarks. Notably, it also performs on par, in terms of downstream performance, with a state-of-the-art method that relies on model perplexity. The latter becomes prohibitively expensive for large-scale models and datasets, while ZipCal is on average $\sim$ 240 $\times$ faster due to its tractable linear complexity. We make the code and the experiments available at https://github.com/FrancescoMonaco/ZipCal.

Figures & tables

Figure 1: Token frequency distribution of the original datasets and the random, COLA, and ZipCal calibration sets.

			COLA						ZipCal
			Calibration Category						Calibration Category
Model	Task	Dense	LangMod	Math	CommQA	NLI	KnowTran	Mean	LangMod	Math	CommQA	NLI	KnowTran	Mean	$\Delta$
	MMLU-M	24.44	23.33	24.44	23.01	21.85	23.33	23.19	23.01	26.29	21.24	22.02	23.01	23.11	-0.08
	GSM8k	78.09	76.55	75.70	74.40	76.54	75.89	75.82	76.88	76.27	75.82	75.70	76.61	76.25	+0.43
	HellaSwag	71.71	72.03	72.09	71.76	71.67	71.86	71.88	71.78	71.90	71.83	71.59	71.80	71.78	-0.10
	WinoGr.	69.46	68.93	68.51	67.93	68.03	68.07	68.30	67.88	68.43	68.82	68.15	68.03	68.26	-0.04

Table 1: Comparison of COLA vs. ZipCal calibration data under Wanda unstructured pruning at

25\%

sparsity.

		Wanda (25%)		GPTQ (W4A16)		AWQ (W4A16)
Task	Dense	Rand.	ZipCal	Rand.	ZipCal	Rand.	ZipCal
MMLU-M	24.44	23.33	24.14	23.33	24.07	23.12	24.07
GSM8K	78.08	74.98	77.17	70.12	70.63	36.13	38.46
HellaSwag	71.70	64.49	64.28	64.23	64.20	64.85	64.85
WinoGr.	73.56	67.86	68.07	67.21	67.39	67.48	68.71
OBQA	47.80	41.83	42.42	42.02	41.10	40.90	40.45

Table 2: Performance comparison against random sampling across different tasks and compression techniques for Meta-Llama-3.1-8B-Instruct .

			COLA						ZipCal
			Calibration Category						Calibration Category
Model	Task	Dense	LangMod	Math	CommQA	NLI	KnowTran	Mean	LangMod	Math	CommQA	NLI	KnowTran	Mean	$\Delta$
	MMLU-M	24.44	23.33	24.44	23.01	21.85	23.33	23.19	23.01	26.29	21.24	22.02	23.01	23.11	-0.08
	GSM8k	78.09	71.75	68.46	68.84	63.99	65.01	67.61	71.99	74.60	65.13	70.96	67.32	70.00	+2.39
	HellaSwag	71.71	72.07	72.14	71.91	72.06	71.54	71.94	71.84	72.71	71.46	71.89	72.37	72.05	+0.11
	WinoGr.	69.46	66.75	68.15	66.34	66.30	66.73	66.85	66.34	67.40	66.69	67.05	67.32	66.96	+0.11

Table 3: Comparison of COLA vs. ZipCal calibration data under GPTQ W4A16 quantization.

			Wanda (25%)			2SSP (25%)			GPTQ (W4A16)			AWQ (W4A16)
Model	Task	Dense	COLA	ZipCal	$\Delta$	COLA	ZipCal	$\Delta$	COLA	ZipCal	$\Delta$	COLA	ZipCal	$\Delta$
Llama-3.1-8B-Instruct	MMLU-M	24.44	23.19	24.00	+0.81	22.22	22.54	+0.32	23.19	23.01	-0.18	24.44	24.44	0.00
	GSM8k	78.09	75.82	75.80	-0.02	4.88	4.13	-0.75	67.61	74.07	+6.46	68.98	70.50	+1.52
	HellaSwag	71.71	71.88	71.86	-0.02	59.41	59.16	-0.25	71.94	71.73	-0.21	71.68	71.95	+0.27
	WinoGr.	69.46	68.30	68.82	+0.52	60.03	59.68	-0.35	66.85	67.95	+1.10	66.75	68.10	+1.35
	OBQA	47.80	46.79	47.90	+1.11	40.25	40.92	+0.67	45.60	44.40	-1.20	45.90	45.90	0.00

Table 4: Comparison of COLA vs. ZipCal (Multi-Domain) performance across Wanda and 2SSP at 25% sparsity, and GPTQ and AWQ using W4A16 compression scheme.

Figure 2: Running time (log-scale) for calibration data selection. The COLA COLA COLA COLA COLA COLA COLA COLA COLA COLA COLA COLA COLA COLA COLA COLA COLA COLA baseline is run for models of different sizes; whereas ZipCal ZipCal ZipCal ZipCal ZipCal ZipCal ZipCal ZipCal ZipCal ZipCal ZipCal ZipCal ZipCal ZipCal ZipCal ZipCal ZipCal ZipCal is model-agnostic, thus we report the measurement of the only run.

Model	Method	COLA	ZipCal	$\Delta$
Llama-3.1-8B
	Wanda (25%)	$62.96\pm 0.15$	$60.06\pm 0.17$	-2.90
	2SSP (25%)	$40.39\pm 0.12$	$42.86\pm 0.12$	2.47
	GPTQ (W4A16)	$58.40\pm 0.15$	$57.97\pm 0.15$	-0.43
	AWQ (W4A16)	$58.01\pm 0.15$	$60.57\pm 0.15$	2.56

Table 6: Comparison of COLA vs. ZipCal across compression methods on Llama-3.1-8B-Instruct using samples truncated at

l=64

tokens.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1: Effect of calibration data context length on model capabilities across compression techniques for LLaMA-3.1-8B-Instruct .

Figure A2: Effect of the number of calibration data samples on model capabilities across compression techniques for LLaMA-3.1-8B-Instruct .

Figure A3: Token frequency distribution of the original datasets and the random, COLA, and ZipCal sampling calibration sets using 16 samples.

Figure A4: Token frequency distribution of the original datasets and the random, COLA, and ZipCal sampling calibration sets using 1024 samples.

			COLA						ZipCal
			Calibration Category						Calibration Category
Model	Task	Dense	LangMod	Math	CommQA	NLI	KnowTran	Mean	LangMod	Math	CommQA	NLI	KnowTran	Mean	$\Delta$
Llama-3.1-8B-Instruct	MMLU-M	24.44	24.40	24.60	24.30	24.40	24.50	24.44	24.50	24.30	24.40	24.20	24.70	24.42	-0.02
	GSM8k	78.09	72.35	62.51	69.67	69.60	70.77	68.98	71.80	63.10	68.90	70.10	70.50	68.88	-0.10
	HellaSwag	71.71	72.20	71.38	71.28	72.10	71.45	71.68	72.40	71.10	71.50	71.80	71.60	71.68	0.00
	WinoGr.	69.46	67.19	67.05	67.52	65.98	66.02	66.75	66.90	67.30	67.20	66.50	65.80	66.74	-0.01

Appendix

Table A1: Comparison of COLA vs. ZipCal calibration data under AWQ W4A16 compression.

			COLA						ZipCal
			Calibration Category						Calibration Category
Model	Task	Dense	LangMod	Math	CommQA	NLI	KnowTran	Mean	LangMod	Math	CommQA	NLI	KnowTran	Mean	$\Delta$
Llama-3.1-8B-Instruct	MMLU-M	24.44	22.30	23.40	21.80	21.10	22.50	22.22	22.10	22.90	21.50	21.30	22.00	21.96	-0.26
	GSM8k	78.09	6.34	13.68	0.00	0.00	4.36	4.88	4.98	8.42	0.23	3.87	5.65	4.63	-0.25
	HellaSwag	71.71	61.32	60.53	60.66	57.32	57.21	59.41	60.26	61.29	58.73	59.30	59.23	59.76	+0.35
	WinoGr.	69.46	60.69	58.68	59.04	60.18	61.56	60.03	61.69	59.63	60.62	61.80	61.72	61.09	+1.06

Appendix

Table A2: Comparison of COLA vs. ZipCal under 2SSP compression at

25\%

sparsity.

		Wanda (25%)			GPTQ (W4A16)			AWQ (W4A16)
Task	Dense	DSnoT	ZipCal	$\Delta$	DSnoT	ZipCal	$\Delta$	DSnoT	ZipCal	$\Delta$
MMLU-M	24.44	24.04	24.14	+0.10	23.33	24.07	+0.74	23.12	24.07	+0.95
GSM8K	78.08	76.15	77.17	+1.02	70.12	70.63	+0.51	36.13	38.46	+2.33
HellaSwag	71.70	63.12	64.28	+1.16	64.23	64.20	-0.03	64.20	64.85	+0.65
WinoGr.	73.56	67.20	68.07	+0.87	67.39	67.39	+0.00	66.14	68.71	+2.57
OBQA	47.80	42.10	42.42	+0.32	42.02	41.10	-0.92	40.04	40.45	+0.41

Appendix

Table A4: Performance comparison against DSnoT across different tasks and compression techniques for Meta-Llama-3.1-8B-Instruct .

			COLA				ZipCal
Model	Task	Dense	Know-ES	Know-ZH	Comm-ZH	Mean	Know-ES	Know-ZH	Comm-ZH	Mean	$\Delta$
Llama-3.1-8B-Instruct	MMLU-ES	59.00	59.88	59.38	60.00	59.75	59.38	58.88	59.50	59.25	-0.50
	XQuAD-ES	28.84	34.30	34.92	33.31	34.18	38.18	38.80	37.19	38.06	+3.88
	XNLI-ES	44.78	45.16	44.30	43.67	44.38	43.59	42.73	42.10	42.81	-1.57
	MMLU-ZH	54.50	56.13	55.00	54.13	55.08	57.37	56.24	55.37	56.33	+1.25
	XQuAD-ZH	24.51	23.85	25.35	22.77	23.99	23.97	25.47	22.89	24.11	+0.12

Appendix

Table A5: Comparison of COLA vs. ZipCal on multilingual tasks under Wanda compression at 25

\%

sparsity.

			COLA				ZipCal
Model	Task	Dense	Know-ES	Know-ZH	Comm-ZH	Mean	Know-ES	Know-ZH	Comm-ZH	Mean	$\Delta$
Llama-3.1-8B-Instruct	MMLU-ES	59.00	53.12	51.88	51.75	52.25	51.50	50.26	50.13	50.63	-1.62
	XQuAD-ES	28.84	33.75	37.87	40.01	37.21	33.46	37.58	39.72	36.92	-0.29
	XNLI-ES	44.78	43.45	45.26	43.78	44.16	42.21	44.02	42.54	42.92	-1.24
	MMLU-ZH	54.50	44.63	51.50	47.13	47.75	51.50	58.37	54.00	54.62	+6.87
	XQuAD-ZH	24.51	26.94	32.81	30.05	29.94	20.14	26.01	23.25	23.13	-6.81

Appendix

Table A6: Comparison of COLA vs. ZipCal on multilingual tasks under GPTQ W4A16 compression.

			COLA				ZipCal
Model	Task	Dense	Know-ES	Know-ZH	Comm-ZH	Mean	Know-ES	Know-ZH	Comm-ZH	Mean	$\Delta$
Llama-3.1-8B-Instruct	MMLU-ES	59.00	54.13	52.75	48.25	51.71	52.51	51.13	46.63	50.09	-1.62
	XQuAD-ES	28.84	31.97	36.14	33.22	33.78	31.68	35.85	32.93	33.49	-0.29
	XNLI-ES	44.78	43.45	41.14	41.73	42.11	42.21	39.90	40.49	40.87	-1.24
	MMLU-ZH	54.50	49.75	51.50	50.37	50.54	56.62	58.37	57.24	57.41	+6.87
	XQuAD-ZH	24.51	24.25	32.17	24.36	26.93	17.44	25.36	17.55	20.12	-6.81

Appendix

Table A7: Comparison of COLA vs. ZipCal on multilingual tasks under AWQ W4A16 compression.

			COLA				ZipCal
Model	Task	Dense	Know-ES	Know-ZH	Comm-ZH	Mean	Know-ES	Know-ZH	Comm-ZH	Mean	$\Delta$
Llama-3.1-8B-Instruct	MMLU-ES	59.00	44.00	41.75	33.75	39.83	45.50	43.25	35.25	41.33	+1.50
	XQuAD-ES	28.84	49.59	50.03	41.25	46.96	49.94	50.38	41.60	47.31	+0.35
	XNLI-ES	44.78	45.46	43.67	40.76	43.30	47.97	46.18	43.27	45.81	+2.51
	MMLU-ZH	54.50	38.00	43.63	34.88	38.83	35.25	40.88	32.13	36.08	-2.75
	XQuAD-ZH	24.51	25.75	26.90	20.40	24.35	22.02	23.17	16.67	20.62	-3.73

Appendix

Table A8: Comparison of COLA vs. ZipCal on multilingual tasks under 2SSP compression with

25\%

sparsity.

			Wanda (25%)			2SSP (25%)			GPTQ (W4A16)			AWQ (W4A16)
Model	Task	Dense	COLA	ZipCal	$\Delta$	COLA	ZipCal	$\Delta$	COLA	ZipCal	$\Delta$	COLA	ZipCal	$\Delta$
Llama-3.1-8B-Instruct	MMLU-ES	59.00	59.75	62.06	+2.31	39.83	42.50	+2.67	52.25	51.00	-1.25	51.71	57.75	+6.04
	XQuAD-ES	28.84	34.18	39.00	+4.82	46.96	53.13	+6.17	37.21	29.90	-7.31	33.78	29.04	-4.74
	XNLI-ES	44.78	44.38	44.21	-0.17	43.30	44.01	+0.71	44.16	43.85	-0.31	42.11	46.46	+4.35
	MMLU-ZH	54.50	55.08	57.25	+2.17	38.83	36.50	-2.33	47.75	55.00	+7.25	50.54	52.75	+2.21
	XQuAD-ZH	24.51	23.99	25.38	+1.39	24.35	31.88	+7.53	29.94	14.92	-15.02	26.93	24.96	-1.97

Appendix

Table A9: Comparison of COLA vs. ZipCal (Multi-Domain and Multi-Lingual) performance across Wanda and 2SSP at 25% sparsity, and GPTQ and AWQ using W4A16 compression scheme.

			COLA						ZipCal
			Calibration Category						Calibration Category
Compression	Task	Dense	LangMod	Math	CommQA	NLI	KnowTran	Mean	LangMod	Math	CommQA	NLI	KnowTran	Mean	$\Delta$
Wanda (25%)	MMLU-M	21.00	20.50	21.20	21.00	21.30	21.00	21.00	20.50	21.20	21.00	21.30	21.00	21.00	0.00
	HellaSwag	60.05	59.24	59.94	59.58	60.05	59.89	59.74	59.02	59.72	59.35	59.83	59.42	59.47	-0.27
	WinoGr.	65.19	65.68	66.38	66.61	66.48	65.75	66.18	64.91	65.61	65.63	65.71	65.04	65.38	-0.80
	OBQA	35.80	35.30	36.00	35.60	36.10	36.00	35.80	34.23	34.93	34.90	35.03	34.70	34.76	-1.04

Appendix

Table A10: Comparison of COLA vs. ZipCal calibration data on Qwen3-8B .

			COLA						ZipCal
			Calibration Category						Calibration Category
Compression	Task	Dense	LangMod	Math	CommQA	NLI	KnowTran	Mean	LangMod	Math	CommQA	NLI	KnowTran	Mean	$\Delta$
Wanda (25%)	MMLU-M	45.18	39.10	39.50	39.20	39.60	39.40	39.36	39.20	39.50	39.10	39.40	39.45	39.33	-0.03
	GSM8k	88.32	75.60	76.10	75.50	76.00	75.90	75.82	75.90	75.60	75.85	76.10	75.75	75.84	+0.02
	HellaSwag	78.16	70.00	70.30	70.90	70.40	70.15	70.15	70.25	70.10	70.30	69.95	70.20	70.16	+0.01
	WinoGr.	73.16	73.10	73.50	73.20	73.60	73.25	73.33	73.20	73.40	73.15	73.50	73.35	73.32	-0.01

Appendix

Table A11: Comparison of COLA vs. ZipCal calibration data on Llama-3.1-70B-Instruct .

			Wanda (25%)			2SSP (25%)			GPTQ (W4A16)			AWQ (W4A16)
Model	Task	Dense	COLA	ZipCal	$\Delta$	COLA	ZipCal	$\Delta$	COLA	ZipCal	$\Delta$	COLA	ZipCal	$\Delta$
Llama-3.1-70B-Instruct	MMLU-M	45.18	39.36	50.00	+10.64	22.22	36.66	+14.44	44.17	45.18	+1.01	43.28	46.66	+3.38
	GSM8k	88.32	75.82	88.32	+12.50	42.77	63.22	+20.45	67.61	79.16	+11.55	68.98	77.46	+8.48
	HellaSwag	78.16	70.15	78.16	+8.01	75.45	78.16	+2.71	70.81	78.16	+7.35	70.82	72.82	+2.00
	WinoGr.	73.16	73.33	73.16	-0.17	70.63	72.45	+1.82	73.42	73.16	-0.26	73.75	72.61	-1.14
	OBQA	48.80	45.43	48.80	+3.37	41.51	49.00	+7.49	45.76	48.80	+3.04	45.27	49.40	+4.13

Appendix

Table A12: Comparison of COLA vs. ZipCal (Multi-Domain) performance across Wanda and 2SSP at 25% sparsity, and GPTQ and AWQ using W4A16 compression scheme on Llama-3.1-70B-Instruct .

LLM Compression

Large language model compression reduces the memory, compute, and context costs of LLMs while aiming to preserve their capabilities.

See all 273 papers

Damage Predicts Recovery: When Calibration Data Matters in Compressing Financial LLMs
Aug 12, 2026Junyi Ye, Mengjia Yu, Debapriya Hazra +1Language Model Calibration Post-Training Quantization
School of Computing, Montclair State University Montclair, New Jersey, USA · Department of Computer Science, New Jersey Institute of Technology Newark, New Jersey, USA
LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry
Sep 10, 2026Sankar Behera, Dhruv Singh, Anshika Agnihotri +3LLM Pruning LLM Compression
CSE, IIT Jammu · Mathematics, IIT Jammu · IT, EC Bikaner +1
Capability Scaling-Down Laws for LLM Compression
Oct 1, 2026Xueqi Cheng, Liang Wu, Kelly Wan +2LLM Compression LLM Pruning
Florida State University · Nokia Applied Research

Aug 12, 2026J/K move · Enter open · S save

Damage Predicts Recovery: When Calibration Data Matters in Compressing Financial LLMs

Junyi Ye, Mengjia Yu, Debapriya Hazra, Guiling Wang

School of Computing, Montclair State University Montclair, New Jersey, USA · Department of Computer Science, New Jersey Institute of Technology Newark, New Jersey, USA

Language Model Calibration Post-Training Quantization LLM Pruning

Open paper AskPDF arXiv

Post-training quantization and pruning rely on a small calibration corpus. Whether specialized domains such as finance require domain-matched calibration data remains unsettled. We argue that the answer depends on the task-level damage caused by compression rather than on domain mismatch. If compression preserves the target capability, changing the calibration corpus has little effect. If compression causes large losses, task-formatted calibration can recover part of the loss. We test this hypothesis across two model families, six compression configurations, three token-matched calibration corpora, and ten financial classification and numerical question-answering tasks. The results support this hypothesis. Quantization largely preserves task performance, and calibration choice has little effect in this case. Pruning reduces numerical QA accuracy by over 40 points. In these damaged settings, another generic corpus does not help, while FinMix, a mixture of financial task examples, recovers a large part of the loss. The link between damage and recovery holds across model families and scales. These findings support a practical rule. Measure task-specific compression damage first, and construct specialized calibration data only when the damage is large.

Abstract

LLM Compression

Explore similar work

Damage Predicts Recovery: When Calibration Data Matters in Compressing Financial LLMs

LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry

Capability Scaling-Down Laws for LLM Compression