Calibration Language Shapes Block-Influence Pruning in Multilingual Language Models

Vedant Pandhare; Piyush Dhoka

cs.CLOct 15, 2026

Calibration Language Shapes Block-Influence Pruning in Multilingual Language Models

Authors: Vedant Pandhare, Piyush Dhoka

Organizations: Independent Researcher

Abstract

Block Influence (BI) ranks Transformer blocks for depth pruning from their input-output cosine change, but the ranking depends on the calibration distribution. We study whether English calibration transfers to Indian languages in Qwen2.5-1.5B-Instruct, Llama-3.2-1B, and Gemma-2-2B. English, Hindi, Marathi, and Tamil use disjoint Wikipedia articles for calibration and evaluation; every selector receives the same 150-chunk budget, and Tamil is excluded from multilingual aggregation to test cross-script transfer. Across all three models, late-block BI is 20.7-55.5% higher for the three Indian languages than for English, and every predeclared late-minus-middle contrast has a 95% bootstrap interval above zero. This mismatch changes the low-BI blocks selected for removal despite high global rank correlations. Averaging BI over equal English, Hindi, and Marathi budgets mitigates the resulting perplexity damage for Qwen at six removed blocks and for Gemma at all tested depths, including held-out Tamil. The mitigation is not universal: on Llama, balancing improves Hindi and Marathi at two removed blocks but severely harms Tamil, while larger pruning budgets select the same blocks and collapse perplexity regardless of calibration. Rank-mean and minimax-rank aggregators do not dominate the raw mean. These results support a diagnostic claim rather than a universal pruning rule: English-only BI systematically misses late-layer activity for Indian-language text, but multilingual calibration must still be validated per architecture and pruning budget.

Figures & tables

Model	Parameters	Blocks	$k=2$	$k=4$	$k=6$
Qwen2.5-1.5B-Instruct	1.54B	28	7.1%	14.3%	21.4%
Llama-3.2-1B	1.24B	16	12.5%	25.0%	37.5%
Gemma-2-2B	2.61B	26	7.7%	15.4%	23.1%

Table 1: Models and pruning fractions. Fractions count decoder blocks, not parameters.

Figure 1: Language-conditioned Block Influence. The gray span is the predeclared late region; the final output-adjacent block is excluded from regional contrasts. Curves share a coarse shape but separate near the low-BI late blocks that can enter the pruning set.

Model	Language	Late shift (%)	Late–middle	95% CI
Qwen2.5-1.5B-Instruct	HI	51.0	0.391	[0.378, 0.405]
	MR	55.5	0.392	[0.378, 0.406]
	TA	39.6	0.363	[0.350, 0.376]
Llama-3.2-1B	HI	20.7	0.221	[0.210, 0.233]
	MR	31.8	0.308	[0.297, 0.319]
	TA	25.3	0.428	[0.417, 0.439]

Table 2: Target-language BI relative to English in the predeclared late region. The late–middle column is Eq. ( 6 ); intervals bootstrap independent calibration articles.

Figure 2: Late-minus-middle log BI ratio to English, with 95% bootstrap intervals. All nine contrasts are above zero.

Model	$k$	HI	MR	TA
Qwen2.5-1.5B-Instruct	2	1.00 (=)	1.00 (=)	1.00 (=)
	4	1.004 [1.000, 1.012]	1.003 [1.000, 1.009]	1.003 [1.000, 1.008]
	6	1.449 [1.434, 1.465]	1.504 [1.484, 1.522]	1.372 [1.361, 1.384]
Llama-3.2-1B	2	1.640 [1.553, 1.730]	3.098 [2.768, 3.487]	0.057 [0.053, 0.061]
	4	1.00 (=)	1.00 (=)	1.00 (=)
	6	1.00 (=)	1.00 (=)	1.00 (=)

Table 3: Perplexity ratio

R_{\ell,k}=\mathrm{PPL}(S_{k}^{\textsc{en}})/\mathrm{PPL}(S_{k}^{\mathrm{mean}})

with hierarchical 95% intervals. The marker = denotes identical layer sets for all five seeds. Tamil is held out from balanced selector construction.

Figure 3: English-only versus raw-mean balanced calibration. Values above one favor balanced selection; values below one favor English-only selection. The log scale exposes the Llama–Tamil reversal.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Repository identifier	Revision
Qwen	Qwen/Qwen2.5-1.5B-Instruct	989aa7980e4c...
Llama	meta-llama/Llama-3.2-1B	4e20de362430...
Gemma	google/gemma-2-2b	c5ebcd40d208...

Appendix

Table 4: Model identifiers and immutable repository revisions used for the reported runs.

Model	Language	PPL	BPC	Tokens/word
Qwen2.5-1.5B	EN	10.207	0.790	1.533
	HI	3.827	1.776	4.996
	MR	5.256	2.284	6.730
	TA	3.620	1.920	9.553
Llama-3.2-1B	EN	10.158	0.755	1.459
	HI	6.093	1.357	2.854

Appendix

Table 5: Intact-model perplexity (PPL), bits per character (BPC), and mean tokenizer tokens per whitespace-delimited word. Values are descriptive; PPL is not compared across tokenizers.

Model	Selector	Statistic	$k=2$	$k=4$	SD	$k=6$
Qwen2.5-1.5B-Instruct	EN-only	Mean	6.68	20.72	0.00	88.89
		Worst	8.45	25.61	0.00	107.5
	Raw mean	Mean	6.68	20.36	0.81	30.84
		Worst	8.45	25.27	0.76	38.03
	Rank mean	Mean	6.68	18.90	0.00	30.84
		Worst	8.45	23.91	0.00	38.03

Appendix

Table 6: Target-language relative PPL degradation (%) for English-only and balanced BI selectors.

Model	Selector	$k=2$	$k=4$	$k=6$
Qwen	EN	15,16	14–17	12,14–17,26
Qwen	Raw mean	15,16	14–17	12–17
Llama	EN	12,13	11–14	7,10–14
Llama	Raw mean	12,14	11–14	7,10–14
Gemma	EN	12,23	10–12,23	9–12,22,23
Gemma	Raw mean	10,12	9–12	8–12,23

Appendix

Table 7: Full-pool BI pruning sets for the two primary selectors. Indices are zero-based.

	Qwen			Llama			Gemma
$k$	EN	LB	Random	EN	LB	Random	EN	LB	Random
2	6.7	6.7	72.8	1461.7	2694.4	429.7	50.8	16.8	198.6
4	20.7	20.4	1122.1	$9.06{\times}10^{5}$	$9.06{\times}10^{5}$	$3.13{\times}10^{5}$	78.5	34.2	$7.48{\times}10^{4}$
6	88.9	30.8	$1.81{\times}10^{7}$	$8.76{\times}10^{5}$	$8.76{\times}10^{5}$	$1.01{\times}10^{7}$	242.3	132.5	$2.11{\times}10^{8}$

Appendix

Table 8: Mean relative PPL degradation (%) over Hindi, Marathi, and Tamil. Random is averaged over ten block sets; EN and raw mean are averaged over five calibration seeds.

LLM Pruning

Large language model pruning removes less useful weights or components to reduce memory and inference costs while preserving model capabilities.

See all 178 papers

Rethinking Layer Redundancy: Calibration Matters More Than Search in LLM Depth Pruning
Apr 27, 2026Minkyu Kim, Vincent-Daniel Yun, Youngrae Kim +3LLM Pruning LLM Inference Acceleration
Neural Superintelligence Lab, MODULABS, Republic of Korea · University of Southern California, United States · Boston University, United States +2
Rethinking Calibration for High-Sparsity LLM Pruning: From Single Sources to Capability Coverage
Jun 2, 2026Hu Xu, Zhaolong Xing, Congcong Liu +5LLM Pruning Model Calibration
Shanghai Jiao Tong University
Speech Block Influence: Component-Specific Layer Scoring for Pruning Speech LLMs
Sep 24, 2026Siyu Yao, Du Q. Huynh, Lian Xu +1LLM Pruning Speech Language Models
The University of Western Australia

Apr 27, 2026J/K move · Enter open · S save

Rethinking Layer Redundancy: Calibration Matters More Than Search in LLM Depth Pruning

Minkyu Kim, Vincent-Daniel Yun, Youngrae Kim, Suin Cho+2

Neural Superintelligence Lab, MODULABS, Republic of Korea · University of Southern California, United States · Boston University, United States+2

LLM Pruning LLM Inference Acceleration Neural Network Pruning

Open paper AskPDF arXiv

Depth pruning improves the inference efficiency of large language models by removing Transformer blocks. Prior work typically treats layer redundancy as an inherent structural property of pretrained networks, emphasizing importance criteria and search algorithms to identify removable layers. In this study, we empirically investigate depth pruning from a functional perspective. Evaluating representative LLM families across diverse calibration configurations and multiple search algorithms, we show that different configurations produce different pruning patterns. Furthermore, under a fixed calibration configuration, complex search algorithms yield marginal performance improvements over simple one-shot methods, converging to similar pruned subsets. Overall, our results suggest that the calibration configuration plays a substantially larger role than the choice of search algorithm in shaping pruning patterns and calibration perplexity, while contributing comparably to variance in downstream reasoning accuracy. This indicates that future pruning efforts may benefit from prioritizing the calibration configuration over search complexity.

Abstract

LLM Pruning

Explore similar work

Rethinking Layer Redundancy: Calibration Matters More Than Search in LLM Depth Pruning

Rethinking Calibration for High-Sparsity LLM Pruning: From Single Sources to Capability Coverage

Speech Block Influence: Component-Specific Layer Scoring for Pruning Speech LLMs