How Pruning Reshapes Features: Sparse Autoencoder Analysis of Weight-Pruned Language Models

Hector Borobia; Elies Seguí-Mas; Guillermina Tormo-Carbó

cs.LGMar 26, 2026

How Pruning Reshapes Features: Sparse Autoencoder Analysis of Weight-Pruned Language Models

Authors: Hector Borobia, Elies Seguí-Mas, Guillermina Tormo-Carbó

Organizations: VRAIN – Valencian Research Institute for Artificial Intelligence, Universitat Politècnica de València, Valencia, Spain · Department of Economics and Social Sciences, Universitat Politècnica de València, Valencia, Spain · Department of Business Organisation, Universitat Politècnica de València, Valencia, Spain

Abstract

Weight pruning is a standard technique for compressing large language models, yet its effect on learned internal representations remains poorly understood. We present the first systematic study of how unstructured pruning reshapes the feature geometry of language models, using Sparse Autoencoders (SAEs) as interpretability probes. Across three model families (Gemma 3 1B, Gemma 2 2B, Llama 3.2 1B), two pruning methods (magnitude and Wanda), and six sparsity levels (0--60%), we investigate five research questions spanning seed stability, feature survival, SAE transferability, feature fragility, and causal relevance. Our most striking finding is that rare SAE features--those with low firing rates--survive pruning far better than frequent ones, with within-condition Spearman correlations of rho = -1.0 in 11 of 17 experimental conditions. This counter-intuitive result suggests that pruning acts as implicit feature selection, preferentially destroying high-frequency generic features while preserving specialized rare ones. We further show that Wanda pruning preserves feature structure up to 3.7x better than magnitude pruning, that pre-trained SAEs remain viable on Wanda-pruned models up to 50% sparsity, and that geometric feature survival does not predict causal importance--a dissociation with implications for interpretability under compression.

Figures & tables

Model	Method	0%	20%	30%	40%	50%	60%
Gemma 3 1B	Magnitude	64.2	93.8	333	4,472	$>$ 10 6	$>$ 10 6
Gemma 3 1B	Wanda	—	286	822	9,315	34,020	110,751
Gemma 2 2B	Magnitude	410	—	2,359	—	303,290	—
Gemma 2 2B	Wanda	—	—	489	—	904	—
Llama 3.2 1B	Magnitude	17.7	—	—	—	2,583	—
Llama 3.2 1B	Wanda	—	—	20.9	—	53.1	—

Table 1: WikiText-2 perplexity across models, pruning methods, and sparsity levels. Lower is better. Gemma 2 dense perplexity is anomalously high (likely due to context length mismatch); relative differences remain informative. Catastrophic values (

>10^{5}

) are abbreviated.

Figure 1: Seed stability (MNN rate at $\tau=0.7$ ) as a function of sparsity. Error bars show standard deviation across seed pairs. While absolute MNN rates are low (2–4%), the pattern of degradation with sparsity is consistent.

Figure 2: Dense $\to$ pruned feature survival (MNN at $\tau=0.7$ ). Wanda consistently preserves more features than magnitude pruning across all models and sparsity levels.

Model	Method	Sparsity	FVU	L0
Gemma 3 1B	Dense	0%	0.011	11.4
	Magnitude	20%	0.015	10.2
	Magnitude	30%	0.024	10.0
	Magnitude	50%	0.077	4.5
	Wanda	30%	0.016	10.4
	Wanda	50%	0.024	11.0

Table 2: Transferability: FVU of official (dense-trained) SAEs applied to pruned model activations. Lower FVU indicates better reconstruction. Wanda-pruned activations remain well-reconstructable up to 50% sparsity.

Figure 3: Transferability of pre-trained (official) SAEs to pruned model activations. FVU is plotted against sparsity for Gemma 3 1B and Gemma 2 2B, separated by pruning method. Wanda-pruned activations remain well within the reconstructable range.

Figure 4: Feature survival rate by firing rate quintile ( $\tau=0.7$ ). Across all models and pruning methods, rare features (Q1) survive substantially better than frequent ones (Q5). Lines represent different pruning method–sparsity combinations.

Model	Condition	Q1 (rare)	Q5 (freq)	Ratio
Gemma 3 1B	Mag. $s=0.3$	0.760	0.146	5.2 $\times$
	Wanda $s=0.3$	0.852	0.351	2.4 $\times$
	Wanda $s=0.5$	0.652	0.006	109 $\times$
	Mag. $s=0.5$	0.225	0.001	225 $\times$
Gemma 2 2B	Wanda $s=0.3$	0.867	0.550	1.6 $\times$
Gemma 2 2B	Mag. $s=0.3$	0.265	0.130	2.0 $\times$

Table 3: Feature survival rate by firing rate quintile (

\tau=0.7

). In all conditions, rare features (Q1) survive at substantially higher rates than frequent features (Q5). The ratio column shows the Q1/Q5 survival advantage.

Figure 5: Feature fragility by firing rate, separated by pruning method. Blue: magnitude pruning; orange: Wanda. The monotonic decline from Q1 (rare) to Q5 (frequent) is robust across all models, methods, and sparsity levels.

Figure 6: Mean KL divergence upon single-feature ablation for robust versus fragile features. Across all models and conditions, robust and fragile features show nearly identical causal effects, indicating a dissociation between geometric survival and causal importance.

Model	Condition	Robust KL	Fragile KL	$\boldsymbol{\Delta}$
Gemma 3 1B	Dense	0.695	0.696	0.1%
Gemma 3 1B	Pruned (Wanda 30%)	0.268	0.266	0.6%
Gemma 2 2B	Dense	0.907	0.896	1.2%
Gemma 2 2B	Pruned (Wanda 30%)	0.684	0.686	0.2%
Llama 3.2 1B	Dense	0.497	0.498	0.1%
Llama 3.2 1B	Pruned (Wanda 30%)	0.565	0.566	0.2%

Table 4: Mean KL divergence upon single-feature ablation. Robust and fragile features show nearly identical causal effects across all models and conditions.

\Delta

is the absolute percent difference.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Tier	Model	Layer	Purpose	Runs	Seeds	Steps
A1	Gemma 3 1B	L12	Primary grid	11	3–5	20,000
A3	Gemma 3 1B	L6, L18	Layer depth	4	3	12,000
B1	Gemma 2 2B	L12	Anchor model	5	3–5	20,000
C1	Llama 3.2 1B	L8	Cross-family	4	3–5	12,000

Appendix

Table A.1 : Summary of experimental runs by tier. Seeds column shows number of SAE random seeds per run.

Model	Method	Sparsity	FVU	Alive	$\boldsymbol{n}$
Gemma 3 1B	Magnitude	0.0	0.367	9,216	5
	Magnitude	0.2	0.355	9,216	3
	Magnitude	0.3	0.335	9,216	3
	Magnitude	0.4	0.293	9,216	3
	Magnitude	0.5	0.285	9,216	3
	Magnitude	0.6	0.351	9,216	3

Appendix

Table B.1 : SAE reconstruction quality (FVU) averaged over seeds. The large majority of SAEs achieved near-complete feature utilization with our training procedure.

LLM Pruning

Large language model pruning removes less useful weights or components to reduce memory and inference costs while preserving model capabilities.

See all 178 papers

When Pruning Meets Interpretability: Preserving Sparse Autoencoder Robustness in LLMs
Aug 26, 2026Suchit Gupte, Xueru Zhang, Mohammad Mahdi KhaliliLLM Pruning Sparse Autoencoders
Department of Computer Science The Ohio State University Columbus, OH 43210, USA
Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders
Aug 22, 2025David Chanin, Adrià Garriga-AlonsoActivation Sparsity LLM Interpretability
University College London · MATS Research
Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects
Jul 22, 2026Seonglae Cho, Zekun Wu, Kleyton Da Costa +3LLM Interpretability Sparse Autoencoders
Holistic AI · University College London

Aug 26, 2026J/K move · Enter open · S save

When Pruning Meets Interpretability: Preserving Sparse Autoencoder Robustness in LLMs

Suchit Gupte, Xueru Zhang, Mohammad Mahdi Khalili

Department of Computer Science The Ohio State University Columbus, OH 43210, USA

LLM Pruning Sparse Autoencoders Neural Network Pruning

Open paper AskPDF arXiv

Sparse autoencoders (SAEs) are widely used to interpret the internal representations of large language models (LLMs), yet their reliability under post-hoc model compression remains poorly understood. We present a systematic study of how pruning affects SAE behavior and theoretically show that, for a fixed SAE, its impact is governed by perturbation energy, a covariance-weighted norm. This perspective exposes a key limitation of magnitude pruning: by ignoring activation geometry, it distorts the learned representation space and degrades SAE functionality. Activation-aware methods such as Wanda and SparseGPT, in contrast, implicitly control perturbation energy and are therefore substantially more robust at preserving SAE behavior. We further reveal a consistent structural vulnerability across all pruning methods: middle layers are significantly more sensitive to pruning than early or late layers. Guided by this insight, we propose a layer-wise sparsity allocation strategy, achieving lower perplexity under the same average pruning sparsity. Experiments across four model architectures validate our theoretical findings. Code is publicly available at https://github.com/osu-srml/sae-robustness-under-pruning/tree/main.

Abstract

LLM Pruning

Explore similar work

When Pruning Meets Interpretability: Preserving Sparse Autoencoder Robustness in LLMs

Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders

Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects