Sparse neural networks are often hypothesized to be more interpretable than dense models, motivated by findings that weight sparsity can produce compact circuits in language models. However, it remains unclear whether structural sparsity itself leads to improved semantic interpretability. In this work, we systematically evaluate the relationship between weight sparsity and interpretability in Vision Transformers using DeiT-III B/16 models pruned with Wanda. To assess interpretability comprehensively, we introduce IMPACT, a multi-level framework that evaluates interpretability across four complementary levels: neurons, layer representations, task circuits, and model-level attribution. Layer representations are analyzed using BatchTopK sparse autoencoders, circuits are extracted via learnable node masking, and explanations are evaluated with transformer attribution using insertion and deletion metrics. Our results reveal a clear structural effect but limited interpretability gains. Sparse models produce circuits with approximately 2.5× fewer edges than dense models, yet the fraction of active nodes remains similar or higher, indicating that pruning redistributes computation rather than isolating simpler functional modules. Consistent with this observation, sparse models show no systematic improvements in neuron-level selectivity, SAE feature interpretability, or attribution faithfulness. These findings suggest that structural sparsity alone does not reliably yield more interpretable vision models, highlighting the importance of evaluation frameworks that assess interpretability beyond circuit compactness.
Figures & tables
Figure 1: The IMPACT (Interpretability Multi-level Pipeline for Assessing Computational Transparency) framework used to evaluate interpretability across four levels of analysis.
Sparsity Ratio
Method
Metric
0.0 (Dense)
0.1
0.3
0.5
0.7
0.8
0.9
Magnitude
Top-1
0.8371
0.8152
0.8173
0.8159
0.8026
0.7661
0.5517
Top-5
0.9655
0.9565
0.9572
0.9563
0.9498
0.9313
0.7873
Wanda
Top-1
0.8371
0.8154
0.8174
0.8158
0.8044
0.7687
0.6151
Top-5
0.9655
0.9563
0.9564
0.9565
0.9510
0.9343
0.8426
Table 1: Accuracy comparison of pruning methods on DeiT-III B/16. Wanda shows slightly stronger performance recovery, particularly at high sparsity. Accuracy degrades substantially beyond the 0.7 sparsity threshold.
Figure 2: Neuron-level interpretability metrics across layers (columns) and sparsity levels (colors). Each distribution spans all 1,000 ImageNet categories.
Sparsity
NMSE ↓
Dead ↓ (%)
Expl. Var. (%) ↑
Dense (0%)
0.0075
85.93
99.25
10%
0.0404
74.97
95.96
30%
0.0396
75.18
96.04
50%
0.0461
72.96
95.39
70%
0.0594
68.31
94.06
Table 2: SAE reconstruction quality for block 11 activations.
Figure 3: SAE feature interpretability for block 11 across sparsity levels.
Model
Node (%)
Size (%) ↓
Acc. (%) ↑
NCQ ↑
Dense
29.03 ± 1.97
32.42 ± 1.75
89.12 ± 6.58
0.602 ± 0.046
Wanda 10%
36.43 ± 1.63
34.23 ± 0.98
92.32 ± 3.86
0.607 ± 0.028
Wanda 30%
36.42 ± 1.74
26.89 ± 0.77
92.31 ± 4.44
0.675 ± 0.035
Wanda 50%
37.83 ± 1.75
19.90 ± 0.61
90.82 ± 5.81
0.727 ± 0.047
Wanda 70%
41.60 ± 2.23
12.94 ± 0.58
89.64 ± 6.22
0.780 ± 0.054
Table 3: Circuit-level comparison across sparsity levels, averaged over 100 ImageNet categories (mean ± std). While sparse models use a larger fraction of their remaining nodes, overall circuit edge size decreases with sparsity, yielding higher NCQ.
Sparsity
Dense
10%
30%
50%
70%
Insertion ↑
0.600
0.703
0.701
0.706
0.663
Deletion ↓
0.212
0.201
0.202
0.203
0.175
Table 4: Attribution faithfulness. Evaluated over all ImageNet categories. A faithful attribution should yield high insertion AUC and low deletion AUC.
Figure 4: Visual attribution comparing the dense DeiT3-B/16 model against models with sparsities of 0.1, 0.3, 0.5, and 0.7.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Tokens
k
Expansion Factor
FVE ↑
NMSE ↓
CLS
128
16
0.9925
0.0075
CLS
128
32
0.9926
0.0074
CLS
128
64
0.9926
0.0074
CLS
32
32
0.9869
0.0131
CLS
64
32
0.9898
0.0102
CLS
256
32
0.9949
0.0051
Appendix
Table 5: SAE reconstruction metrics for the dense DeiT-III B/16 Block 11 activations. The selected configuration ( bold ) represents the saturation point for semantic reconstruction.
Model
k
Exp. Factor ( x )
FVE (Val) ↑
NMSE (Val) ↓
Active
Dead %
Wanda 0.3
128
16
0.9619
0.0381
5,289
56.96%
Wanda 0.3
128
32
0.9618
0.0382
6,092
75.21%
Wanda 0.3
128
64
0.9615
0.0385
7,009
85.74%
Wanda 0.3
256
32
0.9781
0.0219
5,690
76.85%
Wanda 0.5
128
16
0.9557
0.0443
5,721
53.44%
Wanda 0.5
128
32
0.9554
0.0446
6,716
72.67%
Appendix
Table 6: SAE reconstruction sweeps for Wanda-pruned models. Active denotes the count of non-dead features observed during validation.
k Coefficient
Node (%) ↓
Size (%) ↓
Acc. (%) ↑
NCQ ↑
4e-05
41.60 ± 2.23
12.94 ± 0.58
89.64 ± 6.22
0.7804 ± 0.054
5e-05
46.19 ± 3.54
14.49 ± 0.86
89.33 ± 8.39
0.7637 ± 0.071
6e-05
43.71 ± 3.29
13.86 ± 0.76
87.69 ± 8.80
0.7552 ± 0.075
7e-05
33.12 ± 5.44
11.08 ± 1.39
83.77 ± 10.02
0.7449 ± 0.090
8e-05
40.04 ± 2.90
12.99 ± 0.62
83.03 ± 10.89
0.7223 ± 0.094
Appendix
Table 7: Sensitivity of circuit extraction to the sparsity coefficient ( k ) for the Wanda 70% model. Results are averaged over 100 ImageNet categories. The selected coefficient ( bold ) maximizes both Accuracy and NCQ.