From Activation to Intervention: Interpreting Sparse Visual Features through Reconstruction
Authors: Jianyang Gu, Jacob Beattie, Mridul Khurana, Julia Chae, Xi Xiao, Cheng Zhang, Anuj Karpatne, Wei-Lun Chao, Sara Beery, Tanya Berger-Wolf
Organizations: The Ohio State University · Massachusetts Institute of Technology · Virginia Tech · University of Alabama, Birmingham · Texas A&M University · Boston University
Sparse autoencoders (SAEs) decompose visual representations into individual features, offering a promising basis for explaining model behavior. Interpreting these features typically relies on images or pixel regions that most strongly activate them. However, this evidence becomes ambiguous when multiple features with different semantics frequently activate in the same locations, such as texture and color in the same patterns. We propose Reconstructive SAE Intervention (RecSAEI), a framework that shifts feature explanation from passive observation to active visual reconstruction. By intervening on representations along SAE decoder directions and reconstructing the visual output, we provide direct evidence of a feature's semantic meaning. We demonstrate that our method successfully disambiguates feature meanings across multiple datasets and vision encoders. Furthermore, our framework grounds [CLS] SAE interventions in a sparse set of patch SAE decoder directions. Beyond stable target response, this attribution also exposes the compositional relationship between elementary visual patterns and high-level semantics. Our results advocate for intervention reconstruction as an essential complementary signal for explaining neural networks.
Figures & tables
Figure 1: Fine-grained categories expose confounded SAE dimensions . Top-activated samples and pixel regions cannot effectively interpret the semantics they represent. We apply dimensional intervention on patch embeddings and decode them back to images. Comparing the visual changes from reducing or increasing the corresponding activation more clearly reveals the patterns or semantics represented by various SAE dimensions.
Figure 2: Overview of RecSAEI for patch and [CLS] SAE dimensions . Left : For a patch-SAE feature, we identify highly activated spatial patches and intervene on the corresponding sparse dimension along the decoder direction wk . Right : For a [CLS] feature, we first attribute its activation to patch tokens through backpropagation. After decomposing the patch-level gradient into patch-SAE dimensions, we aggregate top- K decoder directions wk to obtain the intervention direction. We apply the intervention to attributed patches and decode changes through RAE reconstruction.
Explanation approach
NABirds
CUB
Cars
ImageNet
Text
Top
Bottom
Intervention
DINOv3
SigLIP 2
DINOv3
SigLIP 2
DINOv3
SigLIP 2
DINOv3
SigLIP 2
✓
62.8
59.8
59.0
59.1
61.5
61.4
74.1
71.1
✓
66.3
61.0
64.3
63.1
67.4
65.5
74.7
71.8
✓
✓
68.2
63.8
65.4
64.8
68.4
66.0
71.4
72.3
✓
✓
73.0
67.9
70.3
69.7
73.2
69.4
76.9
73.5
✓
✓
✓
74.8
69.2
71.4
71.1
74.3
71.0
76.2
73.9
Table 1: Feature-explanation accuracy using different evidence . Text: descriptions of the target feature. Top: images and regions with high activations. Bottom: images with low activations. Intervention: reconstructions after reducing and increasing the feature activation. Intervention reconstructions provide information complementary to both top and bottom examples to explain SAE features.
Figure 3: Qualitative example of intervention-assisted feature explanation . Top and bottom examples alone might bias the benchmark prediction toward pose, prominence, or “in-flight” status. Intervention reconstruction isolates the more specific change toward bright white tail feathers, leading to a more precise feature interpretation.
Figure 4: RecSAEI provides more consistent and selective control of [CLS] SAE features than dense steering. (a) Target [CLS] response after re-encoding the reconstructed image across intervention strengths. Dense steering leads to image collapse under a small steering strength, while RecSAEI saturates later and provides a higher peak response. (b) Target-to-off-target selectivity for different numbers of attributed patch-SAE features. The dense curve with s>0.01 is dashed, as the reconstruction already collapses. (c) Target-to-off-target selectivity against the number of steered dimensions K . We select s=±0.01 for dense and ±0.03 for RecSAEI to roughly match the target response, as shown in 4(a) .
Figure 5: Qualitative comparison of dense and sparse [CLS] steering . Dense steering induces large global appearance changes and an inconsistent target response, whereas RecSAEI gradually changes the bird’s neck color from brown to grey with black spots, with a consistent [CLS] response across intervention strengths. The original column is a direct RAE reconstruction without intervention.
Figure 6: Effect of the number of steered patch-SAE features K . A small K involves a limited number of patch-SAE features and produces only slight visual changes. Increasing K makes changes more significant in both appearance and target response while preserving the overall image appearance. The intervention converges once the major contributing features are included ( K≥128 ), validated by the target response. The original column is a direct RAE reconstruction without intervention.
Figure 7: Patch-SAE decomposition of a [CLS] SAE feature . Highly attributed patch SAE dimensions localize to different image regions and induce distinct visual changes under intervention, revealing the local sparse components that jointly support the global [CLS] feature. The original column is a direct RAE reconstruction without intervention. We apply interventions to patches with the feature activated instead of attributed patches.
Figure 8: Cross-image consistency of intervention on (a) patch SAEs and (b) [CLS] SAEs . Patch-SAE intervention uses activation to select patches for intervention, while [CLS] intervention uses gradient attribution. Both approaches identify relevant patterns, while negative and positive interventions consistently produce similar visual changes: patch dimension 4443 yields more green herbs and [CLS] dimension 412 yields taller and brighter car headlights. The original columns are direct RAE reconstructions without intervention. γ represents the scale of amplifying the target feature, and s is the ratio of the original patch embedding norm.
Monitored
Direction
Trajectory
Strength
target
correctness
monotonicity
correlation
Patch SAE
64.4%
63.1%
0.71
[CLS] SAE
87.0%
73.9%
0.93
Table 2: Direction agreement of sparse-feature steering . Steering generally changes the monitored feature in the intended direction, particularly for [CLS] SAE intervention, but imperfect direction agreement and trajectory monotonicity indicate that feature control is not exact.
Appendix figures & tables27 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Dataset
MSE
L1
Dead
Almost dead
Dense
DINOv3
NABirds
62.15
2,579.18
0
0
432
CUB
72.78
2,517.16
1
1
473
Cars
75.49
2,577.52
2
2
363
ImageNet
114.62
2,813.61
55
166
407
SigLIP 2
NABirds
0.26
406.78
105
105
475
CUB
0.31
358.01
78
78
544
Appendix
Table 3: Validation statistics of patch SAEs across datasets and visual backbones . We report reconstruction MSE, L1 statistics, and the numbers of dead, almost-dead, and dense features for SAEs trained on DINOv3 and SigLIP 2 representations.
Constraint
DINOv3
SigLIP 2
Max low/high activation ratio
0.15
0.15
Min high-image percentile
0.85
0.85
Min activation-percentile gap
0.60
0.50
Min training activation
1.0
0.1
Dense-similarity baseline percentile
99th
99th
Appendix
Table 4: The thresholds for the paired benchmark construction.
Activation
Explanation Method
Top
Top + Intervention
Contour
67.0
70.5
Overlay
70.6
72.1
Appendix
Table 5: Comparison between different forms of indicating the activation region in the top examples. This experiment uses 200 individual SAE features ( 1,000 image pairs) extracted from NABirds using DINOv3 as the encoder.
Table 14
Figure 9: Effect of steering strength on intervention selectivity . (a) Selectivity comparison using steering strengths ( ∣s∣=0.003 for dense steering and ∣s∣=0.01 for RecSAEI ) with roughly matched target response. RecSAEI remains more selective across different numbers of source dimensions K . (b) Selectivity of RecSAEI across intervention strengths. For negative steering, stronger interventions consistently increase selectivity, whereas positive steering achieves its highest selectivity at relatively small strengths and becomes less selective as the intervention grows stronger. Shaded regions indicate variability across evaluated examples.
Figure 10: Comparison of intervention reconstructions across intervention strengths . We vary the steering strength s from negative to positive while keeping the intervention region fixed by the patch attribution. The first row uses our activation-weighted SAE steering with K=128 source features. The remaining rows replace the steering direction with sensitivity-only SAE aggregation, a contrastive-mean direction, and dense patch steering, respectively. The value below each reconstruction reports the target [CLS] feature response.
Source
Mean absolute
Mean token
Change
dimensions
[CLS] change
intervention
ratio
Top attributed
7.70
396.4
1.94%
Random active
0.23
92.8
0.20%
Bottom attributed
0.01
35.3
0.02%
Appendix
Table 8: Target [CLS] response after steering different patch SAE dimensions . Steering top-attributed dimensions produces substantially larger [CLS] activation changes than steering random active or bottom-attributed dimensions.
Figure 11: Comparison between the proposed activation-weighted dimension attribution and sensitivity-only attribution . (a) Normalized target-feature response under different steering strengths. Sensitivity-only attribution yields larger responses at small strengths but also saturates earlier than activation-weighted attribution. (b) Selectivity for decreasing and increasing interventions. Across steering magnitudes and number of used patch-SAE features, activation-weighted attribution achieves consistently higher selectivity than sensitivity-only steering, with the gap widening as K grows.
Figure 12: Comparison between RecSAEI and contrastive steering derived from the mean feature difference between samples with high and low activation. (a) Normalized target response, measured internally before reconstruction and after reconstruction. Dense steering produces the largest internal response, but much of this effect does not survive reconstruction. In contrast, RecSAEI yields a smaller internal response but a substantially stronger and more consistent post-reconstruction target response. The contrastive-mean direction produces almost no controllable response in either space. (b) Selectivity of the reconstructed interventions. RecSAEI is consistently more selective than both dense steering and the contrastive-mean approach.
Steering
Method
Injected
Final
Internal [CLS]
Round-trip [CLS]
Image MSE
Strength
patch L2
patch L2
target change
target change
s=0.01
Dense
140.8
4,163.9
180.9
28.7
4.6×10−3
RecSAEI K=8
140.8
419.3
21.1
4.7
2.3×10−5
RecSAEI K=128
140.8
418.7
44.8
11.4
3.3×10−5
RecSAEI K=256
140.8
441.5
50.7
12.6
3.9×10−5
s=0.03
Dense
422.4
16,024.6
342.5
−18.5
5.5×10−2
Appendix
Table 9: Round-trip stability of dense steering and RecSAEI under matched injected perturbation magnitude . We report the resulting patch-feature change after propagation through the remaining encoder block, the target [CLS] -feature change before reconstruction, the target change after reconstruction and re-encoding, and the image-space MSE relative to direct reconstruction.
Rank
Patch-SAE
Attribution
Average
Average absolute
Sensitivity
dimension
score
activation
sensitivity
Rank
1
14716
79.14
339.67
0.016
3
2
10967
24.86
238.85
0.004
146
3
9392
18.21
59.24
0.008
20
4
7323
16.43
33.04
0.006
51
5
4630
10.13
77.92
0.005
71
Appendix
Table 10: Top attributed patch SAE features for [CLS] 143 intervention. We report the top 8 patch-SAE dimensions ranked by the attribution score used in our activation-weighted steering direction, together with their average activation within the selected patch mask, average absolute sensitivity to the target [CLS] feature, and their rank under sensitivity-only selection.
Figure 13: Attributed Patch-SAE decomposition of [CLS] SAE feature 143 . Highly attributed patch-SAE dimensions localize to different image regions and induce distinct visual changes under intervention, revealing the local sparse components that jointly support the global [CLS] feature.
Figure 14: Attributed Patch-SAE decomposition of [CLS] SAE feature 409 . Highly attributed patch SAE dimensions localize to different image regions and induce distinct visual changes under intervention, revealing the local sparse components that jointly support the global [CLS] feature.
Figure 15: Attributed Patch-SAE decomposition of [CLS] SAE feature 2662 . Highly attributed patch SAE dimensions localize to different image regions and induce distinct visual changes under intervention, revealing the local sparse components that jointly support the global [CLS] feature.
Figure 16: Intervention reconstruction examples on patch-SAE features of NABirds extracted with DINOv3.
Figure 17: Intervention reconstruction examples on patch-SAE features of NABirds extracted with SigLIP 2.
Figure 18: Intervention reconstruction examples on patch-SAE features of CUB extracted with DINOv3.
Figure 19: Intervention reconstruction examples on patch-SAE features of CUB extracted with SigLIP 2.
Figure 20: Intervention reconstruction examples on patch-SAE features of Cars extracted with DINOv3.
Figure 21: Intervention reconstruction examples on patch-SAE features of Cars extracted with SigLIP 2.
Figure 22: Intervention reconstruction examples on patch-SAE features of ImageNet extracted with DINOv3.
Figure 23: Intervention reconstruction examples on patch-SAE features of ImageNet extracted with SigLIP 2.
Figure 24: Examples of suppressed intervention reconstruction on [CLS] SAE features of NABirds extracted with DINOv3.
Figure 25: Examples of amplified intervention reconstruction on [CLS] SAE features of NABirds extracted with DINOv3.
Figure 26: Examples of suppressed intervention reconstruction on [CLS] SAE features of Cars extracted with DINOv3.
Figure 27: Examples of amplified intervention reconstruction on [CLS] SAE features of Cars extracted with DINOv3.
Figure 28: Intervention reconstruction consistency results on [CLS] features (a) 2434 and (b) 3348 of NABirds extracted with DINOv3.
Neural network interpretability studies how models represent information and make predictions, and how to explain, test, or control their internal computations.
Sparse autoencoder (SAE) features are typically discovered from how their activations co-vary with behaviour, yet they are often used through interventions such as steering or ablation. We take intervention attribution, the first-order effect of a specified intervention on the model output, as our estimand. For a broad class of response functions, we show that observational feature-discovery evidence generally does not identify this attribution, and derive an exact gradient-based decomposition of the gap between linear-probe predictions and intervention attribution. Across four model-SAE pairs, median probe-to-attribution magnitude ratios range from tens to hundreds, while attributions closely track small-step steering. For single-latent ablations with nonzero effects, gradient predictions match the realized sign in 91-97% of cases, compared with 17-35% for probe-implied changes. Attribution-selected sets also produce larger absolute mean ablation effects on held-out prompts than sets selected by correlation, rank tests or probe coefficients. Together, these results establish intervention attribution as a principled, gradient-estimable target for selecting SAE features for intervention.
Yifeng Guo, Yanran Li, Xiang Chen
St. Jude Children’s Research Hospital · Columbia University
Sparse autoencoders (SAEs) are a central tool in mechanistic interpretability. However, existing SAEs are primarily trained per layer. The modeling subspace is therefore fixed by layer identity, independent of which token-layer states actually drive each prediction. We argue that this constraint contributes to several limitations observed in layer-wise SAEs, including low feature utilization, high dictionary redundancy, and features that lack direct behavioral grounding. In this paper, we propose ScopeSAE, which selects the modeling subspace per token by attributing each prediction to its most influential token-layer state via normalized gradient-based attribution, and learns features over the resulting prediction-relevant subspace. Empirically, ScopeSAE yields an effect we term reconstruction-better-than-original. Written-back reconstructions of the SAE produce lower next-token cross-entropy than the original activations, an outcome that, to our knowledge, has not previously been reported for SAEs. Through interventional analyses and a KL fine-tuning counter-experiment, we show that this effect is attributable to ScopeSAE's prediction-relevant subspace itself rather than to architectural changes. ScopeSAE further improves effective feature count, interpretability, utilization, and dictionary redundancy over existing layer-based baselines, suggesting that choosing the SAE modeling subspace by predictive relevance leads to more useful and behaviorally meaningful features.
Qingwen Zeng, Zehao Fu, Shuyu Meng +5
The University of Sydney · The Hong Kong University of Science and Technology (Guangzhou) · DeepWisdom +1
Sparse Autoencoders (SAEs) have emerged as a promising tool for interpreting neural networks by decomposing their activations into sparse sets of human-interpretable features. Recent work has introduced multiple SAE variants and successfully scaled them to frontier models. Despite much excitement, a growing number of negative results in downstream tasks casts doubt on whether SAEs recover meaningful features. To directly investigate this, we perform two complementary evaluations. On a synthetic setup with known ground-truth features, we demonstrate that SAEs recover only 9% of true features despite achieving 71% explained variance, showing that they fail at their core task even when reconstruction is strong. To evaluate SAEs on real activations, we introduce three baselines that constrain SAE feature directions or their activation patterns to random values. Through extensive experiments across multiple SAE architectures, we show that our baselines match fully-trained SAEs in interpretability (0.87 vs 0.90), sparse probing (0.69 vs 0.72), and causal editing (0.73 vs 0.72). Together, these results suggest that SAEs in their current state do not reliably decompose models' internal mechanisms.
Anton Korznikov, Andrey Galichin, Alexey Dontsov +3