cs.CVNov 3, 2026

From Activation to Intervention: Interpreting Sparse Visual Features through Reconstruction

Authors: Jianyang Gu, Jacob Beattie, Mridul Khurana, Julia Chae, Xi Xiao, Cheng Zhang, Anuj Karpatne, Wei-Lun Chao, 

Organizations: The Ohio State University · Massachusetts Institute of Technology · Virginia Tech · University of Alabama, Birmingham · 

Abstract

Sparse autoencoders (SAEs) decompose visual representations into individual features, offering a promising basis for explaining model behavior. Interpreting these features typically relies on images or pixel regions that most strongly activate them. However, this evidence becomes ambiguous when multiple features with different semantics frequently activate in the same locations, such as texture and color in the same patterns. We propose Reconstructive SAE Intervention (RecSAEI), a framework that shifts feature explanation from passive observation to active visual reconstruction. By intervening on representations along SAE decoder directions and reconstructing the visual output, we provide direct evidence of a feature's semantic meaning. We demonstrate that our method successfully disambiguates feature meanings across multiple datasets and vision encoders. Furthermore, our framework grounds [CLS] SAE interventions in a sparse set of patch SAE decoder directions. Beyond stable target response, this attribution also exposes the compositional relationship between elementary visual patterns and high-level semantics. Our results advocate for intervention reconstruction as an essential complementary signal for explaining neural networks.

Figures & tables

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. What Do Sparse Autoencoder Feature Discovery Scores Identify? From Activations to Interventions

    Oct 11, 2026Yifeng Guo, Yanran Li, Xiang ChenSparse AutoencodersMechanistic Interpretability

  2. ScopeSAE: Model-Scope Feature Discovery with Interpretable Layer Selection

    Oct 4, 2026Qingwen Zeng, Zehao Fu, Shuyu Meng +5Transformer InterpretabilitySparse Autoencoders