Sparse Autoencoders

Also known as SAE

Overview

Sparse autoencoders learn a dictionary of features that can reconstruct a model’s dense activations using only a few active features at a time, helping researchers investigate superposition in language, vision, and multimodal models. Common approaches include L1- and Top-k-based training, BatchTopK, and structured variants that model feature groups, hierarchies, or relationships across layers and modalities. Researchers use the resulting features to explain model behavior and to steer, audit, or remove targeted behaviors, while testing them with reconstruction and interpretability benchmarks as well as causal interventions. A central open tension is that sparse, human-readable features are not necessarily stable across training runs or faithful causes of behavior, and better reconstruction does not by itself guarantee reliable explanations or control.

Latest papers

CardsList

246 papers29 papers in the last four weeks, down 24% on the four weeks before. 0.2% of all new papers.

  1. Separating Interpretability and Reconstruction in Sparse Autoencoders

    Nov 18, 2026Anh Tien Nguyen, Dimitris Samaras, Mahdi S. HosseiniSparse AutoencodersNeural Network Interpretability

  2. Transferable Named Sparse Features for Morphology-based Gleason Grading

    Nov 18, 2026Argyrios Zafeiriou, Athanasios Kallipolitis, Ilias MaglogiannisProstate Cancer GradingComputational Pathology

  3. Representational Change Under Self-Consuming Training: A Sparse Autoencoder Analysis

    Nov 16, 2026Suchit Gupte, Zhongteng Cai, Xueru Zhang +1Self-Training for Language ModelsSparse Autoencoders

  4. Interpretable deep concepts of organoid morphology

    Nov 16, 2026Lion Jasper Gleiter, Tingying PengExplainable Image ClassificationAutomated Interpretability

  5. Right Token, Right Time: From Per-Token Gating to Trajectory-Aware Activation Steering

    Nov 13, 2026Nima Eshraghi, Lovedeep Gondara, Sagarika Suresh +3Language Model SteeringActivation Steering

  6. Representational Atoms in Neural Networks

    Nov 10, 2026Chenhui Hu, Pengfei Cao, Kang Liu +1Neural Network InterpretabilitySemantic Representation Learning

  7. Massive Activations Confound Sparse Autoencoders

    Nov 10, 2026Nathan Delisle, Chenhao TanSparse AutoencodersMassive Activations

  8. Decomposing Earth Embeddings with Sparse Autoencoders

    Nov 8, 2026Vitus Benson, Fanny Yang, Markus ReichsteinInterpretable MLSparse Autoencoders

  9. Learning a Unified Cross-Model Semantic Dictionary via Gated Bottleneck Sparse Autoencoders

    Nov 7, 2026Jinchi Zhu, Tie LuoInterpretable MLShared Representation Learning

  10. Controllable Molecule Generation via Sparse Representation Editing: An Interpretability-Driven Perspective

    Nov 6, 2026Zhuoran Li, Xu Sun, Chang Wen Chen +1Molecular GenerationMolecule Design

  11. Scaling Laws for SAE Training Data

    Nov 5, 2026Nikita Koriagin, Nikita Balagansky, Daniil GavrilovSparse AutoencodersMemory-Efficient Training

  12. When Are Two SAE Latents the Same Feature?

    Nov 3, 2026Alexandra Suvorikova, Ilgam Latypov, Anton Trukhan +1Sparse AutoencodersLatent Space Alignment

  13. From Activation to Intervention: Interpreting Sparse Visual Features through Reconstruction

    Nov 3, 2026Jianyang Gu, Jacob Beattie, Mridul Khurana +7Neural Network InterpretabilityMechanistic Interpretability

  14. Temporal Island Sparse Autoencoders for Interpreting Clinical Time-Series Models

    Nov 2, 2026Muhan Yeo, Bong Gyun Kang, HyunGi Kim +1Interpretable MLSparse Autoencoders

  15. Editing the Goal: Per-Token SAE Steering and Its Depth Limits in π0.5π_{0.5}

    Oct 31, 2026Rocio Perales Valdes, Animesh GargActivation SteeringMechanistic Interpretability

  16. Sparse Autoencoders as Implicit Diversification Regularisers for Cross-Sectional Cryptocurrency Returns

    Oct 29, 2026Weijia Han, Lisha QuFinancePortfolio Optimization

  17. From Error Prediction to Causal Influence: Auditing SAE Features in Multiple-Choice QA

    Oct 27, 2026Aria Nourbakhsh, Adelaide Danilov, Oleksandr Marchenko Breneur +1Sparse AutoencodersCausal Interpretability

  18. MI-MIDI 2: From Observation to Control with Sparse Autoencoders in Text-to-MIDI Generation

    Oct 23, 2026Jakub Poćwiardowski, Mateusz ModrzejewskiMechanistic InterpretabilitySparse Autoencoders

  19. TaxoSAE: Rethinking Sparse Autoencoders through Taxonomic Logic

    Oct 22, 2026Xinyu Guo, Minglai Yang, Clayton T. Morrison +2Sparse AutoencodersHierarchical Representation Learning

  20. Building Auditable AI Text Detectors with Sparse Autoencoders

    Oct 21, 2026Liam Dugan, Callum McDougall, Matthew Denton +3AI-Generated Text DetectionExplainable Text Classification

  21. VLMProbe: An Open-source Library for VLM Probing and Interpretation

    Oct 21, 2026Gexin Huang, Haisheng Fu, Yanting Yang +4VLM InterpretabilitySparse Autoencoders