Sparse Autoencoders

Also known as SAE

Sparse autoencoders learn a dictionary of features that can reconstruct a model’s dense activations using only a few active features at a time, helping researchers investigate superposition in language, vision, and multimodal models. Common approaches include L1- and Top-k-based training, BatchTopK, and structured variants that model feature groups, hierarchies, or relationships across layers and modalities. Researchers use the resulting features to explain model behavior and to steer, audit, or remove targeted behaviors, while testing them with reconstruction and interpretability benchmarks as well as causal interventions. A central open tension is that sparse, human-readable features are not necessarily stable across training runs or faithful causes of behavior, and better reconstruction does not by itself guarantee reliable explanations or control.

Latest papers

220 papers42 papers in the last four weeks, up 133% on the four weeks before. 0.2% of all new papers.

Reading Task Failure Off the Activations: A Sparse-Feature Audit of GPT-2 Small on Indirect Object Identification
May 21, 2026Mahdi NasermoghadasiTransformer Interpretability Sparse Autoencoders
Research Division, BrightMind AI, Seattle, WA · Texas Tech University, Lubbock, TX
Conceptualizing Embeddings: Sparse Disentanglement for Vision-Language Models
May 21, 2026Piotr Kubaty, Patryk Marszałek, Łukasz Struski +3Disentangled Representation Learning Multimodal Disentangled Representation Learning
Faculty of Mathematics and Computer Science, Jagiellonian University, Kraków, Poland · Doctoral School of Exact and Natural Sciences, Jagiellonian University, Kraków, Poland
Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)
May 18, 2026Michał Brzozowski, Neo Christopher ChungSparse Autoencoders Mechanistic Interpretability
Samsung AI Center, Warsaw, Poland · University of Warsaw, Poland
Are Sparse Autoencoder Benchmarks Reliable?
May 18, 2026David ChaninBenchmark Auditing LLM Interpretability
Decode Research, MATS, UCL
Beyond Linear Superposition: Discovering Climate Features in AI Weather Models with KAN-SAE
May 17, 2026Minjong CheonTransformer Interpretability Climate Science
Department of Computer Science and Engineering Sejong University
Event-Grounded Sparse Autoencoders for Vision-Language-Action Policies
May 17, 2026Xinchen Jin, Aditya Chatterjee, Pranav Kumar +1VLM Interpretability Sparse Autoencoders
Department of Computer Science, Purdue University West Lafayette, IN 47907
Sparse Autoencoders enable Robust and Interpretable Fine-tuning of CLIP models
May 15, 2026Fabian Morelli, Arnas Uselis, Ankit Sonthalia +1Distribution Shift Robustness VLM Robustness
University of Tübingen ∗ · University of Tübingen
The Rate-Distortion-Polysemanticity Tradeoff in SAEs
May 14, 2026Tommaso Mencattini, Francesco Montagna, Francesco LocatelloRate-Distortion Theory Representation Disentanglement
EPFL · Institute of Science and Technology Austria
Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoders
May 13, 2026William Lehn-Schiøler, Magnus Ruud Kjær, Rahul Thapa +10Sparse Autoencoders Mechanistic Interpretability
BrainCapture, Kongens Lyngby, Denmark · DTU Health Tech, Technical University of Denmark, Kongens Lyngby, Denmark · DTU Compute, Technical University of Denmark, Kongens Lyngby, Denmark +5
Descriptive Collision in Sparse Autoencoder Auto-Interpretability: When One Explanation Describes Many Features
May 13, 2026Jordan F. McCannLLM Interpretability Sparse Autoencoders
Independent Researcher
WriteSAE: Sparse Autoencoders for Recurrent State
May 12, 2026Jack YoungRecurrent Language Models Language Model Steering
Indiana University
Disentangled Sparse Representations for Concept-Separated Diffusion Unlearning
May 12, 2026Hyeonjin Kim, Hangyeol Jung, Heechan Yun +2Disentangled Representation Learning Sparse Autoencoders
Yonsei University · KAIST
Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Models
May 12, 2026Boyi Deng, Xu Wang, Yaoning Wang +15LLM Interpretability Sparse Autoencoders
Qwen Team
HH-SAE: Discovering and Steering Hierarchical Knowledge of Complex Manifolds
May 11, 2026Honghan Wu, Tianyan Wang, Jiacong Mi +2Hierarchical Representation Learning Sparse Autoencoders
University of Glasgow, Glasgow, UK · University of International Relations, Beijing, China · University College London, London, UK
The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws
May 11, 2026Eslam Zaher, Maciej Trzaskowski, Quan Nguyen +1Neural Representation Geometry Sparse Autoencoders
ARC Training Centre for Information Resilience (CIRES) · School of Mathematics and Physics, University of Queensland · Institute for Molecular Bioscience, University of Queensland +2
fmxcoders: Factorized Masked Crosscoders for Cross-Layer Feature Discovery
May 10, 2026Andreas D. Demou, Panagiotis Koromilas, James Oldfield +2Transformer Interpretability Sparse Autoencoders
1The Cyprus Institute · University of Athens · 4Archimedes AI/Athena Research Center +2
SMIXAE: Towards Unsupervised Manifold Discovery in Language Models
May 9, 2026Collin FrancelAutoencoders LLM Interpretability
Department of Arts and Sciences, University of Alabama, Tuscaloosa, AL, United States.
Causal Dimensionality of Transformer Representations: Measurement, Scaling, and Layer Structure
May 9, 2026Nilesh Sarkar, Dawar Jyoti DekaTransformer Interpretability Sparse Autoencoders
Erdős AI Lab
Tree SAE: Learning Hierarchical Feature Structures in Sparse Autoencoders
May 8, 2026Tue M. Cao, Hoang X. Nhat, Raed Alharbi +2Hierarchical Representation Learning Sparse Autoencoders
Hanoi University of Science and Technology, Hanoi, Vietnam · University of Florida, Florida, USA · Computer Science Department, Saudi Electronic University
Sparse Autoencoders as Plug-and-Play Firewalls for Adversarial Attack Detection in VLMs
May 8, 2026Hao Wang, Yiqun Sun, Pengfei Wei +2VLM Robustness Adversarial Attacks on VLMs
2Waseda University · 1Magellan Technology Research Institute (MTRI)
Activation Differences Reveal Backdoors: A Comparison of SAE Architectures
May 8, 2026Sachin KumarLLM Backdoor Attacks LLM Interpretability
LexisNexis* · Raleigh, USA
SoftSAE: Dynamic Top-K Selection for Adaptive Sparse Autoencoders
May 7, 2026Jakub Stępień, Marcin Mazur, Jacek Tabor +1Sparse Autoencoders Mechanistic Interpretability
Jagiellonian University · IDEAS Research Institute
Preference Instability in Reward Models: Detection and Mitigation via Sparse Autoencoders
May 7, 2026Shunchang Liu, Xin Chen, Belen Martin Urcelay +1Reward Modeling Sparse Autoencoders
ETH Zürich · Georgia Institute of Technology · ELLIS Institute Finland +1
From Token Lists to Graph Motifs: Weisfeiler-Lehman Analysis of Sparse Autoencoder Features
May 7, 2026Ruben Fernandez-Boullon, Pablo Magariños-Docampo, Javier Perez-RoblesTransformer Interpretability Sparse Autoencoders
University of Vigo
Feature Starvation as Geometric Instability in Sparse Autoencoders
May 6, 2026Faris Chaudhry, Keisuke Yano, Anthea MonodRepresentation Learning Sparse Autoencoders
Imperial College London South Kensington Campus, London SW7 2AZ, UK · Institute for Statistical Mathematics 10-3 Midori-cho, Tachikawa Tokyo 190-8562, Japan
Pairwise matrices for sparse autoencoders: single-feature inspection mislabels causal axes
May 4, 2026Michael A. Riegler, Birk Sebastian Frostelid Torpmann-HagenSparse Autoencoders Mechanistic Interpretability
SimulaMet, Norway · Simula, Norway
Feature Rivalry in Sparse Autoencoder Representations: A Mechanistic Study of Uncertainty-Driven Feature Competition in LLMs
May 3, 2026HarshavardhanLLM Interpretability Sparse Autoencoders
Independent Researcher
GeoSAE: Geometric Prior-Guided Layer-Wise Sparse Autoencoder Annotation of Brain MRI Foundation Models
May 3, 2026Favour Nerrise, Lucy Yin, Mohammad H. Abbasi +2Alzheimer's Disease Medical Imaging Foundation Models
Stanford University · Stanford University, CA
Do Sparse Autoencoders Capture Concept Manifolds?
Apr 30, 2026Usha Bhalla, Thomas Fel, Can Rager +9Neural Representation Geometry Sparse Autoencoders
aHarvard University · bNortheastern University · cTechnion IIT +1
MoRFI: Monotonic Sparse Autoencoder Feature Identification
Apr 29, 2026Dimitris Dimakopoulos, Shay B. Cohen, Ioannis KonstasLLM Hallucination Mitigation LLM Interpretability
University of Edinburgh, UK · Heriot-Watt University, UK

May 21, 2026J/K move · Enter open · S save

Reading Task Failure Off the Activations: A Sparse-Feature Audit of GPT-2 Small on Indirect Object Identification

Mahdi Nasermoghadasi

Research Division, BrightMind AI, Seattle, WA · Texas Tech University, Lubbock, TX

Transformer Interpretability Sparse Autoencoders Language Model Auditing

Open paper AskPDF arXiv

We report a small, reproducible audit of which sparse-autoencoder (SAE) features of GPT-2 small fire differently on failed versus successful trials of the Indirect Object Identification (IOI) task. On 300 prompts, GPT-2 small reaches 79.7% accuracy; 146 of the 24,576 features in the layer-8 residual-stream SAE release of Bloom (2024) clear a Holm-corrected significance threshold and 105 reach a large effect size (|Cohen's d| > 0.8). The strongest single correlate of failure -- feature 17,491, d=+2.93, Neuronpedia label 'cryptographic keys' -- is essentially silent except when the prompt's transferred object is 'the keys,' on which GPT-2 small fails 93.3% of the time vs. 7.5% on the other seven objects (Fisher exact p = 8.79 x 10^-33). We put this correlate through three controls that a mechanistic claim should pass. (i) A causal ablation: zeroing feature 17,491 in the residual stream across all token positions of the 45 keys prompts does not restore accuracy (6.7% -> 4.4%); the feature is a correlate, not a sufficient cause at this layer. (ii) A representation baseline: a logistic regression on the raw 768-dimensional residual stream reaches 5-fold ROC AUC = 0.929, matching the top-100 SAE features (0.927); the SAE basis adds interpretability, not predictive power. (iii) A seed-robustness check: across five random seeds the keys-subset failure rate stays in 75.0--93.3% (the behavioural effect is real), but feature 17,491 is the top-|d| feature in only 1 of 5 runs. The methodological contribution is therefore the audit pipeline (cheap, model-agnostic, surfaces named correlates) rather than any single feature found through it. We release the code, the 300-prompt corpus, the 300x24,576 activation matrix, the ablation and baseline scripts, and the figures. The full pipeline runs on a laptop (Apple M3 Max, no discrete GPU).