Sparse Autoencoder Features

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 45

All topics

A Testable Theory of Atomic Features
Oct 5, 2026Kenny Peng, Jon Kleinberg, Nikhil GargSparse Autoencoder Features
Cornell University
Generative modeling of intrinsically disordered protein regions by reinforcing sparse autoencoder features
Oct 1, 2026Jason X. Liu, Sebastian Ibarraran, Frank Hu +7Protein Design Sparse Autoencoder Features
Stanford University
When Is an SAE Feature Interpretable? A Validation Ladder for EEG Foundation Models
Sep 28, 2026Yucong Cao, Chenqi Li, Tingting ZhuElectroencephalography Foundation Models Sparse Autoencoder Features
Department of Engineering Science University of Oxford
Where Decoder Cosine Similarity Fails for SAE Feature Flow Discovery
Sep 14, 2026Hendrik Droste, Christian Medeiros Adriano, Kathrin Korte +1Sparse Autoencoder Features Feature Learning
Hasso Plattner Institute, Germany · IT University of Copenhagen, Denmark
A Dominant Diffuse Phase in the Sparse Autoencoder Phase Diagram
Sep 9, 2026Alexis D. PlascenciaSparse Autoencoder Features Phase Transitions
Key Path Identification for Resolving Knowledge Conflicts via SAE-based Steering
Sep 8, 2026Wenbo Zhang, Zhongxiang Sun, Zhiguang Han +1Sparse Autoencoder Features Steering
Gaoling School of Artificial Intelligence, Renmin University of China · Nanyang Technological University
SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization
Aug 13, 2026Weihan Meng, Hongzhu Guo, Yi Jing +5Sparse Autoencoder Features Verbalization
Tsinghua University · Peking University · Fudan University
Measuring Semantic Abstractness of SAE Features via Nonlocality
Aug 11, 2026Chuqiao Lin, Shivaji Sondhi, Xiao-Liang QiSparse Autoencoder Features Mechanistic Interpretability
Rudolf Peierls Centre for Theoretical Physics, University of Oxford · Leinweber Institute for Theoretical Physics, Stanford University
CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits
Aug 6, 2026Mehrshad Saadatinia, Parsa Razmara, Ardalan Aryashad +2Linear Activation Steering Steering
University of Southern California, Los Angeles, USA
Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference
Aug 5, 2026Hongsheng Wang, Philipp KoehnMultilingual Language Models Sparse Autoencoder Features
Johns Hopkins University · Baltimore / USA
Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects
Jul 27, 2026Phu Gia Hoang, Anwoy Chatterjee, Tanmoy Chakraborty +2Sparse Autoencoder Features Interpretability
UKP Lab, Technical University of Darmstadt · Department of Electrical Engineering Indian Institute of Technology Delhi, India
Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects
Jul 22, 2026Seonglae Cho, Zekun Wu, Kleyton Da Costa +3Sparse Autoencoder Features Model Activations
Holistic AI · University College London
From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features, from Superposition Geometry to Causal Inertness
Jul 13, 2026Mohamed Abdessalem BalSparse Autoencoder Features Model Auditing
Independent Researcher, Algiers, Algeria
Turn-Averaged SAEs for Feature Discovery and Long-Context Attribution
Jun 26, 2026Kevin Der, Harish Kamath, Ben ThompsonSparse Autoencoder Features Improving Sparse Autoencoders
Anthropic Fellows Program · Anthropic
VASAE: Naming SAE Dictionary Directions with Vocabulary-Aligned Anchoring
Jun 26, 2026Kairui Zhang, Ziwen Yu, Zahraa S. Abdallah +1Sparse Autoencoder Features Improving Sparse Autoencoders
Intelligent Systems Laboratory, University of Bristol, Bristol, UK · University of Bristol, Bristol, UK · University of Amsterdam, Amsterdam, Netherlands
At the Edge of Understanding: Sparse Autoencoders Trace The Limits of Transformer Generalization
Jun 24, 2026Praneet Suresh, Jack Stanley, Sonia Joseph +2Out-Of-Distribution Transformer Architectures
Mila Quebec AI Institute · Meta AI · Sapient Intelligence
SAE Interventions are Unreliable: Post-Intervention Recovery of Suppressed Behavior
Jun 16, 2026Mingyue Cui, Linghui Shen, Xingyi YangSparse Autoencoder Features Latent Feature Interventions
The Hong Kong Polytechnic University
VFUSE: Virulent Feature Understanding with Sparse autoEncoders
Jun 8, 2026Michael Yu, Matthew L. OlsonProtein Design Sparse Autoencoder Features
Raft Bioworks · Independent
Pre-Intervention Prediction of Sparse Autoencoder Steering Side Effects
Jun 6, 2026Evan DuanSparse Autoencoder Features Linear Activation Steering
Re-imagining ISO 26262 in the Age of Autonomous Vehicles: Enhancing Controllability through Transferability and Predictability
Jun 5, 2026Chaitanya Shinde, Hadi Hajieghrary, Paul Schmitt +3Automated Vehicles Controllability
Torc Robotics, Inc., an independent subsidiary of Daimler Truck AG · Reynolds & Moore · Critical Systems Analysis, LLC
How Far Do Auto-Interpretation Labels Generalize: A Controlled Study Across Languages, Scripts, and Rewordings
May 29, 2026Sripad KarneSparse Autoencoder Features Automatic Labelling
Columbia University
On the Relationship Between Activation Outliers and Feature Death in Sparse Autoencoders
May 29, 2026Elana Simon, Etowah Adams, James ZouSparse Autoencoder Features Model Activations
1Stanford University · 2Columbia University
Toward Identifiable Sparse Autoencoders
May 29, 2026Walter Nelson, Theofanis Karaletsos, Francesco LocatelloImproving Sparse Autoencoders Sparse Autoencoder Features
Institute of Science and Technology Austria · 2Pyramidal Inc. · and Achira Inc., USA.
Semantic Optimal Transport for Sparse Autoencoder Feature Matching and Circuit Compression
May 27, 2026Tue M. Cao, Nguyen Do, My T. ThaiSparse Autoencoder Features Distance
Department of Computer and Information Science and Engineering, University of Florida
Feature Geometry of LoRA Adapters: A Sparse Autoencoder Analysis of Representational Divergence in Fine-Tuned Language Models
May 27, 2026Prasanth K KLow-Rank Adaptation Adapters Sparse Autoencoder Features
Independent AI Safety Researcher
Interpretability Transfer from Language to Vision via Sparse Autoencoders
May 24, 2026Alexey Kravets, Da Li, Chuan Li +2Sparse Autoencoder Features Interpretability
Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topography
May 21, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuSparse Autoencoder Features Human Visual Cortex
The University of Hong Kong · Hong Kong, China · Stellaris AI Limited
Reading Task Failure Off the Activations: A Sparse-Feature Audit of GPT-2 Small on Indirect Object Identification
May 21, 2026Mahdi NasermoghadasiSparse Autoencoder Features Spurious Correlations
Research Division, BrightMind AI, Seattle, WA · Texas Tech University, Lubbock, TX
From Correlation to Cause: A Five-Stage Methodology for Feature Analysis in Transformer Language Models
May 21, 2026Caleb MunigetyTransformer Architectures Sparse Autoencoder Features
Independent Researcher
Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)
May 18, 2026Michał Brzozowski, Neo Christopher ChungSparse Autoencoder Features Feature Learning
Samsung AI Center, Warsaw, Poland · University of Warsaw, Poland
Are Sparse Autoencoder Benchmarks Reliable?
May 18, 2026David ChaninSparse Autoencoder Features Autoencoder Architectures
Decode Research, MATS, UCL
Beyond Linear Superposition: Discovering Climate Features in AI Weather Models with KAN-SAE
May 17, 2026Minjong CheonSparse Autoencoder Features Artificial Intelligence Weather Models
Department of Computer Science and Engineering Sejong University
Descriptive Collision in Sparse Autoencoder Auto-Interpretability: When One Explanation Describes Many Features
May 13, 2026Jordan F. McCannSparse Autoencoder Features Model Activations
Independent Researcher
On the Interpretability of Whisper Encodings Using Sparse Autoencoders
May 12, 2026Dan Pluth, Zachary Nicholas Houghton, Yu Zhou +1Speech Encoder Sparse Autoencoder Features
Vail Systems, Inc. · University of Oregon
Embodied AI in Action: Insights from SAE World Congress 2026 on Safety, Trust, Robotics, and Real-World Deployment
May 11, 2026Jan-Mou Li, Paul Schmitt, Wei Tong +4Embodied Artificial Intelligence Artificial Intelligence Systems
SMIXAE: Towards Unsupervised Manifold Discovery in Language Models
May 9, 2026Collin FrancelSparse Autoencoder Features Model Activations
Department of Arts and Sciences, University of Alabama, Tuscaloosa, AL, United States.
Causal Dimensionality of Transformer Representations: Measurement, Scaling, and Layer Structure
May 9, 2026Nilesh Sarkar, Dawar Jyoti DekaSparse Autoencoder Features Transformer Architectures
Erdős AI Lab
Activation Differences Reveal Backdoors: A Comparison of SAE Architectures
May 8, 2026Sachin KumarClean Label Backdoor Attack Sparse Autoencoder Features
LexisNexis* · Raleigh, USA
From Token Lists to Graph Motifs: Weisfeiler-Lehman Analysis of Sparse Autoencoder Features
May 7, 2026Ruben Fernandez-Boullon, Pablo Magariños-Docampo, Javier Perez-RoblesSparse Autoencoder Features Token Co-Occurrence Graphs
University of Vigo
Feature Rivalry in Sparse Autoencoder Representations: A Mechanistic Study of Uncertainty-Driven Feature Competition in LLMs
May 3, 2026HarshavardhanSparse Autoencoder Features Large Language Model Uncertainty
Independent Researcher
Domain-Filtered Knowledge Graphs from Sparse Autoencoder Features
Apr 26, 2026John Winnicki, Abeynaya Gnanasekaran, Eric DarveSparse Autoencoder Features Knowledge Graphs
Institute for Computational and Mathematical Engineering, Stanford University, Stanford, CA, USA
From Concept-Aligned Tokens to Vulnerable Features: Mechanistic Localization of Jailbreaks
Apr 25, 2026Nilanjana Das, Mathew Dawit, Aman Chadha +1Jailbreak Success Rates Unsafe
UMBC, Maryland, U.S.A. · Apple, California, U.S.A.
SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data
Feb 16, 2026David Chanin, Adrià Garriga-AlonsoSparse Autoencoder Features Improving Sparse Autoencoders
University College London · 2MATS · 3Decode Research
Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages
Jul 15, 2025Lyzander Marciano Andrylie, Inaya Rahmanisa, Mahardika Krisna Ihsani +3Sparse Autoencoder Features Multilingual Language Models
Faculty of Computer Science, Universitas Indonesia · Department of Natural Language Processing, MBZUAI
SAEExplainer: Interpreting SAE Features with Activation-Guided Preference Optimization
Date pendingJingyi He, Haiyan Zhao, Ruxue Shi +4Sparse Autoencoder Features Explainable AI Methods
The Chinese University of Hong Kong, Shenzhen · Shanghai Jiao Tong University · NJIT +2

Oct 5, 2026, cs.AIJ/K move · Enter open · S save

A Testable Theory of Atomic Features

Kenny Peng, Jon Kleinberg, Nikhil Garg

Cornell University

Sparse Autoencoder Features

Open paper AskPDF arXiv

We develop and test a theory of language model representations in which there exist atomic features. Our main theoretical insight is that in such a model, sparse dictionaries (e.g., SAEs) of increasing size recover an increasing prefix of the most prevalent atoms in the training data. This "recovery principle" yields three testable predictions: many features in small SAEs are shared by all larger SAEs, SAEs trained on different data share features prevalent in both, and sufficiently large SAEs recover both parent and child features. In contrast to conventional wisdom that SAE features are unstable and "split" as size increases, we find that these predictions hold on SAEs of sizes ranging from 512 to 131,072 trained on two large embedding models. From a theoretical perspective, our results suggest the promise of a scientific theory of representations based on atomic features. Practically, our results suggest the promise of scaling SAEs.

Momentum

Related topics

Latest papers 45