Sparse Autoencoders

Also known as SAE

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, up 100% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 149

All topics

Universal Boosts, Specific Suppressors: Sparse Autoencoder Steering of Medical Vision-Language Models
May 24, 2026Farhad Nooralahzadeh, Benjamin Gundersen, Nicolas Deperrois +7Radiology Report Generation Healthcare
University of Zurich and University Hospital of Zurich, Switzerland · Zurich University of Applied Sciences, Switzerland · Kobe University, Japan +3
Interpretability Transfer from Language to Vision via Sparse Autoencoders
May 24, 2026Alexey Kravets, Da Li, Chuan Li +2VLM Interpretability Sparse Autoencoders
Polymorphism Is Rotation: Operational Mechanistic Interpretability from a Two-Layer Transformer to Pythia-70m
May 23, 2026Jordan F. McCannTransformer Interpretability Transformer
Independent Researcher
Steered Generation via Gradient-Based Optimization on Sparse Query Features
May 21, 2026Sumanta Bhattacharyya, Pedram RooshenasConstrained Motion Planning Language Model Steering
Department of Computer Science University of Illinois Chicago
Multilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selection
May 21, 2026Yusser Al Ghussin, Daniil Gurgurov, Tanja Baeumel +3Multilingual Language Models Language Model Steering
Saarland University · German Research Center for Artificial Intelligence (DFKI) · Centre for European Research in Trusted AI (CERTAIN) +1
Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topography
May 21, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuNeural Processes LLM Interpretability
The University of Hong Kong · Hong Kong, China · Stellaris AI Limited
Reading Task Failure Off the Activations: A Sparse-Feature Audit of GPT-2 Small on Indirect Object Identification
May 21, 2026Mahdi NasermoghadasiTransformer Interpretability Sparse Autoencoders
Research Division, BrightMind AI, Seattle, WA · Texas Tech University, Lubbock, TX
Conceptualizing Embeddings: Sparse Disentanglement for Vision-Language Models
May 21, 2026Piotr Kubaty, Patryk Marszałek, Łukasz Struski +3Disentangled Representation Learning Multimodal Disentangled Representation Learning
Faculty of Mathematics and Computer Science, Jagiellonian University, Kraków, Poland · Doctoral School of Exact and Natural Sciences, Jagiellonian University, Kraków, Poland
Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)
May 18, 2026Michał Brzozowski, Neo Christopher ChungSparse Autoencoders Mechanistic Interpretability
Samsung AI Center, Warsaw, Poland · University of Warsaw, Poland
Are Sparse Autoencoder Benchmarks Reliable?
May 18, 2026David ChaninBenchmark Auditing LLM Interpretability
Decode Research, MATS, UCL
Beyond Linear Superposition: Discovering Climate Features in AI Weather Models with KAN-SAE
May 17, 2026Minjong CheonTransformer Interpretability Climate Science
Department of Computer Science and Engineering Sejong University
Event-Grounded Sparse Autoencoders for Vision-Language-Action Policies
May 17, 2026Xinchen Jin, Aditya Chatterjee, Pranav Kumar +1VLM Interpretability Sparse Autoencoders
Department of Computer Science, Purdue University West Lafayette, IN 47907
Sparse Autoencoders enable Robust and Interpretable Fine-tuning of CLIP models
May 15, 2026Fabian Morelli, Arnas Uselis, Ankit Sonthalia +1Distribution Shift Robustness VLM Robustness
University of Tübingen ∗ · University of Tübingen
The Rate-Distortion-Polysemanticity Tradeoff in SAEs
May 14, 2026Tommaso Mencattini, Francesco Montagna, Francesco LocatelloRate-Distortion Theory Representation Disentanglement
EPFL · Institute of Science and Technology Austria
Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoders
May 13, 2026William Lehn-Schiøler, Magnus Ruud Kjær, Rahul Thapa +10Sparse Autoencoders Mechanistic Interpretability
BrainCapture, Kongens Lyngby, Denmark · DTU Health Tech, Technical University of Denmark, Kongens Lyngby, Denmark · DTU Compute, Technical University of Denmark, Kongens Lyngby, Denmark +5
Descriptive Collision in Sparse Autoencoder Auto-Interpretability: When One Explanation Describes Many Features
May 13, 2026Jordan F. McCannLLM Interpretability Sparse Autoencoders
Independent Researcher
WriteSAE: Sparse Autoencoders for Recurrent State
May 12, 2026Jack YoungLanguage Model Steering Sparse Autoencoders
Indiana University
Disentangled Sparse Representations for Concept-Separated Diffusion Unlearning
May 12, 2026Hyeonjin Kim, Hangyeol Jung, Heechan Yun +2Disentangled Representation Learning Sparse Autoencoders
Yonsei University · Kookmin University
Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Models
May 12, 2026Boyi Deng, Xu Wang, Yaoning Wang +15LLM Interpretability Sparse Autoencoders
Qwen Team
HH-SAE: Discovering and Steering Hierarchical Knowledge of Complex Manifolds
May 11, 2026Honghan Wu, Tianyan Wang, Jiacong Mi +2Hierarchical Representation Learning Sparse Autoencoders
University of Glasgow, Glasgow, UK · University of International Relations, Beijing, China · University College London, London, UK
The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws
May 11, 2026Eslam Zaher, Maciej Trzaskowski, Quan Nguyen +1Neural Representation Geometry Sparse Autoencoders
ARC Training Centre for Information Resilience (CIRES) · School of Mathematics and Physics, University of Queensland · Institute for Molecular Bioscience, University of Queensland +2
fmxcoders: Factorized Masked Crosscoders for Cross-Layer Feature Discovery
May 10, 2026Andreas D. Demou, Panagiotis Koromilas, James Oldfield +2Transformer Interpretability Sparse Autoencoders
1The Cyprus Institute · University of Athens · 4Archimedes AI/Athena Research Center +2
SMIXAE: Towards Unsupervised Manifold Discovery in Language Models
May 9, 2026Collin FrancelAutoencoders LLM Interpretability
Department of Arts and Sciences, University of Alabama, Tuscaloosa, AL, United States.
Causal Dimensionality of Transformer Representations: Measurement, Scaling, and Layer Structure
May 9, 2026Nilesh Sarkar, Dawar Jyoti DekaTransformer Interpretability Sparse Autoencoders
Erdős AI Lab
Tree SAE: Learning Hierarchical Feature Structures in Sparse Autoencoders
May 8, 2026Tue M. Cao, Hoang X. Nhat, Raed Alharbi +2Hierarchical Representation Learning Sparse Autoencoders
Hanoi University of Science and Technology, Hanoi, Vietnam · University of Florida, Florida, USA · Computer Science Department, Saudi Electronic University
Sparse Autoencoders as Plug-and-Play Firewalls for Adversarial Attack Detection in VLMs
May 8, 2026Hao Wang, Yiqun Sun, Pengfei Wei +2VLM Robustness Adversarial Attacks on VLMs
2Waseda University · 1Magellan Technology Research Institute (MTRI)
Activation Differences Reveal Backdoors: A Comparison of SAE Architectures
May 8, 2026Sachin KumarLLM Backdoor Attacks LLM Interpretability
LexisNexis* · Raleigh, USA
SoftSAE: Dynamic Top-K Selection for Adaptive Sparse Autoencoders
May 7, 2026Jakub Stępień, Marcin Mazur, Jacek Tabor +1Sparse Autoencoders Mechanistic Interpretability
Jagiellonian University · IDEAS Research Institute
Preference Instability in Reward Models: Detection and Mitigation via Sparse Autoencoders
May 7, 2026Shunchang Liu, Xin Chen, Belen Martin Urcelay +1Reward Modeling Sparse Autoencoders
ETH Zürich · Georgia Institute of Technology · ELLIS Institute Finland +1
From Token Lists to Graph Motifs: Weisfeiler-Lehman Analysis of Sparse Autoencoder Features
May 7, 2026Ruben Fernandez-Boullon, Pablo Magariños-Docampo, Javier Perez-RoblesTransformer Interpretability Sparse Autoencoders
University of Vigo

May 24, 2026, cs.CVJ/K move · Enter open · S save

Universal Boosts, Specific Suppressors: Sparse Autoencoder Steering of Medical Vision-Language Models

Farhad Nooralahzadeh, Benjamin Gundersen, Nicolas Deperrois, Hidetoshi Matsuom+6

University of Zurich and University Hospital of Zurich, Switzerland · Zurich University of Applied Sciences, Switzerland · Kobe University, Japan+3

Radiology Report Generation Healthcare Object Hallucination in VLMs Sparse Autoencoders Medical VLMs Activation Steering+1

Open paper AskPDF arXiv

Medical vision-language models (VLMs) often hallucinate findings when generating chest X-ray reports: they fabricate findings that are not present in the image, miss important ones, or locate them incorrectly. We mitigate this without weight updates by decoding-time residual steering on a per-token sparse autoencoder (SAE) basis: Top-

K

SAEs on late layers, causal steering against clinical errors, then combined suppress/boost intervention at inference time. On the MIMIC-CXR test split, our inference-only method improves the quality of generated reports for three radiology VLMs (RadVLM, LLaVA-Rad, and CheXOne), with relative improvements of +5.4%, +7.2%, and +17.0% in the clinical composite metric, and statistically significant GREEN gains on all backbones. A cross-model feature alignment shows that the quality-promoting (boost) directions overlap strongly across architectures, whereas hallucination-linked (suppress) directions are model-specific. Therefore, transferable steering must treat suppression per-backbone, rather than sharing a universal suppress list. The same recipe transfers zero-shot to IU-Xray (Green

+7.7\%

rel.) without retraining, confirming that the identified features are properties of the model, not of the training corpus. We release causal feature sets and an interactive feature dashboard: https://cxr-sparse-feature-dashboard.netlify.app/.

Momentum

Related topics

Latest papers 149