Mechanistic Interpretability

Follow this topic Ask about this topic

The field seeks to explain how a model’s internal computations produce particular outputs, rather than relying only on input-output behavior. Researchers use sparse autoencoders and transcoders to identify features, and probes, attribution graphs, activation patching, ablation, and steering to trace or test their roles. Studies examine language-model tasks such as factual recall, reasoning, and refusal, as well as vision, audio, and scientific models. A central challenge is that a signal can be easy to decode without being causally used, while circuit explanations and interventions can vary across prompts, models, and analysis choices.

Latest papers

391 papers63 papers in the last four weeks, up 75% on the four weeks before. 0.4% of all new papers.

VFUSE: Virulent Feature Understanding with Sparse autoEncoders
Jun 8, 2026Michael Yu, Matthew L. OlsonSparse Autoencoders Mechanistic Interpretability
Raft Bioworks · Independent
A retrieval conditioned rebinding circuit for dynamic entity tracking in large language models
Jun 7, 2026Soyoung Oh, Vera DembergLLM Interpretability Mechanistic Interpretability
Saarland University · Max Planck Institute for Informatics
SAEExplainer: Interpreting SAE Features with Activation-Guided Preference Optimization
Jun 7, 2026Jingyi He, Haiyan Zhao, Ruxue Shi +4LLM Interpretability Sparse Autoencoders
The Chinese University of Hong Kong, Shenzhen · Shanghai Jiao Tong University · NJIT +2
Ablation-Reversible Heads Don't Transfer: A Stress Test for Mechanistic Role Claims in Transformers
Jun 6, 2026Philip QuirkeAttention Head Analysis Mechanistic Interpretability
Martian
Shared Semantics, Divergent Mechanisms: Unsupervised Feature Discovery by Aligning Semantics and Mechanisms
Jun 6, 2026Hyunjin Cho, Youngji Roh, Jaehyung KimLLM Auditing Mechanistic Interpretability
Yonsei University
Sparsely gated tiny linear experts
Jun 5, 2026Simon SchugTransformer FFNs Mixture-of-Experts Language Models
Department of Computer Science Princeton University
When Attribution Patching Lies: Diagnosis and a Second-Order Correction
Jun 5, 2026Luyang Zhang, Jialu WangGradient-Based Attribution LLM Interpretability
1Carnegie Mellon University · University of California, Santa Cruz
Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces
Jun 5, 2026Debjyoti Saha RoyMulti-Label Classification CoT Distillation
Khoury College of Computer Sciences, Northeastern University
Inside the Visual Mind: Neuroscience-Motivated Concept Circuits for Interpreting and Steering Vision Transformers
Jun 4, 2026Tang Li, Yanlin Chen, Mengmeng Ma +1Transformer Interpretability Vision Transformer
Department of Computer & Information Science, University of Delaware, Newark, DE, USA · Department of Computer Science, University of Virginia, Charlottesville, VA, USA
Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability
Jun 4, 2026Seyed Arshan Dalili, Mehrdad MahdaviRepresentation Disentanglement Sparse Autoencoders
The Pennsylvania State University
Many Circuits, One Mechanism: Input Variation and Evaluation Granularity in Circuit Discovery
Jun 4, 2026Alireza Bayat Makou, Jingcheng Niu, Subhabrata Dutta +1Circuit Discovery Mechanistic Interpretability
UKP Lab, Technical University of Darmstadt · National Research Center for Applied Cybersecurity ATHENE
From Reward-Hack Activations to Agentic Risk States: Context-Calibrated Mechanistic Monitoring in LLM Agents
Jun 4, 2026Patrick Wilhelm, Odej KaoReward Hacking AI Agent Safety
Distributed Operating Systems, Technische Universität Berlin, Berlin, Germany · BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
When Graph Tokens Sink: A Mechanistic Analysis of Graph Language Models
Jun 2, 2026Ding Zhang, Runtao Zhou, Wenqing Zheng +3LLM Interpretability Mechanistic Interpretability
University of Virginia · Capital One
Decomposing how prompting steers behavior
Jun 2, 2026Fan L. Cheng, Nikolaus KriegeskorteLLM Prompting LLM Interpretability
Columbia University
How Optimality Structures Sparse Dictionaries: A Theory for Understanding SAE Representations
Jun 1, 2026William DorrellRepresentation Learning Sparse Autoencoders
Kempner Institute · Harvard University
Ablating Archetypes: The Stability of Archetypal SAEs is an Artifact of Initialization and Metric Design
Jun 1, 2026Michał Brzozowski, Neo Christopher ChungSparse Autoencoders Mechanistic Interpretability
Samsung AI Center, Warsaw, Poland · University of Warsaw, Poland
Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speech
May 31, 2026Hongfei Du, Jiacheng Shi, Sidi Lu +2TTS Synthesis Sparse Autoencoders
Department of Computer Science, William & Mary, USA.
Query Lens: Interpreting Sparse Key-Value Features with Indirect Effects
May 30, 2026Hwiyeong Lee, Ingyu Bang, Uiji Hwang +2Representation Learning LLM Interpretability
1Hanyang University, Seoul, Republic of Korea.
Latent Space Disentanglement via Activation Steering for Interpretable Attribute Control in Symbolic Music Generation
May 29, 2026Ioannis Prokopiou, Pantelis Vikatos, Maximos Kaliakatsos-Papakostas +2Disentangled Representation Learning Controllable Music Generation
Athens University of Economics and Business Orfium Athens, Greece · Innovation Lab Orfium Athens, Greece · Department of Music Technology and Acoustics Hellenic Mediterranean University Rethymno, Greece +2
Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
May 28, 2026Adly Templeton, Tom Conerly, Jonathan Marcus +23LLM Interpretability Sparse Autoencoders
Anthropic
Semantic Optimal Transport for Sparse Autoencoder Feature Matching and Circuit Compression
May 27, 2026Tue M. Cao, Nguyen Do, My T. ThaiSparse Autoencoders Mechanistic Interpretability
Department of Computer and Information Science and Engineering, University of Florida
Do Agents Think Deeper? A Mechanistic Investigation of Layer-Wise Dynamics in Sequential Planning
May 27, 2026Zhenyu Cui, Xiangzhong LuoLLM Interpretability Mechanistic Interpretability
Revealing Algorithmic Deductive Circuits for Logical Reasoning
May 27, 2026Phuong Minh Nguyen, Tien Huu Dang, Naoya InoueAttention Head Analysis LLM Interpretability
Japan Advanced Institute of Science and Technology
MechRL: Reinforcement Learning Agents Perform Circuit Discovery for Mechanistic Interpretability
May 25, 2026Barsat KhadkaTransformer Interpretability Circuit Discovery
School of Computing Sciences and Engineering The University of Southern Mississippi
Is Inference Mediated by Distinct Semantic Structures in LLMs? A Mechanistic Interpretation
May 25, 2026Nura Aljaafari, Marco Valentino, André FreitasMechanistic Interpretability Language Modeling
University of Manchester, United Kingdom · University of Sheffield, United Kingdom · Idiap Research Institute, Switzerland +1
Transformer Field Theory: A Response-Theoretic Approach to Mechanistic Interpretability
May 24, 2026David N. Olivieri, Antonio F. Pérez RodríguezTransformer Interpretability Mechanistic Interpretability
Departamento de Informática, Universidade de Vigo, Spain · Independent Researcher, Spain
Tiny Brains, Giant Impact: Uncovering the Keystone Neurons of LLM with Just a Few Prompts
May 24, 2026Xiangtian Ji, Yuxin Chen, Zhengzhou Cai +3Fine-Tuning LLM Interpretability
National University of Singapore · Beijing University of Posts and Telecommunications · University of Science and Technology of China
Building Better Activation Oracles
May 23, 2026Jan Bauer, Celeste De Schamphelaere, Adam Karvonen +2LLM Interpretability Mechanistic Interpretability
MATS · Gatsby Unit, UCL · Ghent University +2
Vision-Language Binding in In-Context Image Generation
May 23, 2026Chris Ge, Rohit Gandikota, Antonio Torralba +1Image Generation Mechanistic Interpretability
MIT CSAIL · Northeastern University
CSP-Atlas: Concept-Specific Neural Circuits in a Sparse Python Transformer
May 23, 2026Piotr WilamTransformer Interpretability Circuit Discovery
University College London

Jun 8, 2026J/K move · Enter open · S save

VFUSE: Virulent Feature Understanding with Sparse autoEncoders

Michael Yu, Matthew L. Olson

Raft Bioworks · Independent

Sparse Autoencoders Mechanistic Interpretability Protein Design

Open paper AskPDF arXiv

Generative models have shown remarkable progress in a variety of domains such as protein design, but such power enables the opaque generation of hazardous proteins. In this work, we introduce VFUSE (Virulent Feature Understanding with Sparse autoEncoders), a mechanistic interpretability approach that trains SAEs on diffusion-transformer activations to audit protein models for hazard-aware features. We apply VFUSE to RoseTTAFold3 and RFDiffusion3, popular open-weight models for protein folding and synthesis. We find that for certain blocks, linear probes detect hazardous designs significantly better when fit in the SAE latent space over the original model's representations: improving interpretability without sacrificing model performance. Furthermore, we identify monosemantic features from the SAE that fire only on hazardous designs at up to AUROC

0.84

(

q < 10^{-13}

). To our knowledge this is the first SAE trained on an all-atom diffusion model and the first feature-level virulence audit of a protein design model, paving the way towards safe and interpretable protein design.