Mechanistic Interpretability

Follow this topic Ask about this topic

The field seeks to explain how a model’s internal computations produce particular outputs, rather than relying only on input-output behavior. Researchers use sparse autoencoders and transcoders to identify features, and probes, attribution graphs, activation patching, ablation, and steering to trace or test their roles. Studies examine language-model tasks such as factual recall, reasoning, and refusal, as well as vision, audio, and scientific models. A central challenge is that a signal can be easy to decode without being causally used, while circuit explanations and interventions can vary across prompts, models, and analysis choices.

Latest papers

391 papers63 papers in the last four weeks, up 75% on the four weeks before. 0.4% of all new papers.

Measuring Alignment-Induced Activation Shifts Correctly: A Template-Controlled Difference-in-Differences Protocol
May 23, 2026Yuki NakamuraLLM Alignment Mechanistic Interpretability
The Open University of Japan
Polymorphism Is Rotation: Operational Mechanistic Interpretability from a Two-Layer Transformer to Pythia-70m
May 23, 2026Jordan F. McCannOrthogonal Procrustes Alignment Transformer Interpretability
Independent Researcher
Every Component Is a Lookup: One Linear Graph for Interaction, Composition and Attribution
May 22, 2026Po-Kai Chen, Aske Plaat, Niki van SteinTransformer Interpretability Token-Level Credit Assignment
Leiden University
Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers
May 22, 2026Yongzhong XuTransformer Interpretability Circuit Discovery
Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT?
May 21, 2026Jeanmely Rojas Nunez, Viraj Sawant, Nathan Allen +4Supervised Fine-Tuning Mechanistic Interpretability
Algoverse AI Research · Independent
Transcoders Trace Visual Grounding and Hallucinations in Vision-Language Models
May 21, 2026Dimitrios Damianos, Leon Voukoutis, Georgios Skyrianos +2Object Hallucination in VLMs VLM Interpretability
Institute of Language and Speech Processing, Athena Research Center Athens, Greece
Towards Verifiable Transformers: Solver-Checkable Circuit Explanations
May 21, 2026Neel SomaniNeural Network Verification Transformer
Independent Researcher
Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift
May 21, 2026Sungjun Lim, Heedong Kim, Andrew Lee +1Distribution Shift Robustness OOD Generalization
aYonsei University · bHarvard University
From Circuit Evidence to Mechanistic Theory: An Inductive Logic Approach
May 20, 2026Nura Aljaafari, Danilo S. Carvalho, Andre FreitasCircuit Discovery Mechanistic Interpretability
Department of Computer Science, University of Manchester, United Kingdom · CRUK National Biomarker Centre, University of Manchester, United Kingdom · Idiap Research Institute, Switzerland
A Mechanistic Study of Tabular Foundation Models
May 20, 2026Marin Biloš, James T. Wilson, Anderson Schneider +1Tabular Foundation Models Mechanistic Interpretability
Machine Learning Research, Morgan Stanley
Mechanistic Interpretability for Learning Assurance of a Vision-Based Landing System
May 20, 2026Romeo Valentin, Olivia Beyer Bruvik, Marc R. Schlichting +1Aviation Safety Representation Disentanglement
Stanford Intelligent Systems Laboratory, Stanford University, Stanford, CA, USA
Language-Switching Triggers Take a Latent Detour Through Language Models
May 18, 2026Francis Kulumba, Wissam Antoun, Théo Lasnier +2LLM Backdoor Attacks Mechanistic Interpretability
1Inria Paris · 2Sorbonne Université
Aligned Training: A Parameter-Free Method to Improve Feature Quality and Stability of Sparse Autoencoders (SAE)
May 18, 2026Michał Brzozowski, Neo Christopher ChungSparse Autoencoders Mechanistic Interpretability
Samsung AI Center, Warsaw, Poland · University of Warsaw, Poland
A Distributional View for Visual Mechanistic Interpretability: KL-Minimal Soft-Constraint Principle
May 17, 2026Guancheng Zhou, Yisi Luo, Zhengfu He +5Mechanistic Interpretability
School of Mathematics and Statistics, Xi’an Jiaotong University · Shanghai Innovation Institute · OpenMOSS Team +2
Beyond Linear Superposition: Discovering Climate Features in AI Weather Models with KAN-SAE
May 17, 2026Minjong CheonTransformer Interpretability Climate Science
Department of Computer Science and Engineering Sejong University
Event-Grounded Sparse Autoencoders for Vision-Language-Action Policies
May 17, 2026Xinchen Jin, Aditya Chatterjee, Pranav Kumar +1VLM Interpretability Sparse Autoencoders
Department of Computer Science, Purdue University West Lafayette, IN 47907
Mechanistically Interpretable Neural Encoding Reveals Fine-Grained Functional Selectivity in Human Visual Cortex
May 15, 2026Idan Daniel Grosbard, Mor Geva, Galit YovelMechanistic Interpretability Visual Neuroscience
1Sagol School of Neuroscience · 2Blavatnik School of Computer Science and AI · School of Psychological Sciences
When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability
May 14, 2026ML Nissen Gonzalez, Melwina Albuquerque, Laurence Wroe +3Mechanistic Interpretability Weight-Space Analysis
MARS & University Heidelberg · MARS · MARS & Stanford University +2
The Rate-Distortion-Polysemanticity Tradeoff in SAEs
May 14, 2026Tommaso Mencattini, Francesco Montagna, Francesco LocatelloRate-Distortion Theory Representation Disentanglement
EPFL · Institute of Science and Technology Austria
Exemplar Partitioning for Mechanistic Interpretability
May 14, 2026Jessica RumbelowRepresentation Learning LLM Interpretability
Leap Laboratories
Why Retrieval-Augmented Generation Fails: A Graph Perspective
May 13, 2026Kai Guo, Xinnan Dai, Zhibo Zhang +5Retrieval-Augmented Generation LLM Grounding
Michigan State University · Massachusetts Institute of Technology
Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoders
May 13, 2026William Lehn-Schiøler, Magnus Ruud Kjær, Rahul Thapa +10Sparse Autoencoders Mechanistic Interpretability
BrainCapture, Kongens Lyngby, Denmark · DTU Health Tech, Technical University of Denmark, Kongens Lyngby, Denmark · DTU Compute, Technical University of Denmark, Kongens Lyngby, Denmark +5
Transformers Linearly Represent Highly Structured World Models
May 13, 2026Roman Kniazev, Nathanaël FijalkowCircuit Discovery World Models
LaBRI, CNRS · University of Bordeaux · France
Descriptive Collision in Sparse Autoencoder Auto-Interpretability: When One Explanation Describes Many Features
May 13, 2026Jordan F. McCannLLM Interpretability Sparse Autoencoders
Independent Researcher
In-Context Learning Operates as Concept Subspace Learning
May 12, 2026Wei Tang, Xinyan Jiang, Fakhri Karray +1Representation Learning Mechanistic Interpretability
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · Shanghai Advanced Research Institute, Chinese Academy of Science
OceanCBM: A Concept Bottleneck Model for Mechanistic Interpretability in Ocean Forecasting
May 12, 2026Sanah Suri, Kieran Ringel, Maike SonnewaldConcept Bottleneck Models Mechanistic Interpretability
Department of Computer Science University of California, Davis Davis, CA 95616 · University of Washington, Seattle, WA, USA. · NOAA Geophysical Fluid Dynamics Laboratory, Princeton, NJ, USA.
On the Interpretability of Whisper Encodings Using Sparse Autoencoders
May 12, 2026Dan Pluth, Zachary Nicholas Houghton, Yu Zhou +1Transformer Interpretability Mechanistic Interpretability
Vail Systems, Inc. · University of Oregon
Do Language Models Encode Knowledge of Linguistic Constraint Violations?
May 12, 2026Hardy, Sebastian PadóLLM Interpretability Mechanistic Interpretability
IMS, University of Stuttgart, Stuttgart, Germany · Universitas Mikroskil, Medan, Indonesia.
Dissecting Jet-Tagger Through Mechanistic Interpretability
May 11, 2026Saurabh Rai, Sanmay GangulyTransformer Interpretability Attention Head Analysis
Department of Physics, Indian Institute of Technology, Kanpur. Uttar-Pradesh 208016, India
fmxcoders: Factorized Masked Crosscoders for Cross-Layer Feature Discovery
May 10, 2026Andreas D. Demou, Panagiotis Koromilas, James Oldfield +2Transformer Interpretability Sparse Autoencoders
1The Cyprus Institute · University of Athens · 4Archimedes AI/Athena Research Center +2

May 23, 2026J/K move · Enter open · S save

Measuring Alignment-Induced Activation Shifts Correctly: A Template-Controlled Difference-in-Differences Protocol

Yuki Nakamura

The Open University of Japan

LLM Alignment Mechanistic Interpretability Causal Inference

Open paper AskPDF arXiv

Comparing a model's internal activations before and after alignment is a natural way to ask what safety training changes: one forms the matrix of paired aligned-minus-base activations on safety-relevant inputs and reads off its effective rank or top direction. We show the obvious way to form this matrix is confounded. The aligned model is evaluated under a chat template the base model never saw, so the naive difference conflates the alignment shift with chat formatting. We introduce a four-variant decomposition of the modification matrix (naive, template-controlled, within-aligned, and difference-in-differences, DiD) that separates the two effects. Template control alone removes a 2.0-3.9x inflation of the measured effective rank across Llama-3.1-8B, Gemma-2-9B, and Qwen-2.5-7B; the DiD contrast is what recovers the refusal direction of Arditi et al. (2024), lifting its cosine alignment from 0.18-0.39 to 0.50-0.86. Projection-ablation across the three families confirms the recovered subspace is behaviorally active and that singular-value order is not causal order. We validate the protocol on a controlled testbed and distill it into measurement recommendations for activation-difference studies of alignment.