Mechanistic Interpretability

Follow this topic Ask about this topic

The field seeks to explain how a model’s internal computations produce particular outputs, rather than relying only on input-output behavior. Researchers use sparse autoencoders and transcoders to identify features, and probes, attribution graphs, activation patching, ablation, and steering to trace or test their roles. Studies examine language-model tasks such as factual recall, reasoning, and refusal, as well as vision, audio, and scientific models. A central challenge is that a signal can be easy to decode without being causally used, while circuit explanations and interventions can vary across prompts, models, and analysis choices.

Latest papers

391 papers63 papers in the last four weeks, up 75% on the four weeks before. 0.4% of all new papers.

Representation as a Bottleneck for Mechanistic Interpretability: The Manifestation Unit Protocol
Jun 30, 2026Hussein Chouman, Wataru Sasaki, Tomokazu Matsui +2Transformer Interpretability Mechanistic Interpretability
Ubiquitous Computing Systems Laboratory, Nara Institute of Science and Technology, Nara, Japan.
SemRF: A Semantic Reference Frame for Residual-Stream Dynamics in Language Models
Jun 30, 2026Jian Gu, Aldeida Aleti, Chunyang Chen +1Mechanistic Interpretability Representation Geometry in Language Models
Monash University · Technical University of Munich · Chongqing University
Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?
Jun 30, 2026Philippe Chlenski, Zachariah Carmichael, Ayush Warikoo +5Faithfulness of Language Model Explanations LLM Interpretability
1Meta.
Symbolic Mechanistic Data Attribution: Tracing Training Influence to Learned Behavioral Policies
Jun 28, 2026Reza Habibi, Darian Lee, Magy Seif El-NasrMechanistic Interpretability Training Data Attribution
University of California, Santa Cruz
Vision-Default, Prior-Override: Causal Mechanisms of Perception-Knowledge Conflict in Vision-Language Models
Jun 26, 2026Niclas Lietzow, Danielle Bitterman, Carsten Eickhoff +2Knowledge Conflicts in Language Models VLM Interpretability
University of Tübingen · Harvard University · The University of Texas at Austin
Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models
Jun 26, 2026Yanchen Yin, Dongqi Han, Linghui LiAttention Head Analysis Jailbreak Attacks
Beijing University of Posts and Telecommunications, Beijing, China.
VASAE: Naming SAE Dictionary Directions with Vocabulary-Aligned Anchoring
Jun 26, 2026Kairui Zhang, Ziwen Yu, Zahraa S. Abdallah +1LLM Interpretability Sparse Autoencoders
Intelligent Systems Laboratory, University of Bristol, Bristol, UK · University of Bristol, Bristol, UK · University of Amsterdam, Amsterdam, Netherlands
The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching
Jun 25, 2026Sankaran Vaidyanathan, David Arbour, Aaron Mueller +2Causal Effect Estimation Causal Attribution
University of Massachusetts Amherst · Adobe Research · Boston University
PairSAE: Mechanistic Interpretability from Pair Representations in Protein Co-Folding
Jun 25, 2026Giosue Migliorini, Aristofanis Rontogiannis, Grigori Guitchounts +3Sparse Autoencoders Mechanistic Interpretability
University of California, Irvine · Flagship Pioneering
Inherited Circuits, Learned Semantics: How Fine-Tuning Creates Evasion Vulnerabilities Invisible to Standard Evaluation
Jun 25, 2026Ryan FettermanAdversarial Attacks LLM Fine-Tuning
Cisco Talos
Perfect Detection, Failed Control: The Geometry of Knowing vs. Steering in Language Models
Jun 23, 2026Cosimo Galeone, Anna Ettorre, Minsu Park +2Mechanistic Interpretability Causal Interventions in Language Models
Alomana, Grottaglie, Italy
Can Language Model Agents be Helpful Circuit Explainers in Mechanistic Interpretability?
Jun 23, 2026Ayan Antik Khan, Harsh Kohli, Yuekun Yao +2LLM Agent Evaluation Mechanistic Interpretability
George Mason University · The Ohio State University
What Does a Chemical Language Model Know About Molecules?
Jun 22, 2026Christian Kenneth, Etowah Adams, Liam Bai +1Chemical Language Models Mechanistic Interpretability
Independent · Department of Systems Biology, Columbia University, New York · Generate:Biomedicines, Massachusetts +1
Interpreting Latent CoT Reasoning as Dynamical Systems
Jun 20, 2026Sabari Iyyappan Duraipandian, Shreya Sanjay Boyane, Manju Nagesh +3Latent Dynamics Modeling LLM Interpretability
1San Jose State University. · 2Worcester Polytechnic Institute. · 3George Mason University. +1
Targeted Recovery of Weight-Space Mechanisms From Neural Networks
Jun 19, 2026Antoine Vigouroux, Lee SharkeyTransformer Interpretability Circuit Discovery
1MATS · 2Goodfire
Who Wins the Conflict? Mechanistic Interpretability of Text Bias in Audio LLMs
Jun 17, 2026Hyebin Cho, Suho Yoo, Jaehyuk Jang +2Audio-Language Models Mechanistic Interpretability
School of Electrical Engineering, KAIST
Output Vector Editing for Memorization Mitigation in Large Language Models
Jun 17, 2026Ahmad Dawar Hakimi, Kaiwei Lei, Isabelle Augenstein +1Memorization in Language Models Neural Network Memorization
Center for Information and Language Processing, LMU Munich, Germany · Department of Computer Science, University of Copenhagen, Denmark · Munich Center for Machine Learning, Germany +1
SAE Interventions are Unreliable: Post-Intervention Recovery of Suppressed Behavior
Jun 16, 2026Mingyue Cui, Linghui Shen, Xingyi YangSparse Autoencoders Mechanistic Interpretability
The Hong Kong Polytechnic University
Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditing
Jun 16, 2026Kexin Chen, Yi Liu, Haonan Zhang +3LLM Auditing LLM Interpretability
Zhejiang University · Griffith University
Scalable Circuit Learning for Interpreting Large Language Models
Jun 15, 2026Naiyu Yin, Dennis Wei, Tian Gao +3Circuit Discovery LLM Interpretability
Lehigh University, Mathematics Department, Bethlehem, USA · IBM Research, Yorktown Heights, USA
Demystifying Variance in Circuit Discovery of LLMs
Jun 15, 2026Frank Zhengqing Wu, Francesco Tonin, Volkan CevherPrompt Sensitivity Circuit Discovery
Laboratory for Information and Inference Systems (LIONS) École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland · Laboratory for Informationand Inference Systems (LIONS) École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
Factorized Neural Operators Decompose Dynamic and Persistent Responses
Jun 15, 2026Hao Tang, Yuechen Duan, Jiongyu Zhu +3Mechanistic Interpretability PDE Operator Learning
School of Medicine, University of Dundee, United Kingdom. · School of Data Science, Fudan University, China. · School of Mathematical Sciences, Fudan University, China. +3
Circuit Tracing in Autoregressive Protein Language Models
Jun 14, 2026Darin Tsui, William Deinzer, Daniel Saeedi +1Circuit Discovery Protein Fitness Prediction
School of Electrical and Computer Engineering, Georgia Institute of Technology, Atltanta, GA.
Beyond Importance: Interchange-Sobol Sensitivity Reveals Task-Specific Content Channels in Transformer Components
Jun 12, 2026Yifeng Guo, Jin-Hong Du, Xiang ChenMechanistic Interpretability Causal Interventions in Language Models
St. Jude Children’s Research Hospital Memphis, TN · The University of Hong Kong Hong Kong, CN
When Does Routing Become Interpretable? Causal Probes on Block Attention Residuals
Jun 11, 2026Aydin JavadovTransformer Interpretability Attention Residual Connections
ETH Zurich
Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning
Jun 11, 2026Jiayu Yang, Chao Chen, Shengen Wu +6RL for Language Model Reasoning Mechanistic Interpretability
HKUST(GZ) · University of Cambridge · NTU +2
Where Computation Lives Inside TabPFN: Causal Localisation of Attention Head Function
Jun 11, 2026Atharva Gupta, Dhruv Kumar, Murari Mandal +1Attention Head Specialization Attention Head Analysis
Department of Computer Science, Birla Institute of Technology and Science, Pilani, India · School of Computer Engineering, Kalinga Institute of Industrial Technology, Bhubaneswar, India · Birla AI Labs, Office of Ananya Birla, Aditya Birla Group, India
Unstable Features, Reproducible Subspaces: Understanding Seed Dependence in Sparse Autoencoders
Jun 10, 2026Gleb Gerasimov, Timofei Rusalev, Nikita Balagansky +3Representation Identifiability Sparse Autoencoders
T-Tech
Sparse probes and murky physics: a case study of interpretability challenges in a foundation model for continuum dynamics
Jun 10, 2026Katherine Rosenfeld, Maike SonnewaldNeural Surrogate Modeling Mechanistic Interpretability
Gates Foundation · UC Davis
Can Editing 1 Neuron Fix Repetition Loops in LLMs?
Jun 9, 2026Aristotelis Lazaridis, Aman Sharma, Dylan Bates +3Model Editing Data Repetition in Language Model Training
Edgerunner AI

Jun 30, 2026J/K move · Enter open · S save

Representation as a Bottleneck for Mechanistic Interpretability: The Manifestation Unit Protocol

Hussein Chouman, Wataru Sasaki, Tomokazu Matsui, Hirohiko Suwa+1

Ubiquitous Computing Systems Laboratory, Nara Institute of Science and Technology, Nara, Japan.

Transformer Interpretability Mechanistic Interpretability Explainability Evaluation

Open paper AskPDF arXiv

Mechanistic interpretability has produced a rich inventory of component-level analyses that characterise what neural-network components encode and how they interact. Their outputs, however, are not easily reusable: selectivity tables, circuit diagrams, and feature lists remain locked in per-study notebooks - non-composable, not queryable in natural language, and not directly actionable for downstream audit or intervention. We study the representation layer that sits between these analyses and downstream use as a bottleneck that can be evaluated independently, and introduce Manifestation Units, a typed tuple protocol (E, S, R, D, G) extended with attention-head primitives (T) for transformer architectures, organising per-component statistics into structured fields populated automatically and queried through hybrid retrieval. Instantiated across generative vision (beta-VAE), discriminative vision (CNN), and language (GPT-2), the protocol supports two findings: typed structure substantially outperforms unstructured baselines on retrieval, and CNN filters retrieved by the schema satisfy causal sufficiency and necessity criteria under matched-budget controls. The schema absorbs attention-head primitives without modification, set-recovers known IOI circuit members under retrieval-budget-matched controls, and reveals an irreducible two-field core (S+R) with remaining fields either redundant or actively interfering. We present this as schema infrastructure for mechanistic interpretability rather than frontier-scale validation.