Transformer Interpretability

Follow this topic Ask about this topic

Transformer interpretability asks how models process information internally, from language and vision to speech, biology, and time series. Researchers use probes and representation geometry to find encoded features, sparse autoencoders to decompose activations, and circuit tracing, ablation, and activation patching to test causal roles; attribution methods explain inputs, while steering tests how internal signals can control outputs. Work also explores interpretable architectures that expose computation by design and controlled algorithmic tasks where the expected solution is known. A central challenge is that information being readable in a representation does not prove the model uses it, and findings can vary with architecture, task, and intervention, making causal validation and reliable evaluation essential.

Latest papers

340 papers34 papers in the last four weeks, down 8% on the four weeks before. 0.2% of all new papers.

MetaSAEs: Joint Training with a Decomposability Penalty Produces More Atomic Sparse Autoencoder Latents
Apr 3, 2026Matthew LevinsonSparse Autoencoders Transformer Interpretability
Independent Researcher
Finding Belief Geometries with Sparse Autoencoders
Apr 3, 2026Matthew LevinsonRepresentation Geometry in Language Models Transformer Interpretability
Independent Researcher
Steerable but Not Decodable: Function Vectors Operate Beyond the Logit Lens
Apr 3, 2026Mohammed Suhail B NadafFunction Vectors Transformer Interpretability
Independent
ViT-Explainer: An Interactive Walkthrough of the Vision Transformer Pipeline
Apr 2, 2026Juan Manuel Hernandez, Mariana Fernandez-Espinosa, Denis Parra +1Transformer Interpretability Explainable Image Classification
Pontificia Universidad Católica de Chile · University of Notre Dame
Regularizing Attention Scores with Bootstrapping
Apr 1, 2026Neo Christopher Chung, Maxim LaletinTransformer Interpretability Attention Mechanisms
University of Warsaw · Samsung AI Center, Warsaw
CircuitProbe: Predicting Reasoning Circuits in Transformers via Stability Zone Detection
Apr 1, 2026Rajkiran PanugantiCircuits in Language Models Transformer Interpretability
Independent Researcher
From Dispersion to Attraction: Spectral Dynamics of Hallucination Across Whisper Model Scales
Mar 31, 2026Ivan Viakhirev, Kirill Borodin, Grach MkrtchianTransformer Interpretability Automatic Speech Recognition
Information Technologies, Mechanics and Optics University, Russia · lab260, Moscow Technical University of Communications and Informatics, Russia · BitmanagerAI, UAE
Tracking Equivalent Mechanistic Interpretations Across Neural Networks
Mar 31, 2026Alan Sun, Mariya TonevaMechanistic Interpretability Transformer Interpretability
Carnegie Mellon University · Max Planck Institute for Software Systems
Unveiling the Reasoning Process of Large Language Models
Mar 31, 2026Junjie Zhang, Zhen Shen, Xisong Dong +1Symbolic Reasoning in Language Models LLM Interpretability
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences
Do All Vision Transformers Need Registers? A Cross-Architectural Reassessment
Mar 26, 2026Spiros Baxevanakis, Platon Karageorgis, Ioannis Dravilas +1Vision Transformer Transformer Interpretability
Informatics Institute, University of Amsterdam
From Weights to Concepts: Data-Free Interpretability of CLIP via Singular Vector Decomposition
Mar 25, 2026Francesco Gentile, Nicola Dall'Asen, Francesco Tonini +3Transformer Interpretability VLM Interpretability
University of Trento · University of Pisa · Fondazione Bruno Kessler
Steering Sparse Autoencoder Latents to Control Dynamic Head Pruning in Vision Transformers (Student Abstract)
Mar 23, 2026Yousung Lee, Dongsoo HarAttention Head Pruning Efficient ViTs
Korea Advanced Institute of Science and Technology, Daejeon 34051, Korea
Stream separation improves Bregman conditioning in transformers
Mar 22, 2026James Clayton KerceTransformer Representation Geometry in Language Models
Georgia Tech Research Institute
Interpreting the Synchronization Gap: The Hidden Mechanism Inside Diffusion Transformers
Mar 22, 2026Emil Albrychiewicz, Andrés Franco Valiente, Li-Ching Chen +1Diffusion Models Diffusion Transformer
Leinweber Institute for Theoretical Physics and Department of Physics, University of California, Berkeley, CA, 94720-7300, USA · Theoretical Physics Group, Lawrence Berkeley National Laboratory, Berkeley, CA 94720-8162, USA · Department of Radiation Oncology, University of California, San Francisco +2
Layer-wise Target Propagation: Efficient Component Attribution through Target Centric Propagation
Mar 20, 2026Lasse Marten Jantsch, Dong-Jae Koh, Seonghyeon Lee +1Transformer Interpretability LLM Interpretability
School of Computer Science and Engineering, Kyungpook National University, South Korea
Transformers are Bayesian Networks
Mar 17, 2026Gregory CoppolaBelief Propagation Transformer
Predictive Statistics Shape Emergent World Representations of Grid Walkers
Mar 17, 2026Sasha Brenner, Thomas R. Knösche, Nico ScherfPredictive Representation Learning World Model Learning
Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany · Leipzig University, Germany · ScaDS.AI, Dresden/Leipzig, Germany
Sparse but not Simpler: A Multi-Level Interpretability Analysis of Vision Transformers
Mar 16, 2026Siyu ZhangNeural Network Interpretability Transformer Interpretability
University of Texas at Austin, Austin TX 78712, USA
A Closer Look into LLMs for Table Understanding
Mar 16, 2026Jia Wang, Chuanyu Qin, Mingyu Zheng +3LLM Interpretability Transformer Interpretability
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
Ablate and Rescue: A Causal Analysis of Residual Stream Hyper-Connections
Mar 16, 2026William Peng, Josheev Rai, Kevin Tseng +2Transformer Hyper-Connections
Stanford University · Georgia Institute of Technology · University of California, Berkeley +2
Understanding the Emergence of Seemingly Useless Features in Next-Token Predictors
Mar 14, 2026Mark Rofin, Jalal Naghiyev, Michael HahnTransformer Interpretability LLM Interpretability
School of Computer and Communication Sciences, EPFL · Technical University of Munich · Zuse School ELIZA +1
Causal Attribution via Activation Patching
Mar 13, 2026Amirmohammad Izadi, Mohammadali Banayeeanzade, Alireza Mirrokni +4Feature Attribution Transformer Interpretability
Sharif University of Technology
Interpreting Negation in GPT-2: Layer- and Head-Level Causal Analysis
Mar 12, 2026Abdullah Al Mofael, Lisa M. Kuhn, Ghassan Alkadi +1Language Modeling Circuits in Language Models
∗Department of Computer Science, Southeastern Louisiana University, Hammond, LA, USA · †Department of Mathematics, Southeastern Louisiana University, Hammond, LA, USA
Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control
Mar 12, 2026Ihor KendiukhovMechanistic Interpretability Transformer Interpretability
Department of Computer Science, University of Tübingen, Germany
The Discrete Charm of the MLP: Binary Routing of Continuous Signals in Transformer Feed-Forward Layers
Mar 11, 2026Peter BaloghTransformer FFNs Transformer Interpretability
Attention Gathers, MLPs Compose: A Causal Analysis of an Action-Outcome Circuit in VideoViT
Mar 11, 2026Sai V R ChereddyTransformer Interpretability Video Representation Learning
Independent Researcher
Task-Conditioned Routing Signatures in Sparse Mixture-of-Experts Transformers
Mar 11, 2026Mynampati Sri Ranganadha AvinashMixture-of-Experts Interpretability Mixture-of-Experts Language Models
Independent Researcher | Asthra Labs
Implicit Statistical Inference in Transformers: Approximating Likelihood-Ratio Tests In-Context
Mar 11, 2026Faris Chaudhry, Siddhant GadkariLikelihood Ratio Testing In-Context Learning
Department of Computer Science Imperial College London
Dissecting Chronos: Sparse Autoencoders Reveal Causal Feature Hierarchies in Time Series Foundation Models
Mar 10, 2026Anurag MishraTime-Series Foundation Models Transformer Interpretability
Rochester Institute of Technology
GIAT: A Geologically-Informed Attention Transformer for Lithology Identification
Mar 10, 2026Jie Li, Qishun Yang, Nuo LiEarth Science Transformer Attention
College of Petroleum, China University of Petroleum-Beijing at Karamay, Karamay 834000, China

Apr 3, 2026J/K move · Enter open · S save

MetaSAEs: Joint Training with a Decomposability Penalty Produces More Atomic Sparse Autoencoder Latents

Matthew Levinson

Independent Researcher

Sparse Autoencoders Transformer Interpretability Disentangled Representation Learning

Open paper AskPDF arXiv

Sparse autoencoders (SAEs) are increasingly used for safety-relevant applications including alignment detection and model steering. These use cases require SAE latents to be as atomic as possible. Each latent should represent a single coherent concept drawn from a single underlying representational subspace. In practice, SAE latents blend representational subspaces together. A single feature can activate across semantically distinct contexts that share no true common representation, muddying an already complex picture of model computation. We introduce a joint training objective that directly penalizes this subspace blending. A small meta SAE is trained alongside the primary SAE to sparsely reconstruct the primary SAE's decoder columns; the primary SAE is penalized whenever its decoder directions are easy to reconstruct from the meta dictionary. This occurs whenever latent directions lie in a subspace spanned by other primary directions. This creates gradient pressure toward more mutually independent decoder directions that resist sparse meta-compression. On GPT-2 large (layer 20), the selected configuration reduces mean

|\varphi|

by 7.5% relative to an identical solo SAE trained on the same data. Automated interpretability (fuzzing) scores improve by 7.6%, providing external validation of the atomicity gain independent of the training and co-occurrence metrics. Reconstruction overhead is modest. Results on Gemma 2 9B are directional. On not-fully-converged SAEs, the same parameterization yields the best results, a

+8.6\%

Δ

Fuzz. Though directional, this is an encouraging sign that the method transfers to a larger model. Qualitative analysis confirms that features firing on polysemantic tokens are split into semantically distinct sub-features, each specializing in a distinct representational subspace.