Transformer Interpretability

Ask about this topic

Overview

Transformer interpretability asks how models process information internally, from language and vision to speech, biology, and time series. Researchers use probes and representation geometry to find encoded features, sparse autoencoders to decompose activations, and circuit tracing, ablation, and activation patching to test causal roles; attribution methods explain inputs, while steering tests how internal signals can control outputs. Work also explores interpretable architectures that expose computation by design and controlled algorithmic tasks where the expected solution is known. A central challenge is that information being readable in a representation does not prove the model uses it, and findings can vary with architecture, task, and intervention, making causal validation and reliable evaluation essential.

Latest papers

CardsList

340 papers34 papers in the last four weeks, down 8% on the four weeks before. 0.2% of all new papers.

  1. From the Transformer to a Formal Semantics of Coordinated Transformations

    Oct 29, 2026Marco Giunti, Fabrizia Giulia GaravagliaTransformer Interpretability

    ALOPHIS – Applied Logic, Philosophy, and History of Science, University of Cagliari · Dept. of Pedagogy, Psychology, Philosophy, University of Cagliari

  2. INTR++: INterpretable TRansformer with better Accuracy, Interpretability, and Adaptability

    Oct 27, 2026Dipanjyoti Paul, Arpita Chowdhury, Maneet Chatterjee +4Transformer InterpretabilityExplainable Image Classification

    IIT Kharagpur · IIEST Shibpur · University of Tsukuba +2

  3. Causal Localization of Evidence Effects in Transformers

    Oct 27, 2026Jipeng Li, Pramod Khargonekar, Yanning ShenTransformer InterpretabilityAttention Head Analysis

  4. SAP-ViT: A Self-Attributing Patch Vision Transformer for Medical Image Classification

    Oct 26, 2026Wei Zhang, Wolfgang Nejdl, Ming TangExplainable Image ClassificationVision Transformer

    L3S Research Center, Leibniz University Hannover, 30167 Hannover, Germany

  5. Reader--Writer Coupling in Transformer MLPs

    Oct 21, 2026Giyeong Oh, Sunghee Ahn, Youngjae YuTransformer FFNsTransformer Interpretability

    Seoul National University

  6. Strength Controls Change How Interventions Compare Across Sparse Autoencoder Codes

    Oct 19, 2026Jinchen Zhang, Zheyuan Liu, Yihan Zhu +1Transformer InterpretabilitySparse Autoencoders

    University of Notre Dame

  7. Learning When to Abort in Transformer Planning: A Mechanistic Study

    Oct 18, 2026Shi Feng, Qimin Zhong, Siwei Wang +4Language Model Error DetectionTransformer Interpretability

    Harvard University · Shenzhen University · Microsoft Research Asia +1

  8. The Explainability Illusion: Must ViTs Exhibit Patch-Decoding Interpretability?

    Oct 18, 2026Ronen Schaffer, Hila Manor, Tomer MichaeliTransformer InterpretabilityExplainable Image Classification

    Faculty of Electrical and Computer Engineering, Technion – Israel Institute of Technology

  9. Comparing Sparsified MLPs and Transcoders: Many Transcoder Features Are Neurons in Disguise

    Oct 17, 2026Stepan Shabalin, Mateusz PiotrowskiTransformer InterpretabilityNeuron-Level Interpretability

    Anthropic · Georgia Institute of Technology

  10. What Accuracy Hides In Learned Transformer Programs

    Oct 16, 2026Jonathan Lee, Narmeen Oozeer, Philip QuirkeTransformer InterpretabilityAlgorithmic Reasoning

    Martian

  11. Spurious Attention Heads in Vision Transformers: Mechanistic Identification of Annotation-Driven Bias in Medical Chest X-Ray Analysis

    Oct 15, 2026Joshua Muhumuza, Cornelius KalenziVision TransformerAttention Head Analysis

    Makerere University

  12. Interpreting Language Model Parameters

    Oct 14, 2026Lucius Bushnaq, Dan Braun, Oliver Clive-Griffin +5Mechanistic InterpretabilityTransformer Interpretability

    Goodfire · MATS · Independent

  13. When Does Depth Help? Understanding Layerwise Inference in Tabular Foundation Models

    Oct 14, 2026Wenwen Gong, Haotian Gu, Zhen Yang +6Tabular Foundation ModelsTransformer Inference

    China Agricultural University · Tsinghua University · Nankai University

  14. Do Syntax Heads Influence Model Behavior?

    Oct 13, 2026William Timkey, Tal LinzenTransformer InterpretabilityAttention Head Specialization

    New York University

  15. Learning Source Structure: Bayesian Prediction with Transformers

    Oct 13, 2026Meir Feder, Ruediger UrbankeTransformerSequence Modeling

    School of Electrical and Computer Engineering, Tel Aviv University, Tel Aviv, Israel · School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland

  16. Gluing Local Contexts into Global Meaning: A Sheaf-Theoretic Decomposition of Transformer Representations

    Oct 12, 2026Bryce Grant, Peng WangTransformer InterpretabilityRepresentation Geometry in Language Models

    Case Western Reserve University

  17. SFT Leaves a Reproducible Signature in Activation Geometry: Calibrating Post-Training Interpretability Against Run-to-Run Noise

    Oct 12, 2026Xupeng Chen, Chenqian Le, Jiayi Cheng +2Language Model Post-TrainingLLM Training

    New York University · University of Southern California · Columbia University

  18. RIFT: Token Influence Follows the Path of Least Resistance

    Oct 12, 2026Tingan Jin, Shuhang Dong, Haosong LiTransformer InterpretabilityLLM Interpretability

    University of California, Los Angeles Los Angeles, CA, USA · Independent Researcher

  19. Harmonic Representations in Symbolic Music Transformers are Localized and Controllable

    Oct 11, 2026Adi Silberschein, Megan Wei, Tamar Rott ShahamTransformer InterpretabilityControllable Music Generation

    Weizmann Institute of Science · Brown University · MIT CSAIL

  20. We Have Been Interpreting Diffusion Transformer Backbone Behaviors Wrong

    Oct 10, 2026Benyuan Meng, Longtao Huang, Yang Liu +1Diffusion ModelsDiffusion Transformer

    1Sun Yat-sen University · 2Alibaba Group · 4X-Era AI Lab +1

  21. Exploring Attention Sinks in Text-to-Image Transformers from Diffusion and Autoregressive Models

    Oct 9, 2026Binglei Li, Mengping Yang, Zhiyu Tan +3Attention SinksTransformer Interpretability

    Fudan University · Shanghai Innovation Institute · Shanghai Academy of AI for Science

  22. Fully Interpretable Minimal Transformers: From Geometry to Algorithm

    Oct 7, 2026Raneem Mahajne, Toviah MoldwinTransformer InterpretabilityMechanistic Interpretability

    Edmond and Lily Safra Center for Brain Sciences, The Hebrew University of Jerusalem

  23. Spatial Induction Heads: In-Context Learning of Multidimensional Cellular Automata

    Oct 6, 2026Kimia Kazemian, Menghan Xu, John Thickstun +1In-Context LearningInduction Heads

    Cornell University

  24. The Standardization Trap: Certifying Joint Label Processing in Tabular Foundation Models

    Oct 6, 2026Duong Nguyen, Nicolas Chesneau, Milan BhanTabular Foundation ModelsTransformer Interpretability

    Ekimetrics

  25. Separators Make Carry Propagation Learnable:The Geometry of Latent Carry in a Multiplication Transformer

    Oct 5, 2026Sama Satariyan, Raphael Cousin, G{é}rard BiauNumerical Reasoning in Language ModelsTransformer Interpretability

    SCAI · LPSM,IUF,MEGAVOLT

  26. An equality condition for the Dobrushin bound on attention rollout and how often it holds in trained transformers

    Oct 4, 2026Przemysław RolaTransformer InterpretabilitySelf-Attention

    Department of Mathematics, Kraków University of Economics

  27. ScopeSAE: Model-Scope Feature Discovery with Interpretable Layer Selection

    Oct 4, 2026Qingwen Zeng, Zehao Fu, Shuyu Meng +5Transformer InterpretabilitySparse Autoencoders

    The University of Sydney · The Hong Kong University of Science and Technology (Guangzhou) · DeepWisdom +1

  28. Beyond the Linear Representation Hypothesis: Non-Linear Activation Steering in Text-to-Image Models

    Oct 3, 2026Muhammad Atif Butt, Paweł Skierś, Joost Van De Weijer +1Activation SteeringTransformer Interpretability

    Computer Vision Center, Spain · Computer Sciences Department, Universitat Aut`onoma de Barcelona, Spain · Warsaw University of Technology, Poland +1

  29. Metonymic Circuits for Abstract Concept Grounding in Vision Transformers

    Oct 3, 2026Jing Ding, Ziqiao Ma, Jiayuan Mao +2VLM InterpretabilityVision Transformer

    University of Michigan · Vector Institute · University of Pennsylvania +2

  30. Decoding the Functional Roles of Register and High-Norm Patch Tokens in Vision Transformers

    Oct 2, 2026Neel Varma, Andrew Rufail, Dipika Khullar +1Vision TransformerTransformer Interpretability