Transformer Interpretability

Overview

Transformer interpretability asks how models process information internally, from language and vision to speech, biology, and time series. Researchers use probes and representation geometry to find encoded features, sparse autoencoders to decompose activations, and circuit tracing, ablation, and activation patching to test causal roles; attribution methods explain inputs, while steering tests how internal signals can control outputs. Work also explores interpretable architectures that expose computation by design and controlled algorithmic tasks where the expected solution is known. A central challenge is that information being readable in a representation does not prove the model uses it, and findings can vary with architecture, task, and intervention, making causal validation and reliable evaluation essential.

Latest papers

CardsList

356 papers18 papers in the last four weeks, down 55% on the four weeks before. 0.1% of all new papers.

  1. Interpretable weight-space programming builds transparent transformers and brain encoding models

    Nov 13, 2026Yash Mehta, Chandan Singh, Weijia Xu +6Weight-Space ProgrammingAutomated Interpretability

  2. Stagewise and interpretable emergence of inside-outside computation in Transformer-based language models

    Nov 10, 2026Jack T. ParleyTransformer Training DynamicsTransformer Interpretability

  3. Sanity Checks for Attention Surgery: Null Models and Matched Interventions for Representation-Motivated Design

    Nov 10, 2026Basavaraj R M, Aryan Bharat KumarAttention MechanismsAttention Head Analysis

  4. Beyond Head Importance: Collective Computation in Video Transformers

    Nov 9, 2026Yi Yang, Siyuan Liu, Bingbing NieAttention Head AnalysisTransformer Interpretability

  5. Probing Transformer Representations Layer by Layer, Module by Module

    Nov 8, 2026Ambroise Odonnat, Theo Gnassounou, Laetitia Chapel +2Transformer InterpretabilityVision Foundation Models

  6. Can small transformers learn the language of Coxeter groups?

    Nov 7, 2026J. Elisenda Grigsby, Yiyao Huang, Arushi Katyal +3Transformer InterpretabilityTransformer

  7. Higher Order Component Attribution via Learned Surrogates in High Energy Physics

    Nov 5, 2026Lauri Laatu, Aritra Bal, Benedikt Maier +2Transformer InterpretabilityHigh-Energy Physics

  8. Playing Dumb: How a Chess Transformer Simulates Users Across Skill Levels

    Nov 2, 2026Difan Jiao, George Eilender, Zhenwei Tang +1Transformer Interpretability

  9. Align Once to Explain: Feature Alignment for Scalable B-cosification of Foundational Vision Transformers

    Nov 2, 2026Raphael Maser, Siddhartha Gairola, Sukrut Rao +1ViT ExplainabilityTransformer Interpretability

  10. Beliefs in feedback: Factored representations of interacting agents in transformers

    Nov 1, 2026David A. Ehrlich, Manuel Baltieri, Fernando E. Rosas +1Transformer InterpretabilityMulti-Agent Systems

  11. From the Transformer to a Formal Semantics of Coordinated Transformations

    Oct 29, 2026Marco Giunti, Fabrizia Giulia GaravagliaTransformer Interpretability

  12. INTR++: INterpretable TRansformer with better Accuracy, Interpretability, and Adaptability

    Oct 27, 2026Dipanjyoti Paul, Arpita Chowdhury, Maneet Chatterjee +4Transformer InterpretabilityExplainable Image Classification

  13. SAP-ViT: A Self-Attributing Patch Vision Transformer for Medical Image Classification

    Oct 26, 2026Wei Zhang, Wolfgang Nejdl, Ming TangExplainable Image ClassificationVision Transformer

  14. Reader--Writer Coupling in Transformer MLPs

    Oct 21, 2026Giyeong Oh, Sunghee Ahn, Youngjae YuTransformer FFNsTransformer Interpretability

  15. Learning When to Abort in Transformer Planning: A Mechanistic Study

    Oct 18, 2026Shi Feng, Qimin Zhong, Siwei Wang +4Language Model Error DetectionTransformer Interpretability

  16. The Explainability Illusion: Must ViTs Exhibit Patch-Decoding Interpretability?

    Oct 18, 2026Ronen Schaffer, Hila Manor, Tomer MichaeliTransformer InterpretabilityExplainable Image Classification

  17. Interpreting Language Model Parameters

    Oct 14, 2026Lucius Bushnaq, Dan Braun, Oliver Clive-Griffin +5Mechanistic InterpretabilityTransformer Interpretability

  18. When Does Depth Help? Understanding Layerwise Inference in Tabular Foundation Models

    Oct 14, 2026Wenwen Gong, Haotian Gu, Zhen Yang +6Tabular Foundation ModelsTransformer Inference

  19. Learning Source Structure: Bayesian Prediction with Transformers

    Oct 13, 2026Meir Feder, Ruediger UrbankeTransformerSequence Modeling

  20. SFT Leaves a Reproducible Signature in Activation Geometry: Calibrating Post-Training Interpretability Against Run-to-Run Noise

    Oct 12, 2026Xupeng Chen, Chenqian Le, Jiayi Cheng +2Language Model Post-TrainingLLM Training

  21. RIFT: Token Influence Follows the Path of Least Resistance

    Oct 12, 2026Tingan Jin, Shuhang Dong, Haosong LiTransformer InterpretabilityLLM Interpretability