Transformer Interpretability

Follow this topic Ask about this topic

Momentum

28 papers in the last four weeks, up 180% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 224

All topics

Hierarchical Latent Structures in Data Generation Process Unify Mechanistic Phenomena across Scale
Feb 4, 2026Jonas Rohweder, Subhabrata Dutta, Iryna GurevychTransformer Interpretability Latent Variable Models
UKP Lab, TU Darmstadt
Universal Redundancies in Time Series Foundation Models
Feb 2, 2026Anthony Bao, Venkata Hasith Vattikuti, Jeffrey Lai +1Transformer Interpretability Kernel Regression
ECE Department, UT Austin · Department of Physics, UT Austin · Oden Institute, UT Austin
The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Models
Jan 9, 2026Nora Graichen, Iria de-Dios-Flores, Gemma BoledaTransformer Interpretability LLM Evaluation
Universitat Pompeu Fabra1 · ICREA2
Probing Spectrum-Like Organization of States of Mind in Transformer Representation Spaces
Dec 23, 2025Sophie ZhaoTransformer Interpretability Neural Representation Geometry
School of Computer Science Georgia Institute of Technology
Equivalence of Context and Parameter Updates in Modern Transformer Blocks
Nov 22, 2025Adrian Goldwaser, Michael Munn, Javier Gonzalvo +1Transformer Interpretability Transformer FFNs
University of Cambridge, UK · Google Research
Higher Embedding Dimension Creates a Stronger World Model for a Simple Sorting Task
Oct 21, 2025Brady Bhalla, Honglu Fan, Nancy Chen +1Transformer Interpretability Representation Learning
California Institute of Technology · Pasadena, CA 91125 · Google DeepMind +3
Deductive Logic in Language Models: Horizontal vs Vertical Reasoning
Oct 10, 2025Davide Maltoni, Matteo FerraraTransformer Interpretability CoT Reasoning
Department of Computer Science and Engineering, University of Bologna, Italy
Unveiling the Mechanisms of Multi-Hop Reasoning in Transformers via Identity Bridge
Sep 29, 2025Pengxiao Lin, Zheng-An Chen, Zhi-Qin John XuTransformer Interpretability OOD Generalization
School of Mathematical Sciences, Shanghai Jiao Tong University · Institute of Natural Sciences, MOE-LSC, Shanghai Jiao Tong University · Shanghai Seres Information Technology Co., Ltd, Shanghai 200040, China.
Cross-Attention is Half Explanation in Speech-to-Text Models
Sep 22, 2025Sara Papi, Dennis Fucci, Marco Gaido +2Transformer Interpretability Cross-Attention
Fondazione Bruno Kessler, Italy
REAL: Reading Out Transformer Activations for Precise Localization in Language Model Steering
Jun 10, 2025Li-Ming Zhan, Bo Liu, Chengqiang Xie +2Transformer Interpretability Language Model Steering
Department of Data Science and Artificial Intelligence The Hong Kong Polytechnic University Hong Kong S.A.R.
LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers
Feb 20, 2025Anton Razzhigaev, Matvey Mikhalchuk, Temurbek Rahmatullaev +4Transformer Interpretability Long-Context Language Modeling
AIRI · Skoltech · Lomonosov Moscow State University +1
A Mechanistic Study of Transformers Training Dynamics
Oct 31, 2024Ambroise Odonnat, Wassim Bouaziz, Vivien CabannesTransformer Interpretability Attention Mechanisms
Inria, Univ. Rennes 2 · Mistral AI; work done while at Meta. · FAIR at Meta.
Debunking Grad-ECLIP: A Comprehensive Study on Its Incorrectness and Fundamental Principles for Model Interpretation
Date pendingYongjin Cui, Xiaohui FanGradient-Based Attribution Transformer Interpretability
Zhejiang University
Tracing Computation Density in LLMs
Date pendingCorentin Kervadec, Iuliia Lysova, Iuri Macocco +2Transformer Interpretability LLM Interpretability
Universitat Pompeu Fabra · ICREA

Feb 4, 2026, cs.CLJ/K move · Enter open · S save

Hierarchical Latent Structures in Data Generation Process Unify Mechanistic Phenomena across Scale

Jonas Rohweder, Subhabrata Dutta, Iryna Gurevych

UKP Lab, TU Darmstadt

Transformer Interpretability Latent Variable Models Induction Heads

Open paper AskPDF arXiv

Contemporary studies in mechanistic interpretability have uncovered many puzzling phenomena in the neural information processing of Transformer-based language models, such as induction heads, function vectors, and the Hydra effect. Some of these individual phenomena have been independently tied to different data distributional properties, while some have been loosely associated with model architecture and how Transformers process information. However, a unified understanding of the relationship between data, model architecture, and optimization remains lacking, failing to answer the fundamental question: why do these three phenomena appear universally across different model families and scales, despite their seeming disconnect? In this work, we answer this question by unifying these three phenomena as consequences of hierarchical latent structures in the data generation process, coupled with decorrelated gradients across additive model components and directional concavity in the representation geometry. We validate our theoretical results in a toy model regime and in a large-scale synthetic data regime, comparing them with language models trained on natural language data.

Momentum

Related topics

Latest papers 224