Transformer Interpretability

Follow this topic Ask about this topic

Momentum

28 papers in the last four weeks, up 180% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 224

All topics

Fully Interpretable Minimal Transformers: From Geometry to Algorithm
Oct 7, 2026Raneem Mahajne, Toviah MoldwinTransformer Interpretability Mechanistic Interpretability
Edmond and Lily Safra Center for Brain Sciences, The Hebrew University of Jerusalem
Spatial Induction Heads: In-Context Learning of Multidimensional Cellular Automata
Oct 6, 2026Kimia Kazemian, Menghan Xu, John Thickstun +1In-Context Learning Induction Heads
Cornell University
The Standardization Trap: Certifying Joint Label Processing in Tabular Foundation Models
Oct 6, 2026Duong Nguyen, Nicolas Chesneau, Milan BhanTabular Foundation Models Transformer Interpretability
Ekimetrics
Separators Make Carry Propagation Learnable:The Geometry of Latent Carry in a Multiplication Transformer
Oct 5, 2026Sama Satariyan, Raphael Cousin, G{é}rard BiauNumerical Reasoning in Language Models Transformer Interpretability
SCAI · LPSM,IUF,MEGAVOLT
An equality condition for the Dobrushin bound on attention rollout and how often it holds in trained transformers
Oct 4, 2026Przemysław RolaTransformer Interpretability Self-Attention
Department of Mathematics, Kraków University of Economics
A Comparative Explainability Framework for DeBERTa-v3 in Zero-Shot Medical Abstract Classification
Oct 1, 2026Javier Diaz Esteban-Herreros, David Muñoz-Valero, Raquel Martínez-España +2Transformer Interpretability Perturbation-Based Feature Attribution
Department of Technologies and Information Systems, Universidad de Castilla-La Mancha, Avenida Carlos III, s/n, Toledo, 45071, Spain · University of Murcia, Avenida Teniente Flomesta, 5, Murcia, 3003, Spain · Murcian Bio-Health Institute (IMIB-Arrixaca), Pabellón Docente del Hospital Clínico Universitario Virgen de la Arrixaca, Murcia, 3120, Spain
Capturing In-Context Learning Dynamics with Task Operators
Oct 1, 2026Guangzhi Xiong, Zhenghao He, Bohan Liu +3Transformer Interpretability In-Context Learning
University of Virginia
Shared Weights, Selected Computations: How Looped Transformers Route What Each Loop Does
Sep 30, 2026Jiaju Wu, Yi Hu, Muhan ZhangTransformer Interpretability Transformer Attention
Institute for Artificial Intelligence, Peking University. · School of Mathematical Sciences, Peking University.
D-Scope: Decomposing and Steering Diffusion Transformers with Sparse Autoencoders
Sep 30, 2026Xinyue Xu, Jiahao Zhang, Lijie Hu +2Transformer Interpretability Diffusion Transformer
Pivotal Research · Mohamed bin Zayed University of Artificial Intelligence · Schmidt Sciences +2
Concept Subspaces Compute Beyond the Logit Lens: A Weights-Only Test for Locating Representations Upstream of Readout
Sep 30, 2026Aojie Yuan, Zhiyuan Julian Su, Haiyue Zhang +1Transformer Interpretability Representation Geometry in Language Models
University of Southern California · Duke University · University of Michigan
The Geometry of Inference in Transformer Residual Streams
Sep 29, 2026Timur Mudarisov, Mikhail Burtsev, Radu StateTransformer Interpretability Transformer Inference
University of Luxembourg, Luxembourg · London Institute for Mathematical Sciences, London, UK
Which the Eye Fears: Writing with Read-Blindness Explains Massive Activations in Transformers
Sep 28, 2026Swagatam Mukhopadhyay, Vishal Vivek Saley, Vraj Parikh +1Transformer Interpretability Transformer FFNs
PsiDagger · IIT Delhi
Understanding Confabulation and Rethinking Reconstruction in Activation Explanations
Sep 27, 2026Gert Lek, Zixuan Xia, Pin-Yu Chen +1Transformer Interpretability Faithfulness of Language Model Explanations
Universit´e de Neuchˆatel · Universität Bern · IBM Research +1
Comparing Latent Concept Formation in State Space Models and Transformers via Sparse Autoencoders
Sep 21, 2026Rithin Nagaraj, Rupa Laalasa Oruganti, Prerna Subhashchandra Kunder +1Transformer Interpretability Language Modeling
Department of Computer Science and Engineering PES University Bangalore, India
Prescriptive SVD-Inspired Attention via Spectral Energy Retention
Sep 21, 2026Vasileios Arampatzakis, Vasileios Sevetlidis, George PavlidisTransformer Interpretability Low-Rank Attention
Athena Research Center, Greece
Increasing Skill Level Recruits Deeper Attention Layers in a Frozen Chess Transformer
Sep 20, 2026David LitmanTransformer Interpretability Attention Head Analysis
Computational Neurobiology Laboratory, Salk Institute
Deep Noir: Autonomous Steering Discovery via Architectural Chronometry in Transformer Models
Sep 17, 2026Frank E. Bobe, Gregory D. Vetaw, Darshan W. Bryner +2Transformer Interpretability Language Model Steering
Naval Surface Warfare Center Panama City Division
Generalization through Lexical Abstraction in Transformer Models: The Case of Functional Words
Sep 17, 2026Giuseppe Samo, Vivi Nastase, Paola MerloTransformer Interpretability Word Embeddings
Idiap Research Institute, Martigny, Switzerland · University of Geneva, Swizerland
Weakening Neurons: An Input-Output Functionality in Transformers with Outsize Influence
Sep 16, 2026Sebastian Gerstner, Hilal AlQuabeh, Kentaro Inui +1Transformer Interpretability Transformer FFNs
LMU Munich and Munich Center for Machine Learning · MBZUAI · Tohoku University +1
ResLRP: The Role of Residual Cancellation in Attribution Instability in Vision Transformers
Sep 15, 2026Jim Berend, Reduan Achtibat, Daniel Schäffer +4Transformer Interpretability Residual Learning
Fraunhofer Heinrich Hertz Institute · DSC ScaDS.AI, Leipzig University · ICT Cluster, Singapore Institute of Technology, Singapore +3
What Does Layer-Importance Reveal About Transformers and State-Space Models?
Sep 15, 2026Istabrak Abbes, Nizar Islah, Irina Rish +1Transformer Interpretability Selective SSMs
Chandar Research Lab · Mila - Quebec AI Institute · Université de Montréal +2
Where Decoder Cosine Similarity Fails for SAE Feature Flow Discovery
Sep 14, 2026Hendrik Droste, Christian Medeiros Adriano, Kathrin Korte +1Transformer Interpretability Sparse Autoencoders
Hasso Plattner Institute, Germany · IT University of Copenhagen, Denmark
Sparse Weight and Edge Circuit Discovery in Transformer-based Acoustic Models
Sep 11, 2026Jiankun Wei, Ewan Dunbar, Gerald PennTransformer Interpretability Circuit Discovery
Department of Computer Science University of Toronto Toronto, Canada
Analyzing Traditional and Neural Approaches to Multilingual Readability Assessment
Sep 11, 2026Joshua Wong, Chris TannerTransformer Interpretability Feature Attribution
Harvard University · Massachusetts Institute of Technology · Kensho Technologies in Cambridge, MA, USA
Quantifying Logical Consistency in Transformers via Query-Key Alignment
Sep 10, 2026Eduard Tulchinskii, Anastasia Voznyuk, Laida Kushnareva +4Transformer Interpretability LLM Evaluation
Skolkovo Institute of Science and Technology · AI Foundation and Algorithm Lab · Moscow Institute of Physics and Technology +2
Training Trajectories Determine Circuit Removability in Annealable Soft-Prior Transformers
Sep 9, 2026Zonglin Yang, Ziming Zhao, Wei Tang +5Transformer Interpretability Attention Mechanisms
Criminal Science and Technology, Guangdong Police College, China · Cybersecurity and Law Enforcement, Guangdong Police College, China · Traffic Management Engineering, Guangdong Police College, China
Through the Looking Glass: Directly Reading and Writing Transformers
Sep 9, 2026Mark OskinKnowledge Editing Transformer Interpretability
School of Computer Science and Engineering University of Washington
Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses
Sep 9, 2026Olli TuomiTransformer Interpretability Language Model Introspection
Evident Solutions Oy
"World Knowledge" in the Weights: Reading Concept Circuits of Vision Transformers
Sep 8, 2026Yanlin Chen, Tang Li, Xi PengTransformer Interpretability Vision Foundation Models
Department of Computer & Information Sciences, University of Delaware, USA · Department of Computer Science, University of Virginia, USA
LLM Layers Immediately Correct Each Other
Sep 7, 2026Arjun Patrawala, Jiahai Feng, Erik Jones +1Transformer Interpretability Representation Geometry in Language Models
University of California, Berkeley

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

Fully Interpretable Minimal Transformers: From Geometry to Algorithm

Raneem Mahajne, Toviah Moldwin

Edmond and Lily Safra Center for Brain Sciences, The Hebrew University of Jerusalem

Transformer Interpretability Mechanistic Interpretability Representation Geometry in Language Models Transformer Attention

Open paper AskPDF arXiv

We present a framework for building and interpreting minimal transformer models. By constraining a transformer's embedding dimension and head size to 2, we enable full two-dimensional visualization of its internal representations. Embeddings, query/key/value transforms, attention outputs, residual streams, and decision boundaries can all be seen directly. Our central claim is that the learned geometry implies an algorithm; the arrangement of points and boundaries in R^2 can be read as a step-by-step procedure. We train a transformer on a simple task where it must produce the most recently observed even number whenever the '+' operator appears in a sequence of digits. Once trained, we visually walk through every step of the transformer's computation. We show how the model embeds the tokens and their respective positions in the sequence, transforms them via the Q, K, and V matrices, uses the dot product between the Q and K representations to form the attention matrix, and uses the attention matrix to select values that move the representation of each input token to the region of the domain of the output layer that will correctly predict the next token. We introduce a suite of interpretability visualizations that make the algorithmic interpretation of this procedure explicit. Our framework offers a pedagogical and experimental testbed to explore how transformers use informational geometry to implement next-token prediction.

Momentum

Related topics

Latest papers 224