Transformer Interpretability

Follow this topic Ask about this topic

Momentum

28 papers in the last four weeks, up 180% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 224

All topics

IG-Lens: Exact Additive Probability Attribution Across Transformer Layers via Telescoping Integrated Gradients
Jun 29, 2026Duc Anh NguyenGradient-Based Attribution Integrated Gradients
Hanoi University of Science and Technology
How Token Influence Decays with Distance: A Green-Function View of Trained Language Models
Jun 28, 2026Matthias Brändel, Stephan Köhler, Oliver RheinbachTransformer Interpretability Autoregressive Language Modeling
Faculty of Mathematics and Computer Science Technische Universität Bergakademie Freiberg, Freiberg, Germany
From Black-Box to Clinical Insight: A Multi-Stage Explainable Framework for Speech-Based Cognitive Impairment Detection
Jun 26, 2026Yasaman Haghbin, Sina Rashidi, Ali Zolnour +6Transformer Interpretability Explainable Artificial Intelligence
Independent Researcher · Columbia University, United States · Chalmers University of Technology, Sweden
At the Edge of Understanding: Sparse Autoencoders Trace The Limits of Transformer Generalization
Jun 24, 2026Praneet Suresh, Jack Stanley, Sonia Joseph +2Distribution Shift Robustness Transformer Interpretability
Mila Quebec AI Institute · Meta AI · Sapient Intelligence
Targeted Recovery of Weight-Space Mechanisms From Neural Networks
Jun 19, 2026Antoine Vigouroux, Lee SharkeyTransformer Interpretability Circuit Discovery
1MATS · 2Goodfire
LIG: Layer-wise Integrated Gradients for Within-Layer Flow Analysis in Transformers
Jun 19, 2026Eight Suzuki, Hideitsu Hino, Noboru MurataGradient-Based Attribution Transformer Interpretability
Waseda University · Fujitsu Limited · The Institute of Statistical Mathematics
Explaining Attention with Program Synthesis
Jun 17, 2026Amiri Hayes, Belinda Z Li, Jacob AndreasTransformer Interpretability Attention Head Analysis
NJIT · MIT EECS
Closing the Loop: PID Feedback Control for Interpretable Activation Steering in Symbolic Music Generation
Jun 17, 2026Ioannis Prokopiou, Pantelis Vikatos, Maximos Kaliakatsos-Papakostas +2Transformer Interpretability Controllable Music Generation
Athens University of Economics and Business, Athens, Greece · Orfium Research, Athens, Greece · Hellenic Mediterranean University, Chania, Greece +2
The Discrete-Log Clock: How a Transformer Learns Modular Multiplication
Jun 16, 2026Huu Danh NguyenTransformer Interpretability Frequency-Domain Feature Learning
Stanford University
When Does Depth Matter For In-Context Learning? Adaptive Inference in Deep Transformers
Jun 15, 2026Ravin Raj, Gautam ReddyTransformer Interpretability Adaptive Inference
Joseph Henry Laboratories of Physics, Princeton University, Princeton, New Jersey 08544, USA
Transformers Learn the Mestre-Nagao Heuristic
Jun 13, 2026Pranav Venkata KondaTransformer Interpretability Transformer
Listening with Attention: Entropy-Guided Explainability for Transformer-Based Audio Models
Jun 12, 2026Ravi Ranjan, Utkarsh Grover, Xiaomin Lin +1Transformer Interpretability Explainable Artificial Intelligence
Florida International University · Miami, USA · University of South Florida +1
Decompose Sparsely Where You Should, Absorb Densely Where You Should No
Jun 12, 2026Ruixuan Deng, Zehao Jin, Zekun Wang +1Transformer Interpretability Representation Learning
School of Psychology Georgia Institute of Technology · College of Computing Georgia Institute of Technology · School of Interactive Computing Georgia Institute of Technology
Explaining RhythmFormer: A Systematic XAI Analysis of Periodic Sparse Attention for Remote Photoplethysmography
Jun 11, 2026Louis Chen, Torbjörn E. M. NordlingRemote Photoplethysmography Transformer Interpretability
Department of Mechanical Engineering, National Cheng Kung University, Tainan, Taiwan.
Layer-Resolved Optimal Transport for Hallucination Detection in NMT and Abstractive Summarization
Jun 11, 2026Mariia Onyshchuk, Maksym-Vasyl Tarnavskyi, Marta SumykTransformer Interpretability Hallucination Detection
Faculty of Applied Sciences, Ukrainian Catholic University, Lviv, Ukraine.
When Does Routing Become Interpretable? Causal Probes on Block Attention Residuals
Jun 11, 2026Aydin JavadovTransformer Interpretability Attention Residual Connections
ETH Zurich
The Signs Were Always There: Training-Free Concept Detection and Steering in Raw Transformer Dimensions
Jun 10, 2026Varun Reddy NalagatlaTransformer Interpretability Representation Engineering
Amazon Web Services
Vision Transformers for Face Recognition Need More Registers
Jun 10, 2026Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira +2Transformer Interpretability Transformer
Fraunhofer Institute for Computer Graphics Research IGD, Germany · Department of Computer Science, TU Darmstadt, Germany
Closure-Validated Circuit Discovery in Attention Heads: Co-activation Proposes, Ablation Disposes
Jun 8, 2026Yongzhong XuTransformer Interpretability Circuit Discovery
Integrating gene regulatory priors into Transformer attention with scTransformer for interpretable scRNA-seq analysis
Jun 8, 2026Mikele Milia, Louis Fabrice Tshimanga, Henning Mueller +2Transformer Interpretability Transformer Attention
Department of Information Engineering, University of Padova, Padova, Italy · Department of Neuroscience, University of Padova, Padova, Italy · Padova Neuroscience Center, Padova, Italy +3
Trajectory Geometry of Transformer Representations Across Layers
Jun 8, 2026Vishal Pandey, Gopal Singh, Yacine MahdidTransformer Interpretability Neural Representation Geometry
London, UK · Athens, GR · Montreal, CA
A Unifying View of Attention Sinks: From Mechanisms to Architectural Interventions
Jun 6, 2026Lukas Fesser*, Mozes Jacobs*, Thomas Fel* +2Transformer Interpretability Vision Transformer
Kempner Institute Harvard University Cambridge, MA
Inside the Visual Mind: Neuroscience-Motivated Concept Circuits for Interpreting and Steering Vision Transformers
Jun 4, 2026Tang Li, Yanlin Chen, Mengmeng Ma +1Transformer Interpretability Vision Transformer
Department of Computer & Information Science, University of Delaware, Newark, DE, USA · Department of Computer Science, University of Virginia, Charlottesville, VA, USA
Multiscale POD of Transformer Attention Fields: Scale-Selective Analysis via Morlet Scalogram
Jun 4, 2026Athanasios ZerisTransformer Interpretability Proper Orthogonal Decomposition
Independent Researcher, Athens, Greece.
Where does Absolute Position come from in decoder-only Transformers?
Jun 4, 2026Valeria Ruscio, Umberto Nanni, Fabrizio SilvestriRotary Positional Embeddings Transformer Interpretability
Intuition Machines · Sapienza University of Rome
Causal Evidence of Stack Representations in Modeling Counter Languages Using Transformers
Jun 2, 2026Nishit SinghTransformer Interpretability Transformer
Birla Institute of Technology and Science, Pilani
When Do Attention Circuits Form? Developmental Trajectories of Capability and Attention-Sink Emergence Across Three 1B-ClassArchitectures
Jun 1, 2026Yongzhong XuLanguage Model Pretraining Transformer Interpretability
EIVE: End-to-End Instance-Specific Visual Explanations for Detection Transformers
Jun 1, 2026Jianlin Xiang, Yanshan Li, Linhui DaiTransformer Interpretability Feature Attribution
Institute of Intelligent Information Processing, Shenzhen University; the Guangdong Provincial Key Laboratory of Intelligent Information Processing, Shenzhen University; and the Shenzhen Key Laboratory of Modern Communications and Information Processing, Shenzhen University, Shenzhen, China
Refit the Probe: Single-Direction Ablation Is Not a Necessity Test
May 30, 2026Yuhang JiangTransformer Interpretability Linear Probing
Independent Researcher
Assign and Add: A Mechanistic Study of Compositional Arithmetic
May 29, 2026Brady Exoo, Alberto Bietti, John SousTransformer Interpretability Transformer
Yale University · Flatiron Institute

Jun 29, 2026, cs.LGJ/K move · Enter open · S save

IG-Lens: Exact Additive Probability Attribution Across Transformer Layers via Telescoping Integrated Gradients

Duc Anh Nguyen

Hanoi University of Science and Technology

Gradient-Based Attribution Integrated Gradients Transformer Interpretability Decoder-Only Language Models Layer-Wise Relevance Propagation

Open paper AskPDF arXiv

We ask a simple question about decoder-only transformers: between which two layers is the probability of a predicted token actually produced? Existing layer-wise readout tools answer only approximately. The logit lens and its trained variant report a per-layer level of probability but give no additive decomposition; their estimates are biased and non-monotone across depth. Direct Logit Attribution and related residual-stream methods are additive, but only in logit space, the softmax nonlinearity breaks additivity in probability space, precisely the quantity one usually cares about. Layer Conductance integrates gradients per layer, but attributes each to its own baseline and so does not sum to the total change in prediction. We introduce IG-Lens, a telescoping application of Integrated Gradients along a single path through the hidden states from a baseline to the final layer. Crediting each segment to the layer it terminates at yields a layer-wise attribution whose sum is exactly the change in target probability, with the softmax inside the integration path rather than linearized away. Our default estimator credits each integration step its observed change in target probability (a prediction-aware reweighting in the spirit of IDGI) rather than its raw gradient. Because the readout is a one-dimensional probability, this collapses each segment to a telescoping sum of endpoint values, so completeness holds exactly (to floating point) at any step count, removing Riemann discretization error while suppressing steps that show gradient sensitivity without a change in output. We give the telescoping identity and its proof, verify completeness to floating point, and describe a single-pass batched implementation computing the full token-by-layer map without any backward call. Code: https://github.com/anhnda/IGLens.

Momentum

Related topics

Latest papers 224