Transformer Interpretability

Follow this topic Ask about this topic

Momentum

28 papers in the last four weeks, up 180% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 224

All topics

Unraveling the Real Working Mechanism and Inherent Flaws of GAE: A Method for Interpreting Transformer Processes from an Economic Perspective
Sep 7, 2026Yongjin Cui, Xiaohui FanTransformer Interpretability Explainable Artificial Intelligence
Zhejiang University
Encoded Early, Used Late: Where Transformers Begin to Act on an Inferred Partner's Expertise
Sep 7, 2026Mika Okamoto, Gabriele SartiTransformer Interpretability Language Model Probing
Georgia Institute of Technology · Northeastern University
Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens
Sep 1, 2026Matteo He, William F. Shen, Xinchi Qiu +1Transformer Interpretability LLM Interpretability
Department of Computer Science and Technology, University of Cambridge
Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?
Sep 1, 2026Wenlong Wang, Fergal ReidTransformer Interpretability Cognitive Modeling
Fin AI Research
HiLRP: Toward One Trustworthy Explanation for Vision Transformer: Conservation-Valid Attribution via Attention Primitives
Sep 1, 2026Sathiyamohan Nishankar, Pubudu Sanjeewani, Asanka Perera +1Transformer Interpretability Vision Transformer
Faculty of Engineering, University of Peradeniya, Sri Lanka · School of Computing Technologies, RMIT University, Melbourne, Australia · School of Engineering & Digital Technologies, University of Southern Queensland, Brisbane, Australia +1
Late Transformer Layers Recode Syntax Canonically: Evidence from Greek Scrambling and Cross-Layer Generalisation
Aug 31, 2026Christos Nikolaos Zacharopoulos, Revekka Kyriakoglou, Chara Tsoukala +1Transformer Interpretability Language Modeling
Independent Researcher · Université Paris 8 Vincennes–Saint-Denis Paris, France · Athena Research Center, Athens, Greece +1
Tracing distinguishability through transformer processing with stochastic LayerNorm
Aug 31, 2026Kieran MurphyTransformer Interpretability Layer Normalization
New Jersey Institute of Technology
RACE: Scalable Statistical Estimation of Functional Consistency in LLM Neurons
Aug 25, 2026Runyu Wang, Bo Liu, Xiaxin Zhang +6Transformer Interpretability LLM Interpretability
School of Transportation and Civil Engineering, Nantong University · Chongqing University of Post and Telecommunications · China Southern Power Grid Company Limited +1
Perturbation-based Regional Interpretability through Subtraction Mapping (PRISM): naming-error dissociations in language models and post-stroke aphasia
Aug 13, 2026Xiang Guan, Roger D. Newman-Norlund, Yong Yang +8Transformer Interpretability LLM Interpretability
Department of Computer Science and Engineering, University of South Carolina · Department of Communication Sciences and Disorders, University of South Carolina · ALLT.AI, LLC, Columbia, SC +3
Class Activation Mapping in Explainable Computer Vision: A Method-Centered Review of CNN, Transformer, and Foundation-Model-Era Visual Explanations
Aug 12, 2026AmirHossein Eshghi, Hamid Saadatfar, Seyyed Ali Hoseini +2Transformer Interpretability Feature Attribution
Department of Computer Engineering, University of Birjand, Iran · Department of Applied Mathematics and Computer Science Visual Computing, Technical University of Denmark, Denmark
Geometric and Behavioral Stratification in Transformer Residual Streams
Aug 12, 2026Nelson GudaTransformer Interpretability Transformer
Off-Axis, On Purpose: Where a Transformer Computes Concepts and Why it Does So
Aug 10, 2026Mark OskinTransformer Interpretability Transformer Attention
School of Computer Science and Engineering University of Washington
Spectral Outliers Reveal Dominant Learned Structure in Transformer Attention
Aug 8, 2026Kasun Dewage, Marianna Pensky, Suranadi De Silva +1Transformer Interpretability Transformer Attention
University of Central Florida Orlando, Florida, USA · Virtusa
Why Knowing Both Hops Is Not Enough: Understanding Two-Hop Generalization in Language Models
Aug 7, 2026Zili Zhang, Yilin Wang, Heng Wang +2Transformer Interpretability OOD Generalization
1Xi’an Jiaotong University · University of Illinois Urbana-Champaign
Transformer Geometry Observatory TGO-IV: Developmental Topology Observatory
Aug 7, 2026Kaustubh Kapil, Kishor P. UplaTransformer Interpretability Neural Representation Geometry
Neuromorphic Intelligence Research Collective · Department of Electronics Engineering · Sardar Vallabhai National Institute of Technology (SVNIT), Surat, India
Beyond Attention: Signed Integrated Gradients Attribution in a BiomeGPT-Style Microbiome Transformer
Aug 6, 2026Oren NelsonGradient-Based Attribution Integrated Gradients
Department of Computer Science University of California, San Diego San Diego, CA 92122
IRIS: A Visual Cortex-Inspired Framework for Analyzing Orientation Selectivity in Vision Transformers
Aug 5, 2026Vaishnavi B Mohan, Vijayakrishna Naganoor, Yashas Annadani +1Transformer Interpretability Visual Representation Learning
University of Washington, USA · Microsoft, USA · TU Munich, Germany +1
Grad-CAM for Vision Transformers: A Systematic Taxonomy and Audit of Methodological Ambiguity in Explainable AI
Aug 5, 2026Casey Wall, Longwei Wang, Rodrigue Rizk +1Gradient-Based Attribution Transformer Interpretability
USD Artificial Intelligence Research, Department of Computer Science, University of South Dakota
Sparse Weight Decomposition for Efficient Circuit Extraction
Aug 4, 2026Chuanhao Yan, Xuhan Huang, Yawen Duan +4Transformer Interpretability Mechanistic Interpretability
IQuest Research · Safe AI Forum · University of Oxford +2
Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Model
Aug 4, 2026Abdallah KhemaisTransformer Interpretability Weight-Space Analysis
ISITCOM, University of Sousse
A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation
Aug 4, 2026Abdallah KhemaisTransformer Interpretability Weight-Space Analysis
ISITCOM, University of Sousse
Probing Character-level Transformers for the Spanish L-shaped Morphome
Aug 4, 2026Akhilesh Kakolu Ramarao, Kevin Tang, Wiebke Petersen +1Transformer Interpretability Transformer
Department of English Language and Linguistics, Heinrich Heine University Düsseldorf · Department of Linguistics, College of Liberal Arts and Sciences, University of Florida · Institute of Linguistics and Information Science, Heinrich Heine University Düsseldorf +1
Does Explainability Transfer? A Controlled Benchmark of Attribution Methods on Vision Transformers and CNNs
Aug 3, 2026Sathiyamohan Nishankar, Nethmi Pathirana, Pubudu Sanjeewani +2Transformer Interpretability Feature Attribution
aFaculty of Computing, Sabaragamuwa University of Sri Lanka, Sri Lanka · bFaculty of Engineering, University of Moratuwa, Katubedda, Sri Lanka · cSchool of Computing Technologies, RMIT University, Melbourne, Australia +3
Feed-Forward Steering in Transformer Residual Dynamics
Aug 3, 2026Timur Mudarisov, Mikhail Burtsev, Radu StateTransformer Interpretability Transformer FFNs
University of Luxembourg · 2London Institute of Mathematical Sciences
Transformer Geometry Observatory TGO-III: Semantic Geometry Observatory
Aug 3, 2026Kaustubh Kapil, Kishor P. UplaTransformer Interpretability Neural Representation Geometry
Department of Electronics Engineering Sardar Vallabhai National Institute of Technology (SVNIT), Surat, India
Divergent large language model predictions from convergent representations in ambiguous word pairs
Aug 3, 2026K. Jack Scott, Narun Pat, Veronica LiesaputraTransformer Interpretability Decoder-Only Language Models
Department of Psychology, University of Otago, Dunedin, New Zealand · School of Computing, University of Otago, Dunedin, New Zealand
Interpretability-Guided Soft Pruning of Attention Heads in Vision Transformers
Jul 31, 2026Kamil Książek, Piotr Suszyński, Michał Jan Włodarczyk +2Transformer Interpretability Attention Head Analysis
Centre for Credible Artificial Intelligence Warsaw University of Technology Warsaw, Poland · Faculty of Mathematics and Computer Science Jagiellonian University Krakow, Poland · University of Warsaw Warsaw, Poland
HealthCAT: An Interpretable Encoder-only Transformer Framework for Health Indicator Prediction and Temporal Interpretation of Wearable Sensor Data
Jul 30, 2026Xiaotong Yu, Joshua Y. Kim, HaeJin Lee +1Transformer Interpretability Explainable Artificial Intelligence
Emergent Latent-State Computation under Stochastic Volatility
Jul 28, 2026Xiaoyu Huang, Lulu WangTransformer Interpretability Mechanistic Interpretability
Department of Mathematics Temple University Philadelphia, PA, USA · Department of Data Analytics Dickinson College Carlisle, PA, USA
Interpretable GOHR Agents via Sparse Autoencoders
Jul 27, 2026Shiwei Tan, Yusong Zhao, Weiyi Qin +6Transformer Interpretability Sparse Autoencoders
1Rutgers University · 3Paul B Kantor Consultant · University of Illinois Urbana-Champaign

Sep 7, 2026, cs.AIJ/K move · Enter open · S save

Unraveling the Real Working Mechanism and Inherent Flaws of GAE: A Method for Interpreting Transformer Processes from an Economic Perspective

Yongjin Cui, Xiaohui Fan

Zhejiang University

Transformer Interpretability Explainable Artificial Intelligence Explainability Evaluation

Open paper AskPDF arXiv

We observe a phenomenon that current algorithmic research in the field of explainable artificial intelligence primarily pursues better performance on several proxy metrics. On the one hand, these proxy metrics themselves are more or less flawed and cannot properly measure the quality of methods. On the other hand, metric-oriented research approaches often lead to the neglect of the rationality and interpretability of the methods themselves. Explainable artificial intelligence is abbreviated as XAI. The metric-driven research paradigm has resulted in a lack of interpretability of the relevant XAI methods themselves. Accordingly, there is a need for interpretability research on XAI methods, which can be playfully referred to as XXAI. This paper is one of our works on XXAI. This paper takes Generic Attention-model Explainability (GAE), a widely influential model interpretation method , or rather, XAI method that represents an important technical route, as the research object, and explores the real working mechanism and flaws of this method as well as the technical route it represents. Based on the conclusions of this study, it may be necessary to re-examine or verify GAE-related methods and their domain applications. We argue that GAE is an interpretation method that focuses on the attention process. After pointing out the working mechanism and flaws of GAE, we propose Cumulative Asset Holdings (CAH), a more reasonable Transformer interpretation method integrating both process-based and feature-based ideas from an economic zero-sum games perspective. In addition, it is worth noting that our method is applicable to models with special tokens, where existing methods may suffer from limitations. The model simplification research method and the analysis of additive operations adopted in this study may provide inspiration for other research works in XAI.

Momentum

Related topics

Latest papers 224