Attention Head Analysis

Follow this topic Ask about this topic

The field investigates the roles, interactions, and stability of attention heads, including their contributions to induction, retrieval, reasoning, and multimodal tasks. Common approaches include head ablation and activation patching to test causal effects, alongside probes and spectral or geometric analyses to identify patterns and redundancies. These methods support both interpretability and practical goals such as pruning, bias analysis, and efficient inference across language, vision, and multimodal models. A central challenge is that attention patterns or probe scores do not by themselves establish a head’s causal role: effects can depend on the task, prompt, architecture, and controls used.

Latest papers

82 papers16 papers in the last four weeks, up 45% on the four weeks before. 0.1% of all new papers.

Mechanistic Interpretability of Chain-of-Thought Reasoning via Sequential Activation Patching
Aug 23, 2026Murat Dura, Serkan Öztürk, Selma TekirLLM Interpretability CoT Reasoning
Department of Computer Engineering Izmir Institute of Technology Izmir, Türkiye
Which Question Is Your Attention Metric Answering? Attention Rows as Compositional Data
Aug 11, 2026Marios Papamichalis, Regina RuaneAttention Head Analysis Transformer Interpretability
Human Nature Lab, Yale University, New Haven, CT 06511 · Department of Statistics and Data Science, The Wharton School, University of Pennsylvania, 3733 Spruce Street, Philadelphia, PA 19104-6340
Autonomy-of-Heads: Data-Free Sparse Attention from Frozen Query-Key Geometry
Aug 7, 2026Yehan Yang, Junyuan Shang, Yang Li +3Attention Head Analysis LLM Inference Acceleration
Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Baidu Inc. +1
Interpretability-Guided Soft Pruning of Attention Heads in Vision Transformers
Jul 31, 2026Kamil Książek, Piotr Suszyński, Michał Jan Włodarczyk +2Transformer Interpretability Attention Head Analysis
Centre for Credible Artificial Intelligence Warsaw University of Technology Warsaw, Poland · Faculty of Mathematics and Computer Science Jagiellonian University Krakow, Poland · University of Warsaw Warsaw, Poland
Role-Break in Attention Heads: Understanding and Detecting Hallucinations in VLMs
Jul 31, 2026Mingyu Wang, Weilin Jin, Wenbo Li +5Hallucination Detection in VLMs Attention Head Analysis
Peking University, Beijing, China · Joy Future Academy, Beijing, China
Through the Bottleneck: How Multi-head Latent Attention Separates Content from Position in Language Models
Jul 25, 2026Dhruvil S, Fenil Sojitra, Ravirajsinh ChauhanTransformer Interpretability Low-Rank Attention
1Indian Institute of Technology Madras · 2P P Savani University
A Multi-Dimensional Evaluation of Explainability in Media Bias Detection
Jul 22, 2026Ting Chen, Raina Zhang, Benjamin M. Ampel +1Attention Head Analysis Mechanistic Interpretability
Carnegie Mellon University · Indiana University · Georgia State University
Toward Localizing and Repairing Bias in Transformer Attention Heads
Jul 14, 2026Sigma JahanAttention Head Analysis Social Bias in Language Models
Dalhousie University, Canada
Fingerprint, Not Blueprint: How Positional Schemes Set the Default Spectral Algebra of Attention
Jul 7, 2026Li HengyuRotary Positional Embeddings Attention Head Analysis
Institute for Solid State Physics, The University of Tokyo
Logit-Contribution Scoring Identifies Non-Literal Retrieval Heads
Jul 1, 2026Aryo Pradipta Gema, Beatrice Alex, Pasquale MinerviniLong-Context Retrieval Attention Head Analysis
QUniversity of Edinburgh · KHeriot-Watt University · V Miniml.AI
Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models
Jun 26, 2026Yanchen Yin, Dongqi Han, Linghui LiAttention Head Analysis Jailbreak Attacks
Beijing University of Posts and Telecommunications, Beijing, China.
Protein contacts are already in the attention: a single-forward-pass alternative to the Categorical Jacobian
Jun 20, 2026Rome ThorstensonProtein Structure Prediction Attention Head Analysis
Independent Researcher
Does RoPE Prevent or Degrade Retrieval Heads? A Mechanistic Analysis Across Model Families
Jun 19, 2026Cengizhan BayramRotary Positional Embeddings Long-Context Retrieval
Independent Researcher
Explaining Attention with Program Synthesis
Jun 17, 2026Amiri Hayes, Belinda Z Li, Jacob AndreasTransformer Interpretability Attention Head Analysis
NJIT · MIT EECS
Rethinking the Role of Efficient Attention in Hybrid Architectures
Jun 13, 2026Ziqing Qiao, Yinuo Xu, Chaojun Xiao +6Attention Head Analysis Long-Context Modeling
1Tsinghua University · 2OpenBMB
Where Computation Lives Inside TabPFN: Causal Localisation of Attention Head Function
Jun 11, 2026Atharva Gupta, Dhruv Kumar, Murari Mandal +1Attention Head Specialization Attention Head Analysis
Department of Computer Science, Birla Institute of Technology and Science, Pilani, India · School of Computer Engineering, Kalinga Institute of Industrial Technology, Bhubaneswar, India · Birla AI Labs, Office of Ananya Birla, Aditya Birla Group, India
Closure-Validated Circuit Discovery in Attention Heads: Co-activation Proposes, Ablation Disposes
Jun 8, 2026Yongzhong XuTransformer Interpretability Circuit Discovery
Ablation-Reversible Heads Don't Transfer: A Stress Test for Mechanistic Role Claims in Transformers
Jun 6, 2026Philip QuirkeAttention Head Analysis Mechanistic Interpretability
Martian
Pattern Selectivity is Not Task-Causal Structure: A Cross-Architecture Mechanistic Study of Composed-Task Circuits in 1B-Class Language Models
Jun 3, 2026Yongzhong XuAttention Head Analysis Decoder-Only Language Models
When Do Attention Circuits Form? Developmental Trajectories of Capability and Attention-Sink Emergence Across Three 1B-ClassArchitectures
Jun 1, 2026Yongzhong XuLanguage Model Pretraining Transformer Interpretability
Detection vs. Execution: Single-Bucket Probes Miss Half the Mamba-2 State Sink
May 30, 2026Yuhang JiangMamba Attention Head Analysis
Independent Researcher
Positional versus Symbolic Attention Heads: Learning Dynamics, RoPE Geometry, and Length Generalization
May 29, 2026Felipe Urrutia, Juan José Alegría, Cinthia Sanchez Macias +3Rotary Positional Embeddings Attention Head Analysis
CENIA & Faculty of Mathematics UC Santiago, Chile · CENIA Santiago, Chile · IMC UC & CENIA Santiago, Chile
Revealing Algorithmic Deductive Circuits for Logical Reasoning
May 27, 2026Phuong Minh Nguyen, Tien Huu Dang, Naoya InoueAttention Head Analysis LLM Interpretability
Japan Advanced Institute of Science and Technology
Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers
May 22, 2026Yongzhong XuTransformer Interpretability Circuit Discovery
Stable Attention Response for Reliable Precipitation Nowcasting
May 13, 2026Penghui Wen, Zexin Hu, Sen Zhang +6Precipitation Forecasting Attention Head Analysis
School of Computer Science, The University of Sydney Australia · Independent Researcher Australia · School of Life and Environmental Science, The University of Sydney Australia +3
The Routing and Filtering Structure of Attention
May 12, 2026Shafayeth Jamil, Rehan KapadiaMulti-Head Attention Self-Attention
Ming Hsieh Department of Electrical and Computer Engineering University of Southern California Los Angeles, CA 90089
Dissecting Jet-Tagger Through Mechanistic Interpretability
May 11, 2026Saurabh Rai, Sanmay GangulyTransformer Interpretability Attention Head Analysis
Department of Physics, Indian Institute of Technology, Kanpur. Uttar-Pradesh 208016, India
A Game Theoretic Free Energy Analysis of Higher Order Synergy in Attention Heads of Large Language Models
May 10, 2026Djamel BouchaffraLLM Pruning Attention Head Analysis
DAVID Lab, University of Paris-Saclay, UVSQ Campus, 78035 Versailles, France
When and Why Grouping Attention Heads Accelerates Muon Optimization
May 9, 2026Hongtao Zhang, Wenjie Zhou, Wei Chen +1Attention Head Analysis Muon Optimizer
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · University of Chinese Academy of Sciences
Architecture, Not Scale: Circuit Localization in Large Language Models
May 9, 2026Sohan VenkateshTransformer Interpretability Grouped-Query Attention
Manipal Institute of Technology Bengaluru.

Aug 23, 2026J/K move · Enter open · S save

Mechanistic Interpretability of Chain-of-Thought Reasoning via Sequential Activation Patching

Murat Dura, Serkan Öztürk, Selma Tekir

Department of Computer Engineering Izmir Institute of Technology Izmir, Türkiye

LLM Interpretability CoT Reasoning Circuits in Language Models Activation Patching Causal Interventions in Language Models Attention Head Analysis

Open paper AskPDF arXiv

Large Language Models (LLMs) demonstrate remarkable problem-solving capabilities when guided by Chain-of-Thought (CoT) prompting, yet the internal mechanisms underlying these improvements remain poorly understood. In this work, we investigate where CoT-related causal effects emerge across the generated reasoning trajectory and which attention heads carry signals that contribute to final-answer computation. Because CoT reasoning unfolds over multiple generated tokens, standard activation patching at a single static token position is insufficient to characterize these temporally distributed effects. To address this limitation, we introduce a sequential activation patching framework that traces CoT-conditioned attention-head activations across token positions and aggregates their effects using Part-of-Speech-guided analysis. We further introduce Sequential Multi-Head Patching to evaluate the joint contribution of distributed head sets, together with cross-question and random activation controls. Targeted zero-ablation experiments show that the identified heads are functionally important for successful answer generation and affect several overlapping mechanisms, including reasoning-trajectory maintenance, answer anchoring, exemplar-target separation, and numerical generation. Overall, our results provide evidence for distributed reasoning-support sub-circuits associated with CoT-conditioned computation.