Activation Patching

Follow this topic Ask about this topic

Activation patching replaces a model’s internal states with activations from another run to test which layers, tokens, or components causally affect an output. Researchers use direct and path patching, sequential interventions, and faster attribution-based estimates to trace computations in language models, vision-language models, and other networks. Studies examine tasks such as indirect-object identification, factual recall, multi-step reasoning, and multimodal question answering. A central challenge is that effects depend on interactions and context: a feature may be encoded without influencing the answer, and an intervention may produce the right answer through a mechanism the model would not naturally use.

Latest papers

25 papers5 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Correcting Attribution Patching with a Few Exact Measurements
Oct 19, 2026Luyang Zhang, Minxing Zheng, Jialu WangLLM Interpretability Activation Patching
Carnegie Mellon University · University of California, Santa Cruz
Does the Model Use the Feature? Separating Steering from Mechanism in LLMs
Oct 5, 2026Tong Che, Yilong LiLLM Interpretability Mechanistic Interpretability
NVIDIA Research · University of Wisconsin–Madison
From Patching to Pruning Visual Computation in Vision Language Models
Oct 2, 2026Rahul Chowdhury, Timothy A Rupprecht, Xuan Shen +3Efficient VLM Inference Visual Token Pruning
Northeastern University, USA · EmbodyX Inc., USA · Zhejiang University, China +1
Right Answer, Wrong Mechanism: Detecting Pernicious Divergence in Causal Interventions
Sep 30, 2026Beiming Liu, Minjie ChenCircuits in Language Models Causal Interpretability
PetroChina Southwest Oil and Gasfield Company, China
Social Circuits behind Multi-agent Echo Chambers
Sep 28, 2026Chuiyang Meng, Wenlu Yu, Ming Tang +1Multi-Agent LLM Systems Multi-Agent Systems
Simon Fraser University · University of Alberta · Southern University of Science and Technology
Encoded but Disconnected: Decomposing Vision-Language Model Failures under a Patching Null
Sep 1, 2026Genpei ZhangVLM Interpretability Mechanistic Interpretability
University of Wisconsin–Madison
StateSwap: Probing Support-Elimination Hidden States in Multiple-Choice Questions
Sep 1, 2026Chao Gao, Haijiang Liu, Qiyuan Li +3Prompt Sensitivity Multiple-Choice Question Answering
Department of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands · School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, China · Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan University of Science and Technology, Wuhan, China
Knowing Isn't Always Saying: When Do Spatial Encodings Reach Answers in Vision-Language Models?
Aug 24, 2026Zeyu Wang, Xinming XuVLM Interpretability Vision-Language Grounding
Tsinghua University Beijing, China
Mechanistic Interpretability of Chain-of-Thought Reasoning via Sequential Activation Patching
Aug 23, 2026Murat Dura, Serkan Öztürk, Selma TekirLLM Interpretability CoT Reasoning
Department of Computer Engineering Izmir Institute of Technology Izmir, Türkiye
Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Model
Aug 4, 2026Abdallah KhemaisTransformer Interpretability Weight-Space Analysis
ISITCOM, University of Sousse
A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation
Aug 4, 2026Abdallah KhemaisTransformer Interpretability Weight-Space Analysis
ISITCOM, University of Sousse
Cost Accounting for Reactive Computational Graphs: Exhaustive Sweeps, Sequential Mutation, and the Backward-Locality Gap
Jul 18, 2026Abdallah KhemaisActivation Patching Incremental Computation
ISITCOM, University of Sousse
The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching
Jun 25, 2026Sankaran Vaidyanathan, David Arbour, Aaron Mueller +2Causal Effect Estimation Causal Attribution
University of Massachusetts Amherst · Adobe Research · Boston University
Factual Retrieval in LLMs Is a Redundant, Distributed and Non-Contiguous Process
Jun 19, 2026Hail Hochman, Natalie Shapira, Yoav GoldbergLLM Interpretability Factual Knowledge in Language Models
Bar-Ilan University · Northeastern University
Beyond Importance: Interchange-Sobol Sensitivity Reveals Task-Specific Content Channels in Transformer Components
Jun 12, 2026Yifeng Guo, Jin-Hong Du, Xiang ChenMechanistic Interpretability Causal Interventions in Language Models
St. Jude Children’s Research Hospital Memphis, TN · The University of Hong Kong Hong Kong, CN
Ablation-Reversible Heads Don't Transfer: A Stress Test for Mechanistic Role Claims in Transformers
Jun 6, 2026Philip QuirkeAttention Head Analysis Mechanistic Interpretability
Martian
When Attribution Patching Lies: Diagnosis and a Second-Order Correction
Jun 5, 2026Luyang Zhang, Jialu WangGradient-Based Attribution LLM Interpretability
1Carnegie Mellon University · University of California, Santa Cruz
Transformer Field Theory: A Response-Theoretic Approach to Mechanistic Interpretability
May 24, 2026David N. Olivieri, Antonio F. Pérez RodríguezTransformer Interpretability Mechanistic Interpretability
Departamento de Informática, Universidade de Vigo, Spain · Independent Researcher, Spain
Instruction Tuning Changes How Upstream State Conditions Late Readout: A Cross-Patching Diagnostic
May 8, 2026Yifan ZhouTransformer Interpretability Instruction Tuning
University of California, Los Angeles
Patch-Effect Graph Kernels for LLM Interpretability
May 7, 2026Ruben Fernandez-Boullon, David N. OlivieriGraph Representation Learning Mechanistic Interpretability
Department of Computer Science, University of Vigo, Ourense 32004, Spain
Squish and Release: Exposing Hidden Hallucinations by Making Them Surface as Safety Signals
Mar 27, 2026Nathaniel Oh, Paul AttieHallucination in Language Models LLM Hallucination Mitigation
School of Computer and Cyber Sciences Augusta University Augusta, Georgia 30912
Posterior-Calibrated Causal Circuits in Variational Autoencoders: Why Image-Domain Interpretability Fails on Tabular Data
Mar 22, 2026Dip Roy, Rajiv Misra, Sanjay Kumar Singh +1Variational Autoencoders Causal Interpretability
Department of Computer Science and Engineering, Indian Institute of Technology, Patna, India · Department of Computer Science, Rajarshi School of Management Technology, Varanasi, India · Department of Electronics and Communication Engineering, Jaypee Institute of Information Technology, Noida
Causal Attribution via Activation Patching
Mar 13, 2026Amirmohammad Izadi, Mohammadali Banayeeanzade, Alireza Mirrokni +4Feature Attribution Transformer Interpretability
Sharif University of Technology
DreamReader: An Interpretability Toolkit for Text-to-Image Models
Mar 2, 2026Nirmalendu Prakash, Narmeen Oozeer, Michael Lan +6Diffusion Models T2I Diffusion Models
Singapore University of Technology and Design · Martian · Thoughtworks +2
MINAR: Mechanistic Interpretability for Neural Algorithmic Reasoning
Feb 24, 2026Jesse He, Helen Jenne, Max Vargas +4Neural Algorithmic Reasoning Mechanistic Interpretability
Pacific Northwest National Laboratory, Richland, WA · Halıcı˘oğlu Data Science Institute, University of California, San Diego, San Diego, CA · Department of Computer and Information Science, University of Pennsylvania, Pennsylvaina, PA +1

Oct 19, 2026J/K move · Enter open · S save

Correcting Attribution Patching with a Few Exact Measurements

Luyang Zhang, Minxing Zheng, Jialu Wang

Carnegie Mellon University · University of California, Santa Cruz

LLM Interpretability Activation Patching Gradient-Based Attribution

Open paper AskPDF arXiv

Mechanistic interpretability asks which language model components matter for a behavior, and activation patching answers by changing one component's activation and measuring its effect, at a cost growing with components and prompts. Attribution patching estimates these effects from one gradient pass, but its estimates can be far off, and verifying each one restores the cost. Can the structure of these errors reduce this cost? We propose Guided-AP, which uses a few exact measurements to reveal attribution patching's errors and correct unmeasured estimates. On a single prompt, many components' errors share one sign, so a small exact sample yields one correction for the rest. Over many prompts of one task, components are misestimated similarly, so a few measured prompts show what to verify and how to correct each component's average effect. Our analysis shows that choosing what to verify needs error size, while correction also needs sign. On a single prompt, the correction lowers error by up to 26% across three behaviors at the compute of verifying the largest estimates and also on larger models. Over many prompts, learned corrections cut the error of average effects on factual completion by 75% to 79%, repaying their cost in long analyses.