When Do Attention-Head Ablations Support Causal Claims? Projection-Level Confounds, Floor Effects, and Matched Controls

Juli Huang

cs.LGSep 30, 2026

When Do Attention-Head Ablations Support Causal Claims? Projection-Level Confounds, Floor Effects, and Matched Controls

Authors: Juli Huang

Organizations: Stanford University

Abstract

Attention-head ablation, zeroing a head and measuring the resulting change in task performance, is a common method for inferring which components of a language model are causally responsible for a behavior. We show using GPT-2 small that this inference can be fragile unless the intervention semantics, evaluation metric, and controls are carefully validated. A natural post-projection implementation of "zeroing a head" is nearly uncorrelated with a corrected pre-projection ablation (Pearson r = 0.057) and selects a completely disjoint top-5 set of important heads. We also show that binary accuracy can hide effects at behavioral floors and near ceilings, whereas gold-token log-probability remains graded. Using a discovery/held-out split and 1,000 matched random-head and layer-matched-head control draws, the corrected per-head effect ranking is highly stable across splits (Spearman rho = 0.974), and the top-5 selected heads significantly exceed both control distributions (Monte Carlo p = 0.001). However, evidence for task specificity is not robust on GPT-2. Replication on DistilGPT2 preserves the intervention-semantic and matched-control findings. These results show that single-head ablation does not by itself justify a causal claim; defensible interpretation requires correct intervention placement, a non-saturated continuous metric, and matched held-out controls.

Figures & tables

Quantity	Value	Interval	$N$ / draws
Copying-task accuracy (ceiling)	0.995	[0.985, 1.000]	200
Arithmetic accuracy (floor)	0.000	[0.000, 0.000]	200
Copying-task log-prob (nats)	$-0.83$	[ $-0.90$ , $-0.76$ ]	200
Arithmetic log-prob (nats)	$-7.44$	[ $-7.53$ , $-7.35$ ]	200
Candidate $\Delta$ , held-out	0.412	[0.353, 0.474]	100
Random-control $\Delta$ , held-out	0.029	[ $-0.033$ , 0.113] †	1000

Table 1: Summary of all RQ1/RQ2 quantities. All values are computed from gpt2_results.json (Section 3 ). CIs are 95% percentile bootstrap over examples except where noted. “Control range” is the 5th–95th percentile of 1,000 control draws, not a CI.

Figure 1: A. Accuracy cannot show further degradation at the arithmetic floor and may hide confidence changes near the copying-task ceiling. Log-probability stays continuous. B. The top-5 candidate heads’ held-out effect far exceeds matched random-head and layer-matched control distributions (error bars: 95% CI for the candidate, 5th–95th percentile for controls). C. Specificity (target-task minus control-task effect) is larger for candidates than the random-control mean, but the difference is only suggestive (Monte Carlo $p=0.078$ ).

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Per-head $\Delta$ log-prob under the correct (pre-projection) vs. naive (post-projection) ablation method, all 144 heads, controlled copying-task discovery split. A valid replication of the naive method would lie near the $y=x$ line. Instead the two methods are nearly uncorrelated (Pearson $r=0.057$ ).

Quantity	Value	Interval	$N$ / draws
Copying-task accuracy	0.900	[0.855, 0.940]	200
Arithmetic accuracy (floor)	0.000	[0.000, 0.000]	200
Copying-task log-prob (nats)	$-0.96$	[ $-1.07$ , $-0.85$ ]	200
Arithmetic log-prob (nats)	$-7.38$	[ $-7.50$ , $-7.25$ ]	200
RQ0: Pearson $r$ (naive, correct)	0.001	–	72 heads
RQ0: top-5 head overlap	0/5	–	–

Appendix

Table 2: DistilGPT2 replication of Table 1 , with the same procedure and

N

s. The intervention-semantic and matched-control findings replicate, and the arithmetic floor recurs. Copying accuracy is lower (90.0%), so GPT-2’s near-ceiling example does not replicate. The specificity comparison gives Monte Carlo

p=0.001

Explore similar work

Pruning via Causal Attribution Preserves Reasoning Performance in Large Language Models
Apr 27, 2026Amogh Sheth, Biruk Assefa, Yi Wen Huang +2LLM Reasoning Strategies Inference Cost
Edison Academy Magnet School · State University of New York College at Plattsburgh · The University of Texas at Austin +2
Which Attention Heads are like the Human Head? Not the Ones that Compute
Sep 29, 2026Christopher Pinier, Gustaw Opiełka, Hannes Rosenbusch +3Artificial Intelligence Alignment Code Completion
Psychological Methods, University of Amsterdam, Amsterdam, The Netherlands · Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
Ablation-Reversible Heads Don't Transfer: A Stress Test for Mechanistic Role Claims in Transformers
Jun 6, 2026Philip QuirkeTransformer Attention Transformer Architectures
Martian

Apr 27, 2026, cs.CLJ/K move · Enter open · S save

Pruning via Causal Attribution Preserves Reasoning Performance in Large Language Models

Amogh Sheth, Biruk Assefa, Yi Wen Huang, Andrew Lin+1

Edison Academy Magnet School · State University of New York College at Plattsburgh · The University of Texas at Austin+2

LLM Reasoning Strategies Inference Cost

Open paper AskPDF arXiv

Large language models (LLMs) excel at multi-step reasoning but incur substantial inference cost. We introduce Causal Attribution Pruning (CAP), a training-free method that identifies critical attention heads by measuring their causal impact on reasoning tasks and uses these head-level scores to guide fine-grained weight pruning. For each attention head, CAP estimates the expected performance degradation when the head is masked during forward passes on a small calibration set of reasoning problems. These causal scores are then converted into weight-level importance values for the corresponding projection matrices. Unlike magnitude-only or activation-based criteria, CAP's interventional measurement directly captures each head's functional contribution, yielding relative accuracy gains of up to 61% over Wanda on ARC-Challenge at 20% sparsity. We evaluate CAP on GSM8K, StrategyQA, and ARC-Challenge using Llama-3-8B-Instruct and Mistral-7B-Instruct at 10%, 20%, and 50% sparsity. At moderate sparsity (10-20%), CAP improves over Wanda in most model-benchmark configurations. with especially large gains on ARC-Challenge for Llama-3. Our results suggest that attention-head-level causal attribution can better preserve reasoning performance on downstream benchmarks than correlational pruning criteria at equivalent sparsity, while remaining limited by coarse MLP attribution at 50% sparsity.

Abstract

Explore similar work

Pruning via Causal Attribution Preserves Reasoning Performance in Large Language Models

Which Attention Heads are like the Human Head? Not the Ones that Compute

Ablation-Reversible Heads Don't Transfer: A Stress Test for Mechanistic Role Claims in Transformers