Causal Localization of Evidence Effects in Transformers

Jipeng Li; Pramod Khargonekar; Yanning Shen

cs.CLOct 27, 2026

Causal Localization of Evidence Effects in Transformers

Authors: Jipeng Li, Pramod Khargonekar, Yanning Shen

Abstract

Can controlled symbolic inputs reveal attention heads that influence a language model's decisions from natural language evidence? We introduce controlled evidence aggregation (CEA), which varies the counts and order of two symbols in otherwise fixed prompts and measures changes in a frozen Transformer's preference between two answer tokens. The preference is measured by their logit difference; the prompts prescribe no correct answer. Balanced utility causal selection (BUC) identifies compact head sets by jointly evaluating how much ablating their outputs removes the count effect and how closely patching mean outputs between count conditions recovers the effect in both directions. We derive conditional bounds linking head attribution to intervention effects and specifying when the attribution shortlist can be recovered from finite samples. In Pythia-410M, ablating sets of 1-12 heads selected separately for four ordering regimes removes 65.5-91.6% of the count effect. The sets retain their influence on count comparisons not used for selection. To test relevance beyond symbolic inputs, we evaluate a natural language task asking which state is supported by more reports. Ablating eight heads selected only on symbolic yes/no prompts lowers mean accuracy by 8.2 percentage points, versus 0.8 for random sets matched by size and layer. Our results demonstrate that controlled symbolic probes can identify compact attention head sets whose causal influence extends to natural language decisions.

Figures & tables

Regime	$\Delta_{r}$	Size	BUC shrink / mean match	BUC HL match	Random shrink / HL match
Front	0.239	1	0.906 / 0.974	0.998	0.020 / 0.048
Back	1.965	12	0.655 / 0.759	0.765	0.043 / 0.075
Interleave	1.659	3	0.916 / 0.914	0.937	0.061 / 0.051
Random	$3.184\!\pm\!0.097$	$7\!\pm\!1$	$\mathbf{0.866\!\pm\!0.021}/\mathbf{0.823\!\pm\!0.084}$	0.813	0.069 / 0.061

Table 1: Main BF16 symbolic result at

(b_{L},b_{H})=(50,150)

. BUC mean match averages both directions. The last two columns compare the same HL direction with random controls; their values are reported in Appendix B.3 . Uncertainty is the standard deviation over five executions.

Regime	BUC shrink / match	Sens. shrink / match	ActPatch sum
Front	0.906 / 0.974	0.920 / 0.960	1.865
Back	0.655 / 0.759	0.704 / 0.648	1.285
Interleave	0.916 / 0.914	0.934 / 0.831	1.695
Random	0.866 / 0.823	0.895 / 0.716	1.520

Table 2: BF16 comparison with matched head counts within each regime (Table 1 ), not a common

k=8

. ActPatch ranks 192 sensitivity-shortlisted heads by single-head HL recovery, then reports the jointly evaluated set score

2Q_{1/2}

(Appendix B.10 ).

Regime	Spearman $\rho(b,y)$	Linear $R^{2}$	Slope reduction	Unseen-gap shrink / match
Front	0.986	0.948	0.880	0.832 / 0.874
Back	0.972	0.931	0.646	0.612 / 0.689
Interleave	0.979	0.944	0.896	0.839 / 0.855
Random	0.958	0.906	0.845	0.786 / 0.811

Table 3: Fixed-set count generalization. The final column averages evaluation on three unseen gaps without head reselection.

Target pair	Reselected shrink / match	Fixed-set shrink / match	Jaccard
yes/no	0.812 / 0.846	0.772 / 0.810	0.610
A/B	0.861 / 0.884	0.704 / 0.741	0.390
cat/dog	0.642 / 0.701	0.382 / 0.431	0.240

Table 4: Representative readout-pair results, averaged over matched regime–seed sets. The fixed set was selected with various/successful .

Condition	Accuracy	PairAcc	MinPairAcc	Shrink / match	Slope reduction	Acc. drop
Base behavior	0.772	0.604	0.583	–	–	–
NL set $\rightarrow$ NL	–	–	–	0.734 / 0.766	0.721	15.2
CEA set $\rightarrow$ NL	–	–	–	0.428 / 0.463	0.411	8.2
Layer-matched random	–	–	–	0.059 / 0.072	0.051	0.8

Table 5: Original BF16 natural-language bridge, five-seed mean. The behavior row uses held-out domains and paraphrases. The intervention rows use equal budget

k=8

. Accuracy drop is in percentage points.

Model	Accuracy	PairAcc	MinPairAcc	CEA $\to$ NL shrink	CEA $\to$ NL match	TR	$n_{\mathrm{est}}/12$	$n_{\mathrm{suppr}}$	Acc. drop (pp)
Pythia-410M	0.752	0.566	0.541	0.401	0.442	0.520	11/12	1	7.4
Pythia-1B	0.805	0.651	0.611	0.462	0.501	0.557	12/12	0	8.6
GPT-2	0.681	0.458	0.414	0.306	0.348	0.438	9/12	3	4.9

Table 6: Expanded FP32 NL bridge, separate from Table 5 : model-level point estimates without model-level CIs. Behavior uses gap

(4,12)

; transfer uses fixed symbolic yes/no sets.

n_{\mathrm{est}}

counts estimable TR conditions out of 12;

n_{\mathrm{suppr}}

counts those with a suppressed condition-level interval. Accuracy drop is in percentage points on the causal cohort.

Regime	Minimum shrink	Minimum match	Negative marginals (%)	Minimum $\widehat{\gamma}$	Maximum selection regret
Front	0.81	0.84	31	$-5.8$	0.021
Back	0.52	0.57	17	$-2.6$	0.043
Interleave	0.80	0.81	28	$-4.9$	0.027
Random	0.71	0.68	22	$-3.9$	0.035

Table 7: FP32 operator and local selection audits: point estimates. Shrink and match minima across eight operators need not share a minimizer. Evaluation marginals/ratios use both eight-head pools and tolerance

10^{-3}

. Maximum regret compares exact and greedy selection utilities over both pools and budgets

k=1,\ldots,8

, not held-out performance. These finite summaries have no population CIs (Appendix H ).

Appendix figures & tables44 assets

Supplementary material from the paper’s appendix.

Appendix

	Layers	Heads per layer	Hook tensor
Value	$24$	$16$	attn.hook_z
Notes	no fine-tuning	total $384$ heads	projected through $W_{O}$ before logit-direction projection
	Evaluation dtype	Prefix token length	Symbolic sequence token length
Value	bfloat16	$5$	$250$
Notes	inference only	tokenizer audit	one token per symbol in the tested tokenizer
	Suffix token length	Seq token slice	Answer candidates

Appendix

Table 8: Prompt and tokenization audit for the main Pythia-410M experiment. The Seq token slice refers to symbolic-sequence positions and excludes the final prompt position.

Model	Single token	$w_{+}$	$w_{-}$	Notes
EleutherAI/pythia-70m-deduped	yes	various	successful	audited
EleutherAI/pythia-160m-deduped	yes	various	successful	audited
EleutherAI/pythia-410m-deduped	yes	various	successful	main model
EleutherAI/pythia-1b-deduped	yes	various	successful	audited
gpt2	yes	various	successful	out-of-family baseline
Qwen/Qwen3-4B-Base	yes	various	successful	modern open-weight baseline

Appendix

Table 9: Tokenizer audit for the model sweep. The answer pair is usable only when both answer strings are single-token candidates under the model tokenizer.

Configuration item	Value
Evidence setting	MIXED $(b_{L},b_{H})=(50,150)$
Ordering regimes	Front , Back , Interleave , Random
Seeds	$\{0,1,2,3,4\}$
Selection prompts per evidence level	$48$
Evaluation prompts per evidence level	$96$
Audit prompts per evidence level	$16$

Appendix

Table 10: Selection and evaluation configuration for the main Pythia-410M experiment.

Regime $r$	$\Delta_{r}$	Notes
Front	$0.239\pm 0.000$	fixed deterministic input
Back	$1.965\pm 0.000$	large shift
Interleave	$1.659\pm 0.000$	large shift
Random	$3.184\pm 0.097$	largest and seed-dependent

Appendix

Table 11: Base evidence effects in the MIXED setting

(b_{L},b_{H})=(50,150)

on pythia-410m-deduped .

Regime $r$	$\mathrm{match}^{\mathrm{HL}}$	$\mathrm{match}^{\mathrm{LH}}$	$\mathrm{match}_{\mathrm{avg}}$
Front	$0.998\pm 0.000$	$0.950\pm 0.000$	$0.974\pm 0.000$
Back	$0.765\pm 0.000$	$0.753\pm 0.000$	$0.759\pm 0.000$
Interleave	$0.937\pm 0.000$	$0.891\pm 0.000$	$0.914\pm 0.000$
Random	$0.813\pm 0.092$	$0.833\pm 0.080$	$0.823\pm 0.084$

Appendix

Table 12: Directional mean-patching recovery in the MIXED setting. The average of the two directional scores is the BUC match score reported in the main text.

Regime $r$	$\|H_{\mathrm{BUC}}(r)\|$	BUC shrink	BUC match	Random shrink	Random HL match
Front	$1.00\pm 0.00$	$0.906\pm 0.000$	$0.974\pm 0.000$	$0.020$	$0.048$
Back	$12.00\pm 0.00$	$0.655\pm 0.000$	$0.759\pm 0.000$	$0.043$	$0.075$
Interleave	$3.00\pm 0.00$	$0.916\pm 0.000$	$0.914\pm 0.000$	$0.061$	$0.051$
Random	$7.00\pm 1.00$	$0.866\pm 0.021$	$0.823\pm 0.084$	$0.069$	$0.061$

Appendix

Table 13: Full white-box result with matched random controls in the MIXED setting. BUC values are mean

\pm

standard deviation across five seeds. Random match is measured in the high-to-low direction.

Regime $r$	Seed(s)	$H_{\mathrm{BUC}}(r)$
Front	0–4	$(10,2)$
Back	0–4	$(19,6),\allowbreak(20,2),\allowbreak(13,5),\allowbreak(15,3),\allowbreak(19,9),\allowbreak(17,3),\allowbreak(15,10),\allowbreak(15,5),\allowbreak(19,14),\allowbreak(14,13),\allowbreak(18,1),\allowbreak(11,8)$
Interleave	0–4	$(17,3),\allowbreak(10,2),\allowbreak(15,10)$
Random	0	$(17,3),\allowbreak(14,7),\allowbreak(10,2),\allowbreak(19,14),\allowbreak(16,1),\allowbreak(15,10),\allowbreak(19,6),\allowbreak(16,14)$
Random	1	$(17,3),\allowbreak(14,7),\allowbreak(12,14),\allowbreak(19,14),\allowbreak(10,2),\allowbreak(15,10),\allowbreak(19,6)$
Random	2	$(17,3),\allowbreak(14,7),\allowbreak(12,14),\allowbreak(19,14),\allowbreak(10,2),\allowbreak(15,10)$

Appendix

Table 14: Selected BUC head sets in the MIXED setting. Heads are written as

(\ell,h)

. For Front , Back , and Interleave , the listed set is identical across five seeds.

Regime $r$	Minimum pairwise Jaccard	Interpretation
Front	$1.000$	identical selected set
Back	$1.000$	identical selected set
Interleave	$1.000$	identical selected set
Random	$0.556$	listed seeds 0–2 only

Appendix

Table 15: Stability of the head sets explicitly listed in Table 14 . Deterministic-regime sets are identical across seeds 0–4. The Random value is the minimum over listed seeds 0–2 only; seeds 3 and 4 are not included in this calculation.

Regime $r$	Head(s)	Main routed region	Patch comparison
Front	$(10,2)$	$R_{B}\cup R_{C}$	Larger OV patch match
Back	$(19,6),(20,2),(13,5)$	$R_{C}$ -side routing shift	Positive QK and OV matches
Interleave	$(17,3),(10,2),(15,10)$	Broad over $R_{E}$	Larger OV patch match
Random	$(17,3),(14,7),(19,14)$	Broad over $R_{E}$	Larger OV patch match

Appendix

Table 16: Mechanistic diagnostics for representative selected heads at the answer position.

\Delta\mathsf{Attn}

is a high-minus-low attention-mass contrast, and

\Delta\mathsf{OV}

is a high-minus-low source-wise OV contribution contrast along

d

. The attention and OV difference columns report

\Delta\mathsf{Attn}(R_{B},r)-\Delta\mathsf{Attn}(R_{C},r)

and

\Delta\mathsf{OV}(R_{B},r)-\Delta\mathsf{OV}(R_{C},r)

, respectively. QK / OV match reports pattern-only and OV-only patch recovery.

Regime $r$	$k^{\star}$	Last shrink	Seq shrink	Seq match
Front	$1$	$0.906$	$0.059$	$-0.006$
Back	$12$	$0.655$	$0.022$	$+0.003$
Interleave	$3$	$0.916$	$0.001$	$+0.006$
Random	$7\text{--}8$	$0.835\text{--}0.917$	$\approx 0.008\text{--}0.010$	$\approx-0.001$

Appendix

Table 17: Last versus Seq using the same selected head set. The Seq effect is near zero in every regime.

Source $\backslash$ Target	Front	Back	Interleave	Random
Front	–	$+0.129$	$+0.173$	$+0.066$
Back	$+0.958$	–	$+0.961$	$+0.850$
Interleave	$-0.467$	$+0.230$	–	$+0.611$
Random	$-0.925$	$+0.413$	$+0.687$	–

Appendix

Table 18: Representative off-diagonal cross-regime transfer matrix under Last interventions (seed 0). Rows are source regimes; columns are target regimes. Entries are recovery-match values. Diagonal effects are reported separately by direction in Table 12 .

Source $\backslash$ Target	Front	Back	Interleave	Random
Front	–	$0.129\pm 0.000$	$0.173\pm 0.000$	$0.066\pm 0.001$
Back	$0.958\pm 0.000$	–	$0.961\pm 0.000$	$0.851\pm 0.002$
Interleave	$-0.467\pm 0.000$	$0.230\pm 0.000$	–	$0.616\pm 0.006$
Random	$-0.912\pm 0.021$	$0.428\pm 0.015$	$0.695\pm 0.011$	–

Appendix

Table 19: Off-diagonal cross-regime transfer matrix averaged over seeds. Entries are mean

\pm

standard deviation of the recovery match; within-regime directional effects are in Table 12 .

Regime $r$	Sets	Pearson $\rho(F,N)$	Pearson $\rho(F,\Gamma^{\mathrm{HL}})$	Pearson $\rho(F,\Gamma^{\mathrm{LH}})$	Spearman $\rho(F,N)$	Spearman $\rho(F,S)$
Front	$500$	$0.923$	$0.851$	$0.884$	$0.941$	$0.892$
Back	$500$	$0.875$	$0.782$	$0.816$	$0.899$	$0.835$
Interleave	$500$	$0.908$	$0.814$	$0.842$	$0.927$	$0.861$
Random	$500$	$0.841$	$0.755$	$0.793$	$0.865$	$0.804$

Appendix

Table 20: Many-set T1 diagnostic. Pearson and Spearman correlations are computed over

500

sampled head sets

H

on held-out batches.

Regime $r$	BUC	Sens.-only	ActPatch	Top-norm	Layer-rand (HL)
Front	$1.880$	$1.880$	$1.865$	$0.451$	$0.068$
Back	$1.414$	$1.352$	$1.285$	$0.523$	$0.118$
Interleave	$1.830$	$1.765$	$1.695$	$0.614$	$0.112$
Random	$1.689$	$1.611$	$1.520$	$0.488$	$0.130$

Appendix

Table 21: BF16 selection baselines under matched head budgets. BUC, sensitivity-only, ActPatch, and top-norm use

\mathrm{shrink}+\mathrm{match}_{\mathrm{avg}}

. Layer-random uses shrink plus HL-only match and is a directional control, not an identical-objective comparison.

$\lambda$	Selection emphasis	Mean $\|H_{\lambda}\|$	Mean shrink	Mean match	Mean Jaccard to $H_{1/2}$
$0.00$	recovery only	$6.25$	$0.725$	$0.895$	$0.650$
$0.25$	recovery weighted	$5.75$	$0.810$	$0.880$	$0.885$
$0.50$	balanced	$5.75$	$0.836$	$0.868$	$1.000$
$0.75$	removal weighted	$5.50$	$0.865$	$0.812$	$0.820$
$1.00$	removal only	$5.00$	$0.890$	$0.645$	$0.580$

Appendix

Table 22: Sensitivity to the joint-utility trade-off

\lambda

. Values are averaged over Front , Back , Interleave , and Random in the MIXED setting. The balanced setting

(\lambda=0.50)

matches the primary causal sets used in the main text.

Regime $r$	Metric	$k=1$	$k=2$	$k=3$	$k=4$	$k=6$	$k=8$	$k=12$
Front	shrink	$0.906$	$0.908$	$0.910$	$0.911$	$0.915$	$0.918$	$0.920$
Front	match	$0.974$	$0.975$	$0.976$	$0.976$	$0.978$	$0.979$	$0.981$
Back	shrink	$0.142$	$0.285$	$0.371$	$0.463$	$0.541$	$0.598$	$0.655$
Back	match	$0.165$	$0.312$	$0.404$	$0.518$	$0.627$	$0.693$	$0.759$
Interleave	shrink	$0.465$	$0.781$	$0.916$	$0.919$	$0.924$	$0.927$	$0.931$
Interleave	match	$0.482$	$0.815$	$0.914$	$0.917$	$0.920$	$0.922$	$0.926$

Appendix

Table 23: Greedy-prefix scores at the evaluated head budgets.

j_{m}\in\operatorname*{arg\,max}_{j\notin S_{m-1}}\widehat{\mathrm{shrink}}_{\mathrm{bb}}(S_{m-1}\cup\{j\},r)

Appendix

Algorithm 1 Greedy Occlusion for input-level subset selection

Regime $r$	Subset family	$\|S^{\star}(r)\|$	BB shrink	BB recovery	Random shrink	Random recovery
Front	Greedy Occlusion	$50$	$0.852$	$0.871$	$0.245$	$0.251$
Back	Greedy Occlusion	$50$	$0.915$	$0.898$	$0.258$	$0.262$
Interleave	Greedy Occlusion	$50$	$0.840$	$0.855$	$0.249$	$0.255$
Random	Greedy Occlusion	$50$	$0.795$	$0.812$	$0.252$	$0.248$

Appendix

Table 24: Black-box masking and add-back results using Greedy Occlusion position selection.

Subset family	Description	Status
Contiguous spans	$S=\{a,a+1,\ldots,a+\ell-1\}$	Evaluated
Union of spans	$S$ is a union of several disjoint contiguous spans	Evaluated
Prefix/suffix evidence spans	Spans anchored at the beginning or end of the evidence region	Low performance
White-box-informed positions	Positions scored from internal signals; diagnostic only	Not strict black-box
Matched random subsets	Same size or same span length as $S^{\star}(r)$	Baseline

Appendix

Table 25: Candidate subset families for masking and add-back. White-box-informed positions are diagnostic only and are not used for the strict black-box claim.

Regime change	$\Delta^{\mathrm{order}}(\cdot;b_{L}=50)$	$\Delta^{\mathrm{order}}(\cdot;b_{H}=150)$
Front $\rightarrow$ Back	$-0.258$	$+1.468$
Front $\rightarrow$ Interleave	$-0.194$	$+1.226$
Back $\rightarrow$ Interleave	$+0.064$	$-0.242$
Interleave $\rightarrow$ Random	$+0.321$	$+1.846$

Appendix

Table 26: Order effects at fixed evidence count. Positive shifts indicate that the target regime produces a larger output log-odds response than the source regime.

Setting	$(b_{L},b_{H})$	Mean $\|H\|$	Mean shrink	Mean match	Notes
MIXED	$(50,150)$	$5.75$	$0.836$	$0.868$	main configuration
EXTREME	$(0,150)$	$6.25$	$0.892$	$0.915$	larger contrast
SMALL-GAP	$(75,125)$	$4.50$	$0.712$	$0.745$	lower signal-to-noise
COUNT-SWEEP	multiple	$5.50$	$0.810$	$0.835$	increments of $10$

Appendix

Table 27: Evidence-gap robustness. Values are averaged over the four ordering regimes.

Tail length $L_{A}$	Mean $\Delta_{r}$	Mean $\|H\|$	Mean shrink	Mean match
$0$	$1.950$	$5.25$	$0.850$	$0.880$
$25$	$1.850$	$5.50$	$0.842$	$0.875$
$50$	$1.762$	$5.75$	$0.836$	$0.868$
$100$	$1.520$	$6.50$	$0.815$	$0.840$

Appendix

Table 28: Tail-length ablation. Intervention scores remain positive over the tested tail lengths; the mean output contrast decreases as the tail grows.

Surface	Example evidence tokens	Model	Mean $\|H\|$	Mean shrink	Mean match
ABC symbolic	$B,C,A$	pythia-410m-deduped	$5.75$	$0.836$	$0.868$
Word-symbolic	blue , red , gray	pythia-410m-deduped	$6.25$	$0.812$	$0.845$
Signal-token	signal-x , signal-y , neutral	pythia-410m-deduped	$6.00$	$0.820$	$0.852$
Sentence-template	short evidence sentences	pythia-410m-deduped	$8.50$	$0.765$	$0.790$

Appendix

Table 29: Surface-form validation. The tested word and sentence surfaces yield larger selected sets and positive intervention effects; set size does not measure all processing in the model.

Model	Token pair	Mean $\|H\|$	BUC shrink	BUC match	Random shrink	Random HL match	LAST effect
EleutherAI/pythia-70m-deduped	yes	$3.50$	$0.650$	$0.700$	$0.030$	$0.040$	yes
EleutherAI/pythia-160m-deduped	yes	$4.25$	$0.720$	$0.750$	$0.035$	$0.045$	yes
EleutherAI/pythia-410m-deduped	yes	$5.75$	$0.836$	$0.868$	$0.048$	$0.059$	yes
EleutherAI/pythia-1b-deduped	yes	$6.50$	$0.880$	$0.910$	$0.040$	$0.050$	yes
gpt2	yes	$4.00$	$0.750$	$0.780$	$0.035$	$0.045$	yes
Qwen/Qwen3-4B-Base	yes	$7.50$	$0.895$	$0.925$	$0.035$	$0.042$	yes

Appendix

Table 30: Aggregate model sweep. Each numerical aggregate is the unweighted arithmetic mean of the four corresponding displayed per-regime values in Tables 31 – 32 . Random match uses HL only.

Model	Regime $r$	Token pair	$\|H\|$	BUC shrink	BUC match	Random shrink	Random HL match
pythia-70m	Front	yes	$1.00$	$0.751$	$0.812$	$0.012$	$0.024$
pythia-70m	Back	yes	$5.00$	$0.552$	$0.614$	$0.045$	$0.051$
pythia-70m	Interleave	yes	$4.00$	$0.684$	$0.722$	$0.038$	$0.046$
pythia-70m	Random	yes	$4.00$	$0.613$	$0.652$	$0.025$	$0.039$
pythia-160m	Front	yes	$1.00$	$0.824$	$0.861$	$0.015$	$0.028$
pythia-160m	Back	yes	$7.00$	$0.635$	$0.681$	$0.048$	$0.059$

Appendix

Table 31: Per-regime Pythia model sweep.

Model	Regime $r$	Token pair	$\|H\|$	BUC shrink	BUC match	Random shrink	Random HL match
gpt2	Front	yes	$1.00$	$0.820$	$0.860$	$0.015$	$0.025$
gpt2	Back	yes	$6.00$	$0.680$	$0.710$	$0.045$	$0.055$
gpt2	Interleave	yes	$4.00$	$0.780$	$0.810$	$0.035$	$0.045$
gpt2	Random	yes	$5.00$	$0.720$	$0.740$	$0.045$	$0.055$
Qwen/Qwen3-4B-Base	Front	yes	$2.00$	$0.945$	$0.970$	$0.012$	$0.015$
Qwen/Qwen3-4B-Base	Back	yes	$12.00$	$0.850$	$0.885$	$0.045$	$0.055$

Appendix

Table 32: Per-regime GPT-2 and Qwen3 model sweep.

Item	Value
Hardware	NVIDIA A100 GPU
Software environment	Python / PyTorch / Transformers / TransformerLens; the BF16 environment records are separate from the supplied FP32 package (Appendix F.5 )
Primary model	EleutherAI/pythia-410m-deduped
Evaluation precision	bfloat16 inference
Training or fine-tuning	none
Main seeds	$\{0,1,2,3,4\}$

Appendix

Table 33: Compute and reproduction summary for the reported experiments.

Target pair	Reselected shrink / match	Fixed-set shrink / match	Mean Jaccard
yes/no	0.812 / 0.846	0.772 / 0.810	0.610
A/B	0.861 / 0.884	0.704 / 0.741	0.390
cat/dog	0.642 / 0.701	0.382 / 0.431	0.240

Appendix

Table 34: Representative readout-pair results. Reselected BUC tests procedure robustness. Fixed reference-set transfer tests reuse of the set selected with various/successful .

Regime	Spearman $\rho$	Linear $R^{2}$	Slope reduction	Gap shrink / match
Front	0.986	0.948	0.880	0.832 / 0.874
Back	0.972	0.931	0.646	0.612 / 0.689
Interleave	0.979	0.944	0.896	0.839 / 0.855
Random	0.958	0.906	0.845	0.786 / 0.811

Appendix

Table 35: Fixed-set count-sweep results. The final column averages the three unseen gaps without reselection.

Regime	Shrink difference	Mean-match difference	$Q_{1/2}$ difference
Front	$-0.014$	$+0.014$	$0.0000$
Back	$-0.049$	$+0.111$	$+0.0310$
Interleave	$-0.018$	$+0.083$	$+0.0325$
Random	$-0.029$	$+0.107$	$+0.0390$

Appendix

Table 36: BF16 removal–recovery trade-off, derived from Table 2 . Every difference is BUC minus sensitivity-only. Values are dimensionless metric differences, not accuracy percentage points.

Regime	Accuracy	PairAcc	MinPairAcc	Spearman $\rho(m,y_{\mathrm{NL}})$	Linear $R^{2}$	Mean $\lvert\Delta^{\mathrm{NL}}_{r}\rvert$
FRONT	$0.742\pm 0.019$	$0.556\pm 0.028$	$0.533\pm 0.031$	$0.879\pm 0.014$	$0.774\pm 0.022$	$0.93\pm 0.07$
BACK	$0.759\pm 0.018$	$0.581\pm 0.026$	$0.560\pm 0.029$	$0.891\pm 0.013$	$0.798\pm 0.021$	$1.07\pm 0.08$
INTERLEAVE	$0.812\pm 0.015$	$0.674\pm 0.024$	$0.652\pm 0.026$	$0.931\pm 0.009$	$0.863\pm 0.017$	$1.28\pm 0.06$
RANDOM	$0.773\pm 0.021$	$0.605\pm 0.031$	$0.585\pm 0.034$	$0.901\pm 0.015$	$0.814\pm 0.025$	$1.12\pm 0.09$
Mean	$0.772\pm 0.014$	$0.604\pm 0.021$	$0.583\pm 0.023$	$0.901\pm 0.010$	$0.812\pm 0.016$	$1.10\pm 0.06$

Appendix

Table 37: Behavioral validation on held-out domains and Bank B paraphrases. Accuracy, paired-query accuracy, and

\lvert\Delta^{\mathrm{NL}}_{r}\rvert

use the main gap

(4,12)

. Spearman correlation and linear

R^{2}

use the full count sweep. Values are mean and standard deviation over five seeds.

Gap	Semantic gap	Accuracy	PairAcc	Mean $\lvert\Delta^{\mathrm{NL}}_{r}\rvert$
$(0,16)$	$16$	$0.861\pm 0.014$	$0.748\pm 0.022$	$2.05\pm 0.11$
$(2,14)$	$12$	$0.829\pm 0.016$	$0.690\pm 0.025$	$1.62\pm 0.09$
$(4,12)$	$8$	$0.772\pm 0.014$	$0.604\pm 0.021$	$1.10\pm 0.06$
$(6,10)$	$4$	$0.646\pm 0.023$	$0.397\pm 0.030$	$0.46\pm 0.06$

Appendix

Table 38: Held-out gap generalization with the same model and no head reselection. Values are averaged over four ordering regimes and five seeds. The semantic gap is

\lvert b_{H}-b_{L}\rvert

Regime	$H_{\mathrm{NL}}\!\rightarrow\!\mathrm{NL}$ shrink/match	$H_{\mathrm{CEA}}^{\mathrm{yn}}\!\rightarrow\!\mathrm{NL}$ shrink/match	Random $\rightarrow\mathrm{NL}$ shrink/match	CEA-transfer SlopeRed	CEA-transfer AccDrop
FRONT	$0.694/0.731$	$0.391/0.425$	$0.057/0.070$	$0.373$	$6.9$
BACK	$0.721/0.758$	$0.427/0.462$	$0.054/0.069$	$0.411$	$8.2$
INTERLEAVE	$0.789/0.814$	$0.472/0.509$	$0.060/0.073$	$0.455$	$9.4$
RANDOM	$0.733/0.761$	$0.422/0.454$	$0.065/0.076$	$0.406$	$8.3$
Mean	$0.734/0.766$	$0.428/0.463$	$0.059/0.072$	$0.411$	$8.2$

Appendix

Table 39: Per-regime natural-language causal localization and fixed symbolic-to-natural-language transfer. All sets have budget

k=8

. The final two columns report the slope reduction and restricted-accuracy decrease caused by ablating the fixed symbolic CEA set on the natural-language target. Accuracy decrease is in percentage points.

Source set $\rightarrow$ target	$k$	Target shrink	Target match	SlopeRed
$H_{\mathrm{NL}}\rightarrow\mathrm{NL}$	$8$	$0.734\pm 0.024$	$0.766\pm 0.026$	$0.721\pm 0.028$
$H_{\mathrm{CEA}}^{\mathrm{yn}}\rightarrow\mathrm{NL}$	$8$	$0.428\pm 0.031$	$0.463\pm 0.034$	$0.411\pm 0.033$
Layer-matched random $\rightarrow\mathrm{NL}$	$8$	$0.059\pm 0.008$	$0.072\pm 0.009$	$0.051\pm 0.010$
$H_{\mathrm{CEA}}^{\mathrm{yn}}\rightarrow\mathrm{CEA}$	$8$	$0.812\pm 0.021$	$0.846\pm 0.023$	$0.834\pm 0.025$
$H_{\mathrm{NL}}\rightarrow\mathrm{CEA}$	$8$	$0.499\pm 0.029$	$0.539\pm 0.031$	$0.486\pm 0.033$
Layer-matched random $\rightarrow\mathrm{CEA}$	$8$	$0.050\pm 0.007$	$0.060\pm 0.008$	$0.043\pm 0.009$

Appendix

Table 40: Aggregate fixed-set transfer between symbolic CEA and the natural-language bridge. All values are averaged over four ordering regimes and five seeds. Base and post-ablation accuracy are shown only for the labeled natural-language target. Accuracy decrease is in percentage points.

Module	Comparison	Relation to the BF16 study
Operators	Fixed selected set under eight ablation/patch combinations	Operator-dependence audit
T3	Exhaustive local subsets, sampled larger subsets, greedy versus exact selection	Tests objective conditions, separately from the many-set T1 correlation
Baselines	Five methods and three random-control families at five common budgets	Extends the original budget-matched comparison
Generalization	Count, unseen gaps, cue wording, equal-budget regime transfer, order, and seeds	Tests count, order, cue, and seed dependence
Readouts	Random token-pair registry, fixed-prompt scan, and two reselection protocols	Extends the seven targeted readout pairs
Selection null	Full adaptive search under paired label swaps	Controls for adaptive selection

Appendix

Table 41: FP32 experimental comparisons and their relation to the BF16 study.

Setting	Value
Master seed	1729
Symbolic evidence and tail lengths	$L_{E}=200$ , $L_{A}=50$
Primary low/high counts	$(b_{L},b_{H})=(50,150)$
Selection / evaluation / calibration rows	48 / 96 / 48 per count and regime
Attribution shortlist / single-head pool	192 / 48 heads
Greedy maximum budget / weight	$K_{\max}=12$ , $\lambda=1/2$

Appendix

Table 42: FP32 experimental settings. Execution microbatches are specified separately in Appendix F .

Ablation	Directional patch	Donor dependence
Zero	Donor mean	Marginal means of calibration donors
Pooled mean	Donor mean	Pooled calibration ablation mean
Zero	Matched example	Corresponding opposite-side evaluation partner
Zero	Joint resampling	One donor row for all selected heads
Zero	Independent-head resampling	Separate donor row for each head
Zero	Empirical joint Gaussian	Joint mean and full empirical covariance

Appendix

Table 43: Operator combinations. Stochastic conditions use 12 draws; deterministic mean and exact matched-example conditions use one evaluation.

Split	Ordered state pairs
Selection	local/remote; legal/illegal; stable/unstable; known/unknown
Validation	inside/outside; positive/negative; enabled/disabled; above/below
Test	valid/invalid; safe/unsafe; alive/dead; connected/disconnected; hot/cold; approved/rejected; ready/unready; secure/insecure

Appendix

Table 44: Shared state-pair split derived from the supplemental master seed. The prompt declares the two named states mutually exclusive and exhaustive within each task; this is a task assumption, not a claim about unrestricted word meaning.

Bank	Direct support for $u_{c}$	Support for $u_{c}$ by negating $v_{c}$
A	Report $j$ : the item is $u_{c}$ .	Report $j$ : the item is not $v_{c}$ .
B	Record $j$ describes the item as $u_{c}$ .	Record $j$ rules out the item being $v_{c}$ .
C	Witness $j$ says that the item remains $u_{c}$ .	Witness $j$ says that the item is not $v_{c}$ .
D	Statement $j$ : $u_{c}$ is the reported condition.	Statement $j$ : $v_{c}$ is not the reported condition.

Appendix

Table 45: Exact supplemental report templates. Here

j

is the report number,

u_{c}

is the supported state, and

v_{c}

is the other state. Exchange the two states for negative-polarity reports.

Use	Domains / banks	Rows and seed convention
NL selection	4 / A	24 per count/regime; causal seed 1729
NL validation	4 / A	24 per count/regime; not used to choose heads
Behavior, original prompts	8 / B–D	9,216 per model/behavioral seed; three seeds
Behavior, four shortcut controls	8 / B–D	9,216 additional rows per model; main gap, first seed only
Causal target	8 / one of B–D	96 per count/regime/bank; causal seed 1729
Causal calibration	8 / matching bank	48 per count/regime/bank; separate generation key

Appendix

Table 46: Expanded NL sampling units. Rows count questions, including both query orientations; intervals retain the paired evidence units.

Setting	A100	L4	T4
Initial intervention branches per prompt	64	32	8
Logits-only NL behavior microbatch	16	8	2
Model precision	FP32	FP32	FP32
Shared execution and validation settings
Metric set block	256 sets
Prefix CPU cache budget	768 MiB

Appendix

Table 47: Hardware-specific execution defaults. Sample budgets are unchanged. Run metadata and engine configurations record the actual settings; imported measurements retain their original hardware provenance.

Files in the run’s tables/ directory	Recorded content
primary_buc.csv	Current-cohort reference sets and primary scores
operator_audit_summary.csv , operator_audit_draws.csv	Operator summaries and Monte Carlo draws
t3_local_summary.csv , t3_local_subsets.csv	Local conditions and complete subset values
t3_greedy_vs_exact.csv , t3_full_shortlist_sampled.csv	Selection/evaluation comparisons and larger-pool samples
fixed_budget_baselines.csv , fixed_budget_random_controls.csv	Five methods and three control families
paired_baseline_differences.csv	Matched method differences and uncertainty

Appendix

Table 48: Condition-level files generated by the FP32 implementation. Rows represent experimental units or conditions. The package coverage is specified in the accompanying text.

Work line	Model setting	Behavioral target	Causal criterion
Indirect-object identification and factual recall	Frozen pretrained models	Semantic task behavior	Study-specific interventions
Synthetic algorithm studies	Often task-trained models	Designed algorithm	Study-specific mechanism tests
Patching, ACDC, EAP	Model- and target-dependent	Researcher-supplied target	General localization machinery
CEA/BUC	Frozen pretrained models	Controlled count/order readout; controlled NL bridge	Same set tested by ablation and two-way patching

Appendix

Table 49: Boundary between nearby lines of work. “Same set” means that one selected set is evaluated by ablation and both mean-patching directions.

Quantity	Point estimate or span	95% bootstrap CI
BUC $-$ sensitivity, Front	$+0.006$	–
BUC $-$ sensitivity, Back	$+0.030$	–
BUC $-$ sensitivity, Interleave	$+0.031$	–
BUC $-$ sensitivity, Random	$+0.039$	$[+0.015,\,+0.065]$
Off-diagonal match: observed span	$-0.84$ to $+0.89$	–
Cue-change shrink: observed span	0.64–0.93	–

Appendix

Table 50: FP32 outcomes outside Tables 6 and 7 . A dash indicates that no CI is reported; it does not denote zero uncertainty. Only the RANDOM baseline difference and raw null-test utility receive bootstrap intervals here.

Transformer Interpretability

Transformer interpretability studies how attention, feed-forward layers, and hidden representations support model predictions and behavior.

See all 339 papers

Sep 30, 2026

When Do Attention-Head Ablations Support Causal Claims? Projection-Level Confounds, Floor Effects, and Matched Controls

Attention-head ablation, zeroing a head and measuring the resulting change in task performance, is a common method for inferring which components of a language model are causally responsible for a behavior. We show using GPT-2 small that this inference can be fragile unless the intervention semantics, evaluation metric, and controls are carefully validated. A natural post-projection implementation of "zeroing a head" is nearly uncorrelated with a corrected pre-projection ablation (Pearson r = 0.057) and selects a completely disjoint top-5 set of important heads. We also show that binary accuracy can hide effects at behavioral floors and near ceilings, whereas gold-token log-probability remains graded. Using a discovery/held-out split and 1,000 matched random-head and layer-matched-head control draws, the corrected per-head effect ranking is highly stable across splits (Spearman rho = 0.974), and the top-5 selected heads significantly exceed both control distributions (Monte Carlo p = 0.001). However, evidence for task specificity is not robust on GPT-2. Replication on DistilGPT2 preserves the intervention-semantic and matched-control findings. These results show that single-head ablation does not by itself justify a causal claim; defensible interpretation requires correct intervention placement, a non-saturated continuous metric, and matched held-out controls.

Juli Huang

Stanford University

Attention Head Analysis Causal Interventions in Language Models

Ask PDF

Jun 6, 2026

Ablation-Reversible Heads Don't Transfer: A Stress Test for Mechanistic Role Claims in Transformers

In mechanistic interpretability, attention heads are commonly elevated to role claims (e.g., "this head represents addition") when they are necessary for a behavior, encode it linearly, and recover that behavior when restored after ablation. We show this evidence is insufficient: across three 7-8B instruction-tuned models and five computation families, heads passing all three checks routinely fail to transfer the computation when their activations are patched into a different prompt under matched controls. We introduce KID (Knowing / Intent / Doing), a role-assignment lens for attention heads, and pair it with a three-stage pipeline: capability-selective screening (CSS), singular value decomposition (SVD), and activation transduction under matched controls. Our results document a preliminary role taxonomy (including prompt-trajectory stabilizers, answer-side logit-bias heads, and soft computation-pattern carriers) and show that the same-answer control (a transduction target sharing the answer string but not the requested computation) is an underused check that exposes broad state transfer masquerading as semantic specificity.

Philip Quirke

Martian

Attention Head Analysis Mechanistic Interpretability Causal Interventions in Language Models Activation Patching Attention Head Specialization

Ask PDF

Oct 13, 2026

Do Syntax Heads Influence Model Behavior?

Transformer language models develop attention heads that specialize in syntactic relations, for example, attending from verbs to their subjects. Do these heads play a causal role in the model's syntactic predictions? Evidence for such a role is mixed to negative: even when we force these heads to attend to a distractor noun other than the subject, the model's syntactic predictions are still more consistent with the subject than with the distractor. Interventions on specialized syntax heads had a significantly stronger effect on syntactic predictions than interventions on random heads, but only in one of the two measures we propose.

William Timkey, Tal Linzen

New York University

Transformer Interpretability Attention Head Specialization Language Modeling Causal Interventions in Language Models

Ask PDF

Abstract

Transformer Interpretability

Explore similar work

When Do Attention-Head Ablations Support Causal Claims? Projection-Level Confounds, Floor Effects, and Matched Controls

Ablation-Reversible Heads Don't Transfer: A Stress Test for Mechanistic Role Claims in Transformers

Do Syntax Heads Influence Model Behavior?