What Would Falsify It? A Variable Specific Evidence Standard for Mechanistic Claims About Self Explanation

Arshia Eftekhari zadeh

cs.AISep 26, 2026

What Would Falsify It? A Variable Specific Evidence Standard for Mechanistic Claims About Self Explanation

Authors: Arshia Eftekhari zadeh

Organizations: School of Electrical and Computer Engineering University of Tehran, Tehran, Iran

Abstract

When a language model explains an answer it has already given, does it reuse the computation that produced the answer or reconstruct a story from the answer alone? Attribution, transportability and recoverability are each compatible with causal use without establishing it. We propose an evidence standard: pair each positive statistic with a variable specific null that removes the tested variable's identity while matching relevant nuisance dimensions as far as possible, and audit unmatched dimensions. We apply this standard to a known cause. A cue naming a wrong option raises the rate of choosing that option by 64 to 68 percentage points across three models. Explanations mention the cue in 1.8 percent of items or fewer in three of four models tested. Three estimator classes yield favorable statistics, but none establishes causal sensitivity to the cue contrast under its own control in the three-model analysis. In the strongest case, a recovered cue direction reaches $R^2$ of 0.95 and exceeds a geometry matched random direction in all three seeds, while a direction fitted by the same pipeline with cue labels scrambled reproduces 61 to 76 percent of its effect at comparable realized edit magnitude. A fourth model passes one interchange endpoint, but unequal edit magnitudes and a contrast that changes both cue identity and cue-answer agreement limit its interpretation. These experiments leave causal access unresolved. They establish an evidentiary requirement: favorable mechanistic statistics must survive controls for variable identity and nuisance structure. Reusable controls separate generic from identity specific transport effects, fit null directions with scrambled labels, and audit realized intervention magnitudes.

Figures & tables

inferential step	positive statistic	nuisance alternative	falsifying control
attribution $\to$ access	a sparse set of components recovers most of the effect	generic component importance	components matched on layer, kind and generic importance recover as much
transport effect $\to$ semantic transport	patching decision pass activations moves the explanation readout	any foreign activation moves it, or positional misalignment	a donor carrying $C^{\prime}$ moves it by the same amount as a donor carrying $C$ , and a length matched neutral donor moves it too
recoverability $\to$ causal use	a direction recovers $C$ out of sample	encoded but inert, or site chosen by literal encoding	a direction fitted to shuffled cue labels, or one matched to the activation covariance, produces an equal effect

Table 1: The falsifying control matched to each inferential step’s nuisance alternative.

Panel of Figure 1 — Figure 1: The inferential ladder and its falsification controls. Solid arrows are established by construction and by decision side measurement. Dashed arrows are the three substitutions of Section 2.2, each with the control that would falsify it. The two endpoints of Section 2.5 read the final box independently and neither rescues the other.

model	flip rate	mention	$n_{\text{usable}}$	minority	required	power
Qwen3-1.7B	68.3pp	1.75%	1811	201	154	0.969
Llama-3.2-3B	63.8pp	0.43%	1680	336	296	0.940
Qwen3-8B	65.0pp	6.43%	1499	389	429	0.860

Table 2: Behavioral gate, authority channel, full pool. Mention is the adjudicated mention rate, minority and required are the counted and needed minority arm sizes, and power is design power. Usable

n

is smaller than the 2,116 eligible items because it counts only items on which the cue moved the answer and the explanation did not name the cue, which is the population the mechanistic tests are defined on.

model	cue $N$	cue $S$	cue excess (of 12)	Gate A	Gate B	failure mode
Qwen3-1.7B	+0.0000 [ $-0.0029$ , +0.0030]	$-0.0002$ [ $-0.0007$ , +0.0003]	0/12	fails 12/12	passes 12/12	no resolved target effect, ratio not substantively interpreted
Qwen3-8B	$-0.000009$ [ $-0.000145$ , +0.000130]	+0.000116 [+0.000008, +0.000224]	4/12	fails 10/12	passes 12/12	signal too small, magnitude imbalance
Llama-3.2-3B	+0.0028 [+0.0022, +0.0035]	$-0.0000$ [ $-0.0003$ , +0.0003]	5/12	fails 11/12	passes 12/12	signal not specific against shuffle
Additional model:
Llama-3.2-1B	$-0.0006$ [ $-0.0013$ , +0.0001]	+0.00047 [+0.00020, +0.00075]	6/12	passes 6/12	passes 12/12	criterion satisfied on interchange, audited magnitude confound

Table 3: Cross model summary, test splits, gates computed identically throughout. Llama-3.2-1B was run after the other three, and it is the one case in which the criterion is satisfied.

estimator	favorable evidence	specificity control	outcome	conclusion
component attribution	top- $k$ concentrates the effect (2.85 $\times$ on the decision pass)	matched random components, power gate	fails at 0.57 and 1.85 against 2.0	concentration $\not\Rightarrow$ access
whole component transport	donor changes the readout (0.19)	donor identity contrast, neutral against invalid sham	$G\neq 0$ , $C\approx 0$ , $\Delta_{\text{ID}}=-0.0017$ $[-0.041,+0.022]$	generic perturbation $\not\Rightarrow$ transport
multiclass cue direction	$R^{2}_{\text{cue}}$ 0.91 to 0.97 out of fold	shuffled label and covariance matched nulls	Gate A fails on all three models, passes on interchange in the fourth model	recoverability $\not\Rightarrow$ use in the main model set

Table 4: Each estimator’s favorable statistic and the control that falsified the inference built on it.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

arm	$G$ (generic)	$C$ (cue specific)	$\Delta_{\text{ID}}$
neutralized	$-0.0051$ $[-0.050,+0.035]$	$-0.0009$ $[-0.020,+0.011]$	$-0.0017$ $[-0.041,+0.022]$
invalid option sham control	$-0.0534$ $[-0.102,-0.013]$	$+0.0012$ $[-0.026,+0.022]$	$+0.0023$ $[-0.051,+0.045]$

Appendix

Table 5: Donor identity decomposition of the transport readout.

split	direction	necessity $N$	interchange $S$
development, $n=35$	cue	$-0.0019$ $[-0.0089,+0.0045]$	$-0.0009$ $[-0.0019,+0.0001]$
	shuffle	$-0.0094$ $[-0.0155,-0.0033]$	$+0.0000$ $[-0.0004,+0.0005]$
	covariance matched	$+0.0014$ $[-0.0069,+0.0090]$	$+0.0000$ $[-0.0001,+0.0002]$
selection, $n=58$	cue	$-0.0030$ $[-0.0070,+0.0012]$	$-0.0009$ $[-0.0018,+0.0001]$
	shuffle	$-0.0005$ $[-0.0047,+0.0038]$	$-0.0004$ $[-0.0011,+0.0002]$
	covariance matched	$+0.0063$ $[+0.0013,+0.0115]$	$+0.0000$ $[-0.0001,+0.0001]$

Appendix

Table 6: Necessity and interchange effects, cue direction and nulls, development and selection splits.

direction	necessity $N$	interchange $S$
cue	$+0.0000$ $[-0.0029,+0.0030]$	$-0.0002$ $[-0.0007,+0.0003]$

Appendix

Table 7: Cue direction’s necessity and interchange effects, test split.

null	seed	necessity $N$	interchange $S$
shuffle	0	$-0.0022$ $[-0.0053,+0.0010]$	$-0.0002$ $[-0.0005,+0.0001]$
shuffle	1	$-0.0026$ $[-0.0055,+0.0005]$	$-0.0001$ $[-0.0004,+0.0001]$
shuffle	2	$-0.0010$ $[-0.0031,+0.0011]$	$-0.0001$ $[-0.0003,+0.0001]$
covariance matched	0	$+0.0052$ $[+0.0020,+0.0086]$	$-0.0000$ $[-0.0001,+0.0001]$
covariance matched	1	$+0.0055$ $[+0.0024,+0.0086]$	$-0.0000$ $[-0.0001,+0.0000]$
covariance matched	2	$+0.0030$ $[-0.0002,+0.0061]$	$-0.0000$ $[-0.0001,+0.0000]$

Appendix

Table 8: Null direction necessity and interchange effects by seed, test split.

comparison	seed	necessity $N$	interchange $S$
cue $-$ shuffle	0	$+0.0022$ $[-0.0012,+0.0059]$	$+0.0000$ $[-0.0004,+0.0005]$
cue $-$ shuffle	1	$+0.0026$ $[-0.0012,+0.0063]$	$-0.0000$ $[-0.0005,+0.0004]$
cue $-$ shuffle	2	$+0.0011$ $[-0.0023,+0.0045]$	$-0.0001$ $[-0.0005,+0.0004]$
cue $-$ covariance matched	0	$-0.0052$ $[-0.0097,-0.0006]$ (CI excludes zero)	$-0.0002$ $[-0.0007,+0.0003]$
cue $-$ covariance matched	1	$-0.0055$ $[-0.0099,-0.0013]$ (CI excludes zero)	$-0.0001$ $[-0.0007,+0.0004]$
cue $-$ covariance matched	2	$-0.0029$ $[-0.0068,+0.0009]$	$-0.0002$ $[-0.0007,+0.0004]$

Appendix

Table 9: Cue minus null specificity per realization, test split.

split	cells exactly zero	interchange (all directions)	necessity nonzero: cue / shuffle / covariance matched
development, $n=35$	35/36 (development check)	[0, 0]	not recorded
selection, $n=58$	338/348	[0, 0]	2 / 4 / 4
test, $n=200$	1126/1123/1122 of 1164 (seeds 0, 1, 2)	[0, 0] shuffle and covariance matched, cue $+0.0067$ $[+0.0000,+0.0202]$	12/11/14 (seed 0), 12/14/13 (seed 1), 12/7/21 (seed 2)

Appendix

Table 10: Observable endpoint cell counts. For the test split the nonzero cue interchange cell is a single item with the same value in all three seeds, and necessity nonzero counts are given per seed (s0, s1, s2).

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 778 papers

May 31, 2026

Not All Explanations Simulate Equally: Comparing Verbalized Feature Attributions and Self-Generated Rationales

Natural-language explanations are often treated as a unified interface for understanding model behavior, but different explanation sources may support simulation in different ways. This paper compares two families of explanations for question answering models: verbalized feature attributions and self-generated rationales. We evaluate them under a shared counterfactual simulation setting, using an LLM judge as predictor and measuring whether it can better predict a model's answers to follow-up questions when given its explanation. Across multiple instruction-tuned models, we analyze how explanation source, verbalization strategy, and feature granularity affect the simulatability of explanations. Our results show that explanation format and granularity affect simulatability: attribution-based explanations and self-generated rationales differ in how much they improve counterfactual prediction, with effects that vary across models and formats.

Pingjun Hong, Benjamin Roth

Faculty of Computer Science, University of Vienna, Vienna, Austria · UniVie Doctoral School Computer Science, University of Vienna, Vienna, Austria · Faculty of Philological and Cultural Studies, University of Vienna, Vienna, Austria

Feature Attribution LLM Interpretability Explainability Evaluation

Ask PDF

Sep 15, 2026

An Empirical Study of Counterfactual Self-Explanations in LLMs

Large language models can easily generate explanations for their own outputs, but such self-explanations are not necessarily faithful to the model's behavior. We study this issue through counterfactual self-explanations, where a model minimally edits an input so that its own prediction changes. Across sentiment analysis and natural language inference, we evaluate ten instruction-tuned models from the LLaMA-3 and Qwen-2.5 families, measuring faithfulness, minimality, and alignment with human-annotated rationales. Our results show that model scale is the strongest determinant of explanation quality: larger models are substantially more likely to generate counterfactuals that flip their own predictions and target decision-relevant evidence. In contrast, the rationale-guided condition produces edit-minimal counterfactuals that are also more human-aligned. However, it does not consistently improve faithfulness. Overall, counterfactual self-explanations can provide useful behavioral evidence about model decisions, but their reliability depends strongly on model capacity and should be empirically validated rather than assumed.

Giannis Kalyvas, Giorgos Filandrianos, Orfeas Menis Mastromichalakis +2

National Technical University of Athens, Athens, Greece · Instituto de Telecomunicações, Lisbon, Portugal

Counterfactual Explanations Faithfulness of Language Model Explanations LLM Interpretability Counterfactual Reasoning in Language Models LLM Reliability

Ask PDF

May 8, 2026

Position: Mechanistic Interpretability Must Disclose Identification Assumptions for Causal Claims

Mechanistic interpretability papers increasingly use causal vocabulary: circuits, mediators, causal abstraction, monosemanticity. Such claims require explicit identification assumptions. A purposive audit of 10 papers across four methodological strands finds no dedicated identification-assumptions section and a recurring pattern: validation metrics such as faithfulness, completeness, monosemanticity, alignment, or ablation effects are reported as causal support without stating the assumptions that make them identifying. A two-human-coder audit on

n=30

reproduces the direction of the main finding: dedicated identification sections are absent, and validation-metric substitution is common, though exact Dim B/D counts are coding-rule sensitive. The paper proposes a disclosure norm: state whether the claim is causal, name the identification strategy, enumerate assumptions, stress at least one, and explain how conclusions shift if assumptions fail. Validation is not identification.

Zezheng Lin, Fengming Liu

Preprint.

Mechanistic Interpretability Causal Identifiability Causal Interpretability Explainability Evaluation

Ask PDF

Abstract

LLM Interpretability

Explore similar work

Not All Explanations Simulate Equally: Comparing Verbalized Feature Attributions and Self-Generated Rationales

An Empirical Study of Counterfactual Self-Explanations in LLMs

Position: Mechanistic Interpretability Must Disclose Identification Assumptions for Causal Claims