Explanation Multiplicity: Circuit-Level Interpretability Evidence Does Not Survive Defensible Analytic Variation

Ajay Pravin Mahale

cs.AIAug 13, 2026

Explanation Multiplicity: Circuit-Level Interpretability Evidence Does Not Survive Defensible Analytic Variation

Authors: Ajay Pravin Mahale

Organizations: Hochschule Trier

Abstract

The EU AI Act requires providers of high-risk systems to file technical documentation describing how the system reaches its decisions. Mechanistic interpretability is the obvious source of such evidence, and circuit discovery is its most developed instrument. We ask whether that evidence survives the condition under which it would be relied upon: two competent analysts, the same system, the same tool, different defensible settings. We pre-registered a crossed grid of seven analytic axes, every level taken from a published implementation, and mapped each discovered circuit through a deterministic claim map to a structured Annex IV statement. Across 15,840 pre-registered specifications on GPT-2 small and the indirect object identification task, of which 7,561 produced a claim, the derived statement flips across 73.2% of specification pairs (95% CI 0.725 to 0.738) and the modal claim commands 41.1% of the space. The evidence fails a filability criterion at every tolerance a conformity assessment body would plausibly accept. Standardising the single most influential choice, the evaluation metric, leaves the flip rate at 59.4%. Removing circuit size from the claim entirely and holding it fixed leaves 27.1% (95% CI 0.255 to 0.286), still above the pre-registered threshold. The circuits underlying these claims are structurally near-disjoint, median pairwise Jaccard overlap 4%, and functionally uncorrelated at Cohen's kappa 0.015, so the instability is not one mechanism described in different words. We give the filability criterion as a standalone protocol, and we report that one of the seven documented discovery objectives does not execute at all on the library's own canonical task. The study covers one model and one task, and whether the conclusion holds at scale is untested.

Figures & tables

Binning	$F$
committed bins	0.7316
all thresholds $\times$ 0.5	0.7448
all thresholds $\times$ 0.8	0.7435
all thresholds $\times$ 0.9	0.7394
all thresholds $\times$ 1.1	0.7289
all thresholds $\times$ 1.25	0.7314

Table 1: Flip rate under alternative size binnings. The committed bins are not a maximum: four alternatives give a higher

F

, including halving every threshold. Across binnings differing by a factor of four the range is 0.70 to 0.77, and removing the size term from the claim entirely still leaves 0.5733.

Metric	Kept	Discarded	Rate
comprehensiveness	3,947	13	0.003
KL divergence	1,467	2,493	0.630
logit difference	1,270	2,690	0.679
sufficiency	877	3,083	0.779

Table 2: Discard rate by evaluation metric under the pre-registered metric-relative threshold rule. The unevenness was disclosed before the pre-registration was locked, from a six-cell validation slice.

	Decision rule	Result	Outcome
P0	$\bar{J}$ reported, not tested	0.1396 [0.1377, 0.1418]	premise holds
H2	$F>0.20$ , CI lower bound above	0.7316 [0.7247, 0.7380]	confirmed
H3	not separated from random null	separated, intervals disjoint	rejected
H4	$F-(1-\text{agreement})>0.10$	0.3344 [0.3297, 0.3394]	supported

Table 3: Pre-registered outcomes and their results. Thresholds were fixed on 6 August 2026 and the plan was tagged before the sweep began. Two results carry qualifications that only the within-size decomposition reveals, and those are reported in Sections 7.5 and 7.6 rather than deferred to an appendix.

Rank	Layer band	Size class	Count	Share
1	early layers	distributed	3,107	0.4109
2	late layers	sparse	2,153	0.2848
3	late layers	moderate	657	0.0869
4	late layers	distributed	489	0.0647
5	early layers	sparse	477	0.0631
6	middle layers	moderate	283	0.0374

Table 4: The nine claim classes produced by

\varphi_{\text{overseer}}

at medium granularity, ordered by frequency. Aggregating over size class, the layer attribution splits 47.9% early, 43.6% late and 8.5% middle; the first of these is the COARSE modal share of 0.4788, computed independently.

Addressee	Granularity	Classes	$F$	$\pi^{*}$	Filable
affected	COARSE	2	0.3950	0.7291	no
affected	MEDIUM	4	0.6597	0.4875	no
affected	FINE	8	0.6744	0.4786	no
overseer	COARSE	3	0.5733	0.4788	no
overseer	MEDIUM	9	0.7316	0.4109	no
overseer	FINE	12	0.7681	0.4109	no

Table 5: Flip rate and modal share for both addressees at all three granularities. Filability is evaluated at the loosest pre-registered tolerance,

\alpha=0.20

, which requires

\pi^{*}\geq 0.80

. No cell passes.

Figure 1: Specification curve for $\varphi_{\text{overseer}}$ at medium granularity. Upper panel: claim class against specification rank, classes ordered by frequency, so each step’s width is that class’s share of the space. The dashed line marks the size-matched random null’s modal share; the pooled comparison is confounded by circuit size, discussed in Section 7.5 . Lower panel: which analytic level is active for each specification. The solid band on the comprehensiveness row is the composition effect of the discard rule in Table 2 ; the seed rows are unstructured, which is the same result as seed removing 0.0009 from $F$ in Table 6 .

Axis standardised	Residual $F$	95% CI	Removes
Evaluation metric	0.5939	0.5803 to 0.6063	0.1377
Ablation operator	0.7014	0.6939 to 0.7075	0.0302
Discovery objective	0.7018	0.6938 to 0.7084	0.0298
Threshold $\tau$	0.7070	0.6981 to 0.7149	0.0246
Corruption	0.7092	0.7008 to 0.7166	0.0224
Prompt variant	0.7232	0.7157 to 0.7300	0.0084

Table 6: Residual flip rate once each axis is standardised, computed with Equation 3 . All intervals are valid: group sizes 1,512 to 2,520, bootstrap bias below 0.0006 on every axis, and every observed value lies inside its own interval.

Size	$n$	Discovered $F$	Classes	Null $F$	Null classes
10	2,492	0.4543	3	0.7331	6
20	489	0.3380	4	0.6622	3
50	519	0.4639	3	0.7310	6
100	242	0.5792	3	0.6584	3
200	199	0.5122	4	0.6366	3
500	236	0.4258	3	0.1421	3

Table 7: Flip rate within each circuit size, medium granularity. The null collapses to a single claim class at and above 1,000 edges, which is what depresses its pooled flip rate. The discovered multiverse also collapses at 5,000 and 10,000 edges, so 2,692 of 7,561 specifications contribute no flips at all and the instability lives entirely below 2,000 edges.

Mechanistic Interpretability

Mechanistic interpretability studies how neural networks implement behaviors through internal features, circuits, and causal information flow.

See all 391 papers

Rethinking Circuit Evaluation: Do Circuits Explain Model Errors?
Sep 28, 2026Li Zhang, Chuqin Geng, Mark Zhang +4Faithfulness of Language Model Explanations Mechanistic Interpretability
University of Toronto · McGill University · Tsinghua University
Demystifying Variance in Circuit Discovery of LLMs
Jun 15, 2026Frank Zhengqing Wu, Francesco Tonin, Volkan CevherPrompt Sensitivity Circuit Discovery
Laboratory for Information and Inference Systems (LIONS) École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland · Laboratory for Informationand Inference Systems (LIONS) École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
Certified Circuits: Stability Guarantees for Mechanistic Circuits
Feb 26, 2026Alaa Anani, Tobias Lorenz, Bernt Schiele +2Circuit Discovery Mechanistic Interpretability
Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany · CISPA Helmholtz Center for Information Security, Saarbrücken, Germany

Sep 28, 2026J/K move · Enter open · S save

Rethinking Circuit Evaluation: Do Circuits Explain Model Errors?

Li Zhang, Chuqin Geng, Mark Zhang, Chen Yang+3

University of Toronto · McGill University · Tsinghua University

Faithfulness of Language Model Explanations Mechanistic Interpretability Circuits in Language Models Explainability Evaluation

Open paper AskPDF arXiv

Mechanistic interpretability (MI) aims to explain a model's behaviour through analyzing its internal computations; circuit-based explanations aim to isolate these computations with compact subnetworks validated by ablating the rest of the model. We show that circuits validated this way may fail to recover the underlying mechanism of the model's behaviour by closely reproducing its successful decisions while failing to account for most of its errors. Such explanations should account for the model's particular errors as well as its successes. We evaluate this requirement by measuring exact answer agreement separately on model successes and failures, across circuit sizes and ablation settings, on IOI, Docstring, and six model-task settings from the Mechanistic Interpretability Benchmark. We discover that many tested circuits closely replicate correct behaviour while missing most of the model's errors. On indirect object identification (IOI) for GPT-2 small, under mean ablation, the manual circuit and tested automated circuits, including one trained against the model's full output distribution, agree with the model on 97.3-99.5% of prompts it answers correctly but only 11.4-41.7% of errors. An IOI case study shows that lost errors are recoverable by restoring omitted attention-heads which raise error reproduction from 14.2% to 75.1% on a separate held-out set with 0.41 percentage point decrease on correct agreement, exceeding matched random extensions and scalar-biased control. Intervention traces show how omitted computations produce specific wrong answers for a reproducible subset of errors. In all, these findings show circuits can preserve task success without adequately explaining model's failures, and support exact error reproduction as a necessary, but not sufficient, test of circuit-based explanations of model behaviour.

Abstract

Mechanistic Interpretability

Explore similar work

Rethinking Circuit Evaluation: Do Circuits Explain Model Errors?

Demystifying Variance in Circuit Discovery of LLMs

Certified Circuits: Stability Guarantees for Mechanistic Circuits