Same Behavior, Different Mechanism: What Finite Experiments Can Certify About Integrated Information

Linqian Wu; Diaoxuan He

cs.LGNov 5, 2026

Same Behavior, Different Mechanism: What Finite Experiments Can Certify About Integrated Information

Authors: Linqian Wu, Diaoxuan He

Organizations: University of Waterloo, Waterloo, Ontario, Canada · Perimeter Institute for Theoretical Physics, Waterloo, Ontario, Canada · University of Debrecen, Debrecen, Hungary

Abstract

Interpretability, AI auditing and assessments of machine consciousness increasingly rest on mechanism-level quantities: functionals of a system's causal transition law that describe how it computes, not what it outputs. Experiments see that law only through finite, partial and noisy interventions. We ask when such a quantity is identifiable from an experiment's access, continuous in the mechanism and certifiable from finite data, and answer exactly for a deliberately demanding case - the integrated information $\varphi$ of integrated information theory - in stochastic networks of four and five units with known ground truth. Behavior can be silent: a recurrent and a feed-forward network with identical input-output laws at their response delay carry 0.28 and 0 bits, yet internal recordings separate them within 4-53 trials. Partial access can be just as blind: in our examples, units left out of a one-step recording impose precision floors of 0.30-1.02 bits that no sample size lowers, and in all-but-self parity networks, recordings of any length see several hidden units only through the product of their fidelities - a symmetry that one intervention broke in exact checks. The measure has sharp edges: because $\varphi$ selects a partition, and the cause and effect it specifies, by one criterion and reports another, infinitesimal changes of mechanism that cross an edge change it by 25-100% at 99% of the edges we found in random networks, and about 1% of surrogate models that fit interventional data almost perfectly misreport it by more than 20%. One-sided claims survive: the same data certify a lower bound on both sides of an edge, and we prove $\varphi>1.004$ bits throughout a 64-dimensional region of mechanisms and certify it from about $10^7$ controlled transitions with false-certificate probability at most $α$ . Measurability is thus a design question with computable answers.

Figures & tables

Figure 1: Three barriers and one way through. (A) A recurrent network and a feed-forward chain with identical input–output laws at their response delay differ in integrated information $\varphi$ . (B) An experiment prepares known states, records some units after some delay, infers the mechanisms compatible with the data and reports a claim about $\varphi$ . Three barriers limit what any such experiment can establish; robust claims can still be certified. (C) Input–output data carry no evidence about the difference in A at any number of trials, whereas recording internal units yields evidence at every trial (expected evidence between the two fully specified systems; Section 3 ). (D) When one unit is hidden from a one-step recording, every point on the curve and the red dot (the exact four-way tie; $\varphi$ from $0$ to $1.13$ bits) is compatible with the same recordings, so a valid $95\%$ interval must contain each of these values with probability at least $0.95$ and has expected width of at least $0.9\times 1.129=1.016$ bits, whatever the sample size ( Section 4 ). (E) Over a two-dimensional slice through one weight and one bias of a random mixed-sign network (darker: larger $\varphi$ ; same slice as Figure 4 B), $\varphi$ forms a mosaic of tiles separated by cliffs (white: partition switches; dashed: a switch of the specified states). An arbitrarily small change of mechanism that crosses a cliff changes $\varphi$ by $25$ – $100\%$ at $99\%$ of the edges we found, and a surrogate trained near an edge misreported $\varphi$ by a factor of $2.5$ – $2.7$ for about a thousand gradient steps ( Section 5 ). (F) Exact probability of certifying $\varphi>1.004$ bits with simultaneous Clopper–Pearson intervals, against the number of controlled transitions, for sources at the center (solid) and halfway to the boundary (dashed) of the proved region (green), and for a source one cell outside it (grey; boundary sources are likewise never certified). False certificates have probability at most $\alpha$ ( Section 7 ).

	Cause	What does not help	What helps
Black boxes	behaviorally equivalent mechanisms differ in $\varphi$	more input–output trials (zero evidence)	recording internal units: expected log-likelihood ratio $\ln 20$ (20:1) in 4–53 trials (two specified candidates)
Hidden units	marginalization; in all-but-self parity networks, only the product of hidden fidelities reaches the recordings	more trials; with several hidden units, longer recordings of the same units	recording more units; later readouts; interventions on hidden units
Sharp edges	the measure selects by one criterion and reports another	point estimates; plug-in values from well-fitting models	warranted structural knowledge; reporting the lower envelope
Certificates	a theorem over a region of mechanisms, or a lower envelope	—	one-sided claims: confidence set inside the region, false certificates $\leq\alpha$

Table 1: Three barriers and one way through. What fails, why, and what works, for inferring a mechanism-level quantity from finite interventions.

Figure 2: Same function, different mechanism. (A) The recurrent parity network ( $\varphi=0.28$ bits) and the feed-forward chain ( $\varphi=0$ ) unrolled over two updates. Highlighted arrows (red in the recurrent network, dark in the chain) trace how the input (unit 1 at $t$ ) reaches the output: through every other unit in the recurrent network, along a single path in the chain. (B) Exact probability that a unit is on, from each of the 16 initial states, grouped by the input bit. Top: the output (unit 1 of the recurrent network, unit 3 of the chain) at $t{+}2$ coincides in the two systems. Bottom: interior unit 2 at $t{+}1$ differs. (C) Expected log-likelihood ratio (nats) in favor of the recurrent network, when it is the true system, as trials accumulate, for four internal recordings (sensor error $0.10$ ; both systems known; balanced interventions). Markers show where the expected log-likelihood ratio reaches $\ln 20$ (odds of 20:1), labeled with the number of trials. Input–output data give zero evidence at any number of trials.

Figure 3: Hidden units: floors and symmetries. (A) Three designs record the four-unit parity network after the same intervention; the dashed unit is never recorded. (B) Mean width of $95\%$ intervals for $\Delta\varphi$ between two conditions that differ only in the hidden unit (256 paired datasets per design and budget), and how often the sign of $\Delta\varphi$ was resolved. Open and filled markers: budgets of 32,768 and 131,072 per condition; dashed line: the true $\Delta\varphi=0.24$ bits, for scale. (C) With recorded reliabilities fixed at $421/512$ , every hidden reliability on the horizontal axis gives the same one-step recording of units 1–3, so every value of $\varphi$ on the curve and at the red dot (the exact four-way tie), from $0$ to $1.13$ bits, is compatible with the data. A valid $95\%$ interval must contain each of these values with probability at least $0.95$ , which gives the floor in D. (D) Lower bounds on the expected width of any $95\%$ interval ( Proposition 2 ) for recorded reliabilities $421/512$ , with and without the exact four-way tie, and for two sets of distinct recorded reliabilities; no number of trials lowers them. Dashed line: $B_{4}=1.21$ bits, the largest possible $\varphi$ for four units. (E) $\varphi$ (all-zero state) of a five-unit network with recorded reliabilities $(0.78,0.80,0.83)$ over the fidelities $a_{4},a_{5}$ of its two hidden units. Observation reveals only the product $a_{4}a_{5}$ (white curves: $a_{4}a_{5}=0.2$ , $0.32$ (thick; through configurations a and b), $0.5$ and $0.7$ ; Theorem 3 ). (F) Total-variation distance between the recorded trajectories of a and b (equal product) and of a pair with a different product, as the recording is extended. Equal-product trajectories are identical by Theorem 3 ; the residual $10^{-16}$ is floating-point round-off. Setting one hidden unit once (diamonds: unit 4 filled, unit 5 open) separates a from b. (G) $\varphi$ along curves of constant product. The shaded band is the tie-free spread of $\varphi$ along $a_{4}a_{5}=0.32$ ; at the two exact ties on that curve, where a hidden fidelity equals that of recorded unit 1, $\varphi=0.380$ bits (red dots).

Figure 4: Sharp edges are common. (A) $\varphi$ (solid) and $\varphi_{\mathrm{raw}}$ (dashed) of the parity network as one unit’s reliability varies with the other three at $421/512$ ; left of the tie the two coincide. At the four-way tie $\varphi=1.13$ bits (filled dot); an arbitrarily small decrease gives $0.33$ bits (open dot, the one-sided limit), and larger decreases give less. Right: over the same $10^{-4}$ perturbation the full cause–effect structure ( $\Phi$ , summed over all its distinctions and relations) changes by $0.02\%$ and the selected system not at all. (B) $\varphi$ and $\varphi_{\mathrm{raw}}$ over a two-dimensional slice through one weight and one bias of a random mixed-sign network (dot: the network). White lines mark discontinuities from partition selection, where the selected cut changes its number of severed connections; the dashed line marks a switch of the specified states, across which $\varphi$ and $\varphi_{\mathrm{raw}}$ both jump to zero. (C) Sizes of 849 discontinuities found on 1,976 random parameter paths through the 988 states with $\varphi>0$ of 317 random networks, relative to the larger side, for excitatory (light) and mixed-sign (dark) networks side by side. Partition switches sit exactly at $1-e_{a}/e_{b}$ for the edge counts of the competing cuts ( Proposition 4 ); jumps to or from zero come from switches of the specified states. Two edges lie off this lattice: a $4{:}9$ switch at which the cap binds ( $32.6\%$ ) and a simultaneous state and $3{:}3$ partition switch ( $31.5\%$ ). (D) Fraction of random paths (two per state with $\varphi>0$ ) that cross an edge within a given parameter distance of the state.

Figure 5: A good fit is not a good measurement. (A) Mean absolute error of $\varphi$ estimated from 65,536 complete transitions of the parity network, assuming four free reliabilities or one shared reliability, when all four are equal (left) and when one is lowered by $10^{-4}$ (right); 256 datasets each. (B) Full-batch maximum-likelihood training of a surrogate network on $2^{16}$ interventional transitions from a mixed-sign target chosen to lie within $0.03$ of an edge. Top: predictive divergence from the target. Bottom: $\varphi$ (solid) and $\varphi_{\mathrm{raw}}$ (dashed) of the surrogate and the true $\varphi$ (dotted); the shaded window marks where the surrogate’s $\varphi$ is $2.5$ – $2.7$ times the true value. (C) Relative error of the plug-in values against the fit’s predictive divergence, for 100 random excitatory networks, five sample sizes and 64 fits each. Dots: plug-in $\varphi$ of each fit; lines: median and 99th percentile of the relative error of the plug-in $\varphi_{\mathrm{raw}}$ . (D) Probability of a relative error above $20\%$ against the number of transitions, for $\varphi$ (dark) and $\varphi_{\mathrm{raw}}$ (blue) in excitatory (solid, circles) and mixed-sign (dashed, squares) networks; zero failures among 6,400 fits are not drawn on the log scale.

Figure 6: From attainable to delivered evidence. (A) Coverage of the true

\varphi

(heterogeneous source, sensor error

0.10

, 32,768 recording and 128 calibration trials) when sensor error is ignored, plugged in as an estimate, or propagated jointly with recording uncertainty. Bars show coverage (31, 89 and 256 of 256 datasets), whiskers are exact

95\%

binomial intervals, and the dashed line is the nominal

95\%

. (B) Mean interval width on identical two-step data with the Box and Joint procedures for four sources (216 cells, 55,296 datasets); blue line: four recorded units on their own data, averaged over all 216 cells (all sources pooled). (C) Mean interval width under a correct model (finest grid) against the rate of rejecting a wrong model (coarse grid; a recorded unit’s bias shifted by

0.20

), for eight random logistic networks (Bias family, hidden bias

0.25

, budget 131,072). (D) Rejection rate against the size of the unmodeled shift (colors as in C); the dotted line is the bound

\alpha=0.05

on rejecting a correct model.

Figure 7: One-sided claims can be certified. (A) A theorem guarantees

\varphi>1.004

bits for every law in a 64-dimensional region. If a simultaneous confidence set lies inside the region, the claim is certified; if it crosses the boundary, no claim is made. (B) The region and the rule drawn to scale for one of the 64 probabilities: four test sources (top) and the ranges that every estimate must fall in for certification, with Hoeffding (grey) and exact Clopper–Pearson (green) intervals. (C) Exact probability of certification against the number of controlled transitions for the center (solid) and halfway (dashed) sources, with exact Clopper–Pearson (green) and Hoeffding (grey) intervals. Circles: Monte Carlo simulations with the Hoeffding rule (256 datasets each; filled, center; open, halfway); crosses: boundary and outside sources, never certified (0/256). (D) The four uniform lower bounds in the proof: selected raw loss, cause and effect specification, and differentiation (

-\log_{2}

of the largest specified probability). The smallest sets the certified value (filled: the two bounds within

0.01

bits of it). No candidate of at most three units can exceed one bit (grey).

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Quantity	Partition selection	Role
$\varphi$ (target)	minimum loss per severed connection; ties to the larger raw loss	IIT 4.0 system integrated information
$\varphi_{\mathrm{raw}}$	minimum raw loss	comparison, not part of the theory; continuous in the parity family, jumps only where the specified states switch
$\varphi_{\tau}$	as $\varphi$ , with comparison tolerance $10^{-13}$	values saved by software
$\Phi$	—	sum over the full cause–effect structure (all distinctions and relations)

Appendix

Table 2: Selection and reporting rules. All rows share the transition law, the allowed cuts and the cap.

Case	$\varphi$ (this work)	$\varphi$ (earlier implementation)	$\varphi_{\mathrm{raw}}$
$\delta=0$	1.129294308	1.129294308	0.328078083
$\delta=2^{-24}$	0.328078011	0.328078011	0.328078011
$\delta=2^{-16}$	0.328059756	0.328059756	0.328059756
$\delta=15/4096$	0.323686219	0.323686219	0.323686219
$\delta=1/32$	0.291027385	0.291027385	0.291027385
$(0.78,0.80,0.83,0.85)$	0.282429091	0.282429091	0.282429091

Appendix

Table 3: Reference values of

\varphi

in the first parity network (all states).

\delta

is the decrease of one reliability from

421/512

Design	Recorded	Cost per trial	Installation
3 units, $t{+}1$	units 1–3, one update after the intervention	$2+3c$	0
4 units, $t{+}1$	all four units, one update after	$2+4c$	512
3 units, $t{+}2$	units 1–3, two updates after	$3+3c$	0

Appendix

Table 4: Recording designs in the first parity network.

Budget	Design	Sign resolved	Unresolved	Mean width of $\Delta\varphi$ interval (bits)
32,768	3 units, $t{+}1$	0	256	2.41
32,768	4 units, $t{+}1$	256	0	0.98
32,768	3 units, $t{+}2$	0	256	1.62
131,072	3 units, $t{+}1$	0	256	2.21
131,072	4 units, $t{+}1$	256	0	0.43
131,072	3 units, $t{+}2$	256	0	0.97

Appendix

Table 5: Directional contrasts between two conditions that differ only in the hidden unit (recorded reliabilities

(0.78,0.80,0.83)

;

\Delta\varphi=0.2433

bits; 256 paired datasets per row; budgets are per condition). Every interval covered the true difference and none supported the wrong sign.

Procedure	Mean width (bits)	Min. cell coverage
Moment	1.112	248/256
Box	1.138	256/256
Joint	0.914	247/256
4 units, $t{+}1$ (own data)	0.693	255/256

Appendix

Table 6: Two-step inference on identical data (55,296 datasets, 216 equally weighted cells). Left: overall. Right: Joint by source, and its change relative to Box.

Design	Mean width (bits)	Coverage	Reject, shift 0.05	Reject, 0.10	Reject, 0.20
3 units, $t{+}1$	0.214	0.976	0.067	0.228	0.952
4 units, $t{+}1$	0.034	0.986	0.033	0.153	0.851
3 units, $t{+}2$	0.035	0.993	0.001	0.011	0.120

Appendix

Table 7: Eight random logistic networks, hidden bias

0.25

, budget 131,072 (Bias family). Widths and coverage at the finest grid; rejection rates for unmodeled shifts of a recorded unit’s bias at the coarse grid (equal-cost designs).

Source	$N=2^{23}$	$2^{25}$	$2^{27}$
center	2/256	256/256	256/256
halfway	0/256	30/256	256/256
boundary	0/256	0/256	0/256
one cell outside	0/256	0/256	0/256

Appendix

Table 8: Certification of

\varphi>1.004

bits. Left: Monte Carlo results with the Hoeffding rule (256 datasets per cell); every one of the 3,072 confidence sets contained its true law. All four test sources have

\varphi>1.004

at every state, so these runs measure power and conservativeness; false certificates are bounded by

\alpha

( Lemma B.8 ). Right: transitions needed for

95\%

certification probability under three exact confidence constructions.

Source	true $\varphi$	$N$	mean bound	smallest bound	bound valid	plug-in MAE
four equal reliabilities	1.129	$2^{12}$	0.288	0.265	256/256	0.798
		$2^{16}$	0.318	0.312	256/256	0.801
		$2^{20}$	0.326	0.324	256/256	0.802
one lowered by $10^{-4}$	0.328	$2^{12}$	0.287	0.266	256/256	0.015
		$2^{16}$	0.318	0.313	256/256	0.004
		$2^{20}$	0.325	0.324	256/256	0.002

Appendix

Table 9: One-sided certificates at the edge of the parity network (256 datasets per row). The certified lower bound is valid in every dataset on both sides of the edge, while the plug-in estimate with four free reliabilities fails at the tie.

$N$	median KL (nats)	$\varphi$	$\varphi_{\mathrm{raw}}$	$\varphi$	$\varphi_{\mathrm{raw}}$
		Excitatory		Mixed-sign
$2^{10}$	$9.5\times 10^{-3}$	0.164	0.098	0.541	0.489
$2^{12}$	$2.4\times 10^{-3}$	0.045	0.003	0.303	0.257
$2^{14}$	$5.9\times 10^{-4}$	0.020	0	0.114	0.088
$2^{16}$	$1.5\times 10^{-4}$	0.013	0	0.032	0.019
$2^{18}$	$3.7\times 10^{-5}$	0.008	0	0.010	0.002

Appendix

Table 10: Learned surrogate models: median predictive divergence of the fitted model (nearly identical in the two families) and probability that the plug-in value errs by more than

20\%

(100 random networks per family, 64 fits per network and sample size).

Units	Family	Networks	Scans	Edges	Edge within distance 1	Median jump	Jumps $\geq 25\%$
4	excitatory	200/200	1,262	424	26%	25%	100%
4	mixed-sign	117/200	714	425	44%	50%	99%
5	excitatory	60/60	143	73	33%	33%	99%
5	mixed-sign	37/60	116	76	48%	45%	97%

Appendix

Table 11: Edges in random four- and five-unit logistic networks. Each scan follows one random direction through a state with

\varphi>0

over a parameter distance of one on either side (two directions per state for four units, one for five); every discontinuity is confirmed by bisection. Networks: those with at least one state with

\varphi>0

, of the number drawn. Jumps are relative to the larger side.

Causal Identifiability

Causal identifiability asks when data and stated assumptions uniquely determine causal structures, effects, or counterfactuals.

See all 133 papers

Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretability
Jul 9, 2026Amir AsiaeeMechanistic Interpretability Causal Interventions in Language Models
Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN 37232, USA
From Circuit Evidence to Mechanistic Theory: An Inductive Logic Approach
May 20, 2026Nura Aljaafari, Danilo S. Carvalho, Andre FreitasCircuit Discovery Mechanistic Interpretability
Department of Computer Science, University of Manchester, United Kingdom · CRUK National Biomarker Centre, University of Manchester, United Kingdom · Idiap Research Institute, Switzerland
Certifying Interventional Agreement Among Observationally Equivalent Causal Models
Sep 26, 2026Sourena Khanzadeh, Daniel Platnick, Marjan Alirezaie +1Causal Inference Causal Intervention
Toronto Metropolitan University · Flybits Labs, Creative AI Hub · MIT Media Lab, Massachusetts Institute of Technology

Jul 9, 2026J/K move · Enter open · S save

Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretability

Amir Asiaee

Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN 37232, USA

Mechanistic Interpretability Causal Interventions in Language Models Explainability Evaluation Causal Intervention

Open paper AskPDF arXiv

Mechanistic interpretability often evaluates explanations by intervening on a model: swapping hidden states, patching activations, ablating components, or comparing a compressed model to the original one. These experiments are usually summarized by a point estimate, even though the evaluation may be monitored while it runs or adapted toward suspected failures. This makes it hard to tell whether a reported fidelity or patching effect is a stable causal claim or a consequence of finite sampling and evaluation choices. We introduce Certified Interventional Fidelity (CIF), a statistical layer for interventional interpretability evaluations. CIF first writes the quantity being reported as a causal estimand: an expectation of a bounded score over a stated input distribution and a stated intervention distribution. It then provides confidence intervals and anytime-valid confidence sequences for this estimand, including under adaptive intervention sampling via bounded mixture importance weighting. We instantiate CIF with Hoeffding-style sequences and variance-adaptive betting sequences, the latter reducing certification cost by 10-30x in our experiments. On MNIST abstractions and GPT-2 Small IOI circuits, CIF certifies high-fidelity claims, shows when apparent method differences are not statistically supported, and makes sensitivity to the intervention distribution explicit.

Abstract

Causal Identifiability

Explore similar work

Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretability

From Circuit Evidence to Mechanistic Theory: An Inductive Logic Approach

Certifying Interventional Agreement Among Observationally Equivalent Causal Models