cs.LGNov 5, 2026

Same Behavior, Different Mechanism: What Finite Experiments Can Certify About Integrated Information

Authors: Linqian Wu, Diaoxuan He

Organizations: University of Waterloo, Waterloo, Ontario, Canada · Perimeter Institute for Theoretical Physics, Waterloo, Ontario, Canada · University of Debrecen, Debrecen, Hungary

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Interpretability, AI auditing and assessments of machine consciousness increasingly rest on mechanism-level quantities: functionals of a system's causal transition law that describe how it computes, not what it outputs. Experiments see that law only through finite, partial and noisy interventions. We ask when such a quantity is identifiable from an experiment's access, continuous in the mechanism and certifiable from finite data, and answer exactly for a deliberately demanding case - the integrated information φ\varphi of integrated information theory - in stochastic networks of four and five units with known ground truth. Behavior can be silent: a recurrent and a feed-forward network with identical input-output laws at their response delay carry 0.28 and 0 bits, yet internal recordings separate them within 4-53 trials. Partial access can be just as blind: in our examples, units left out of a one-step recording impose precision floors of 0.30-1.02 bits that no sample size lowers, and in all-but-self parity networks, recordings of any length see several hidden units only through the product of their fidelities - a symmetry that one intervention broke in exact checks. The measure has sharp edges: because φ\varphi selects a partition, and the cause and effect it specifies, by one criterion and reports another, infinitesimal changes of mechanism that cross an edge change it by 25-100% at 99% of the edges we found in random networks, and about 1% of surrogate models that fit interventional data almost perfectly misreport it by more than 20%. One-sided claims survive: the same data certify a lower bound on both sides of an edge, and we prove φ>1.004\varphi>1.004 bits throughout a 64-dimensional region of mechanisms and certify it from about 10710^7 controlled transitions with false-certificate probability at most αα. Measurability is thus a design question with computable answers.

Figures & tables

Figure 1: Three barriers and one way through. (A) A recurrent network and a feed-forward chain with identical input–output laws at their response delay differ in integrated information φ\varphi . (B) An experiment prepares known states, records some units after some delay, infers the mechanisms compatible with the data and reports a claim about φ\varphi . Three barriers limit what any such experiment can establish; robust claims can still be certified. (C) Input–output data carry no evidence about the difference in A at any number of trials, whereas recording internal units yields evidence at every trial (expected evidence between the two fully specified systems; Section 3 ). (D) When one unit is hidden from a one-step recording, every point on the curve and the red dot (the exact four-way tie; φ\varphi from 00 to 1.131.13 bits) is compatible with the same recordings, so a valid 95%95\% interval must contain each of these values with probability at least 0.950.95 and has expected width of at least 0.9×1.129=1.0160.9\times 1.129=1.016 bits, whatever the sample size ( Section 4 ). (E) Over a two-dimensional slice through one weight and one bias of a random mixed-sign network (darker: larger φ\varphi ; same slice as Figure 4 B), φ\varphi forms a mosaic of tiles separated by cliffs (white: partition switches; dashed: a switch of the specified states). An arbitrarily small change of mechanism that crosses a cliff changes φ\varphi by 2525 – 100%100\% at 99%99\% of the edges we found, and a surrogate trained near an edge misreported φ\varphi by a factor of 2.52.5 – 2.72.7 for about a thousand gradient steps ( Section 5 ). (F) Exact probability of certifying φ>1.004\varphi>1.004 bits with simultaneous Clopper–Pearson intervals, against the number of controlled transitions, for sources at the center (solid) and halfway to the boundary (dashed) of the proved region (green), and for a source one cell outside it (grey; boundary sources are likewise never certified). False certificates have probability at most α\alpha ( Section 7 ).
 CauseWhat does not helpWhat helps
Black boxesbehaviorally equivalent mechanisms differ in φ\varphimore input–output trials (zero evidence)recording internal units: expected log-likelihood ratio ln⁡20\ln 20 (20:1) in 4–53 trials (two specified candidates)
Hidden unitsmarginalization; in all-but-self parity networks, only the product of hidden fidelities reaches the recordingsmore trials; with several hidden units, longer recordings of the same unitsrecording more units; later readouts; interventions on hidden units
Sharp edgesthe measure selects by one criterion and reports anotherpoint estimates; plug-in values from well-fitting modelswarranted structural knowledge; reporting the lower envelope
Certificatesa theorem over a region of mechanisms, or a lower envelope—one-sided claims: confidence set inside the region, false certificates ≤α\leq\alpha
Table 1: Three barriers and one way through. What fails, why, and what works, for inferring a mechanism-level quantity from finite interventions.
Figure 2: Same function, different mechanism. (A) The recurrent parity network ( φ=0.28\varphi=0.28 bits) and the feed-forward chain ( φ=0\varphi=0 ) unrolled over two updates. Highlighted arrows (red in the recurrent network, dark in the chain) trace how the input (unit 1 at tt ) reaches the output: through every other unit in the recurrent network, along a single path in the chain. (B) Exact probability that a unit is on, from each of the 16 initial states, grouped by the input bit. Top: the output (unit 1 of the recurrent network, unit 3 of the chain) at t+2t{+}2 coincides in the two systems. Bottom: interior unit 2 at t+1t{+}1 differs. (C) Expected log-likelihood ratio (nats) in favor of the recurrent network, when it is the true system, as trials accumulate, for four internal recordings (sensor error 0.100.10 ; both systems known; balanced interventions). Markers show where the expected log-likelihood ratio reaches ln⁡20\ln 20 (odds of 20:1), labeled with the number of trials. Input–output data give zero evidence at any number of trials.
Figure 3: Hidden units: floors and symmetries. (A) Three designs record the four-unit parity network after the same intervention; the dashed unit is never recorded. (B) Mean width of 95%95\% intervals for Δφ\Delta\varphi between two conditions that differ only in the hidden unit (256 paired datasets per design and budget), and how often the sign of Δφ\Delta\varphi was resolved. Open and filled markers: budgets of 32,768 and 131,072 per condition; dashed line: the true Δφ=0.24\Delta\varphi=0.24 bits, for scale. (C) With recorded reliabilities fixed at 421/512421/512 , every hidden reliability on the horizontal axis gives the same one-step recording of units 1–3, so every value of φ\varphi on the curve and at the red dot (the exact four-way tie), from 00 to 1.131.13 bits, is compatible with the data. A valid 95%95\% interval must contain each of these values with probability at least 0.950.95 , which gives the floor in D. (D) Lower bounds on the expected width of any 95%95\% interval ( Proposition 2 ) for recorded reliabilities 421/512421/512 , with and without the exact four-way tie, and for two sets of distinct recorded reliabilities; no number of trials lowers them. Dashed line: B4=1.21B_{4}=1.21 bits, the largest possible φ\varphi for four units. (E) φ\varphi (all-zero state) of a five-unit network with recorded reliabilities (0.78,0.80,0.83)(0.78,0.80,0.83) over the fidelities a4,a5a_{4},a_{5} of its two hidden units. Observation reveals only the product a4a5a_{4}a_{5} (white curves: a4a5=0.2a_{4}a_{5}=0.2 , 0.320.32 (thick; through configurations a and b), 0.50.5 and 0.70.7 ; Theorem 3 ). (F) Total-variation distance between the recorded trajectories of a and b (equal product) and of a pair with a different product, as the recording is extended. Equal-product trajectories are identical by Theorem 3 ; the residual 10−1610^{-16} is floating-point round-off. Setting one hidden unit once (diamonds: unit 4 filled, unit 5 open) separates a from b. (G) φ\varphi along curves of constant product. The shaded band is the tie-free spread of φ\varphi along a4a5=0.32a_{4}a_{5}=0.32 ; at the two exact ties on that curve, where a hidden fidelity equals that of recorded unit 1, φ=0.380\varphi=0.380 bits (red dots).
Figure 4: Sharp edges are common. (A) φ\varphi (solid) and φraw\varphi_{\mathrm{raw}} (dashed) of the parity network as one unit’s reliability varies with the other three at 421/512421/512 ; left of the tie the two coincide. At the four-way tie φ=1.13\varphi=1.13 bits (filled dot); an arbitrarily small decrease gives 0.330.33 bits (open dot, the one-sided limit), and larger decreases give less. Right: over the same 10−410^{-4} perturbation the full cause–effect structure ( Φ\Phi , summed over all its distinctions and relations) changes by 0.02%0.02\% and the selected system not at all. (B) φ\varphi and φraw\varphi_{\mathrm{raw}} over a two-dimensional slice through one weight and one bias of a random mixed-sign network (dot: the network). White lines mark discontinuities from partition selection, where the selected cut changes its number of severed connections; the dashed line marks a switch of the specified states, across which φ\varphi and φraw\varphi_{\mathrm{raw}} both jump to zero. (C) Sizes of 849 discontinuities found on 1,976 random parameter paths through the 988 states with φ>0\varphi>0 of 317 random networks, relative to the larger side, for excitatory (light) and mixed-sign (dark) networks side by side. Partition switches sit exactly at 1−ea/eb1-e_{a}/e_{b} for the edge counts of the competing cuts ( Proposition 4 ); jumps to or from zero come from switches of the specified states. Two edges lie off this lattice: a 4:94{:}9 switch at which the cap binds ( 32.6%32.6\% ) and a simultaneous state and 3:33{:}3 partition switch ( 31.5%31.5\% ). (D) Fraction of random paths (two per state with φ>0\varphi>0 ) that cross an edge within a given parameter distance of the state.
Figure 5: A good fit is not a good measurement. (A) Mean absolute error of φ\varphi estimated from 65,536 complete transitions of the parity network, assuming four free reliabilities or one shared reliability, when all four are equal (left) and when one is lowered by 10−410^{-4} (right); 256 datasets each. (B) Full-batch maximum-likelihood training of a surrogate network on 2162^{16} interventional transitions from a mixed-sign target chosen to lie within 0.030.03 of an edge. Top: predictive divergence from the target. Bottom: φ\varphi (solid) and φraw\varphi_{\mathrm{raw}} (dashed) of the surrogate and the true φ\varphi (dotted); the shaded window marks where the surrogate’s φ\varphi is 2.52.5 – 2.72.7 times the true value. (C) Relative error of the plug-in values against the fit’s predictive divergence, for 100 random excitatory networks, five sample sizes and 64 fits each. Dots: plug-in φ\varphi of each fit; lines: median and 99th percentile of the relative error of the plug-in φraw\varphi_{\mathrm{raw}} . (D) Probability of a relative error above 20%20\% against the number of transitions, for φ\varphi (dark) and φraw\varphi_{\mathrm{raw}} (blue) in excitatory (solid, circles) and mixed-sign (dashed, squares) networks; zero failures among 6,400 fits are not drawn on the log scale.
Figure 6: From attainable to delivered evidence. (A) Coverage of the true φ\varphi (heterogeneous source, sensor error 0.100.10 , 32,768 recording and 128 calibration trials) when sensor error is ignored, plugged in as an estimate, or propagated jointly with recording uncertainty. Bars show coverage (31, 89 and 256 of 256 datasets), whiskers are exact 95%95\% binomial intervals, and the dashed line is the nominal 95%95\% . (B) Mean interval width on identical two-step data with the Box and Joint procedures for four sources (216 cells, 55,296 datasets); blue line: four recorded units on their own data, averaged over all 216 cells (all sources pooled). (C) Mean interval width under a correct model (finest grid) against the rate of rejecting a wrong model (coarse grid; a recorded unit’s bias shifted by 0.200.20 ), for eight random logistic networks (Bias family, hidden bias 0.250.25 , budget 131,072). (D) Rejection rate against the size of the unmodeled shift (colors as in C); the dotted line is the bound α=0.05\alpha=0.05 on rejecting a correct model.
Figure 7: One-sided claims can be certified. (A) A theorem guarantees φ>1.004\varphi>1.004 bits for every law in a 64-dimensional region. If a simultaneous confidence set lies inside the region, the claim is certified; if it crosses the boundary, no claim is made. (B) The region and the rule drawn to scale for one of the 64 probabilities: four test sources (top) and the ranges that every estimate must fall in for certification, with Hoeffding (grey) and exact Clopper–Pearson (green) intervals. (C) Exact probability of certification against the number of controlled transitions for the center (solid) and halfway (dashed) sources, with exact Clopper–Pearson (green) and Hoeffding (grey) intervals. Circles: Monte Carlo simulations with the Hoeffding rule (256 datasets each; filled, center; open, halfway); crosses: boundary and outside sources, never certified (0/256). (D) The four uniform lower bounds in the proof: selected raw loss, cause and effect specification, and differentiation ( −log⁡2-\log_{2} of the largest specified probability). The smallest sets the certified value (filled: the two bounds within 0.010.01 bits of it). No candidate of at most three units can exceed one bit (grey).
Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

QuantityPartition selectionRole
φ\varphi (target)minimum loss per severed connection; ties to the larger raw lossIIT 4.0 system integrated information
φraw\varphi_{\mathrm{raw}}minimum raw losscomparison, not part of the theory; continuous in the parity family, jumps only where the specified states switch
φτ\varphi_{\tau}as φ\varphi , with comparison tolerance 10−1310^{-13}values saved by software
Φ\Phi—sum over the full cause–effect structure (all distinctions and relations)
Appendix
Table 2: Selection and reporting rules. All rows share the transition law, the allowed cuts and the cap.
Caseφ\varphi (this work)φ\varphi (earlier implementation)φraw\varphi_{\mathrm{raw}}
δ=0\delta=01.1292943081.1292943080.328078083
δ=2−24\delta=2^{-24}0.3280780110.3280780110.328078011
δ=2−16\delta=2^{-16}0.3280597560.3280597560.328059756
δ=15/4096\delta=15/40960.3236862190.3236862190.323686219
δ=1/32\delta=1/320.2910273850.2910273850.291027385
(0.78,0.80,0.83,0.85)(0.78,0.80,0.83,0.85)0.2824290910.2824290910.282429091
Appendix
Table 3: Reference values of φ\varphi in the first parity network (all states). δ\delta is the decrease of one reliability from 421/512421/512 .
DesignRecordedCost per trialInstallation
3 units, t+1t{+}1units 1–3, one update after the intervention2+3c2+3c0
4 units, t+1t{+}1all four units, one update after2+4c2+4c512
3 units, t+2t{+}2units 1–3, two updates after3+3c3+3c0
Appendix
Table 4: Recording designs in the first parity network.
BudgetDesignSign resolvedUnresolvedMean width of Δφ\Delta\varphi interval (bits)
32,7683 units, t+1t{+}102562.41
32,7684 units, t+1t{+}125600.98
32,7683 units, t+2t{+}202561.62
131,0723 units, t+1t{+}102562.21
131,0724 units, t+1t{+}125600.43
131,0723 units, t+2t{+}225600.97
Appendix
Table 5: Directional contrasts between two conditions that differ only in the hidden unit (recorded reliabilities (0.78,0.80,0.83)(0.78,0.80,0.83) ; Δφ=0.2433\Delta\varphi=0.2433 bits; 256 paired datasets per row; budgets are per condition). Every interval covered the true difference and none supported the wrong sign.
ProcedureMean width (bits)Min. cell coverage
Moment1.112248/256
Box1.138256/256
Joint0.914247/256
4 units, t+1t{+}1 (own data)0.693255/256
Appendix
Table 6: Two-step inference on identical data (55,296 datasets, 216 equally weighted cells). Left: overall. Right: Joint by source, and its change relative to Box.
DesignMean width (bits)CoverageReject, shift 0.05Reject, 0.10Reject, 0.20
3 units, t+1t{+}10.2140.9760.0670.2280.952
4 units, t+1t{+}10.0340.9860.0330.1530.851
3 units, t+2t{+}20.0350.9930.0010.0110.120
Appendix
Table 7: Eight random logistic networks, hidden bias 0.250.25 , budget 131,072 (Bias family). Widths and coverage at the finest grid; rejection rates for unmodeled shifts of a recorded unit’s bias at the coarse grid (equal-cost designs).
SourceN=223N=2^{23}2252^{25}2272^{27}
center2/256256/256256/256
halfway0/25630/256256/256
boundary0/2560/2560/256
one cell outside0/2560/2560/256
Appendix
Table 8: Certification of φ>1.004\varphi>1.004 bits. Left: Monte Carlo results with the Hoeffding rule (256 datasets per cell); every one of the 3,072 confidence sets contained its true law. All four test sources have φ>1.004\varphi>1.004 at every state, so these runs measure power and conservativeness; false certificates are bounded by α\alpha ( Lemma B.8 ). Right: transitions needed for 95%95\% certification probability under three exact confidence constructions.
Sourcetrue φ\varphiNNmean boundsmallest boundbound validplug-in MAE
four equal reliabilities1.1292122^{12}0.2880.265256/2560.798
  2162^{16}0.3180.312256/2560.801
  2202^{20}0.3260.324256/2560.802
one lowered by 10−410^{-4}0.3282122^{12}0.2870.266256/2560.015
  2162^{16}0.3180.313256/2560.004
  2202^{20}0.3250.324256/2560.002
Appendix
Table 9: One-sided certificates at the edge of the parity network (256 datasets per row). The certified lower bound is valid in every dataset on both sides of the edge, while the plug-in estimate with four free reliabilities fails at the tie.
  ExcitatoryMixed-sign
NNmedian KL (nats)φ\varphiφraw\varphi_{\mathrm{raw}}φ\varphiφraw\varphi_{\mathrm{raw}}
2102^{10}9.5×10−39.5\times 10^{-3}0.1640.0980.5410.489
2122^{12}2.4×10−32.4\times 10^{-3}0.0450.0030.3030.257
2142^{14}5.9×10−45.9\times 10^{-4}0.02000.1140.088
2162^{16}1.5×10−41.5\times 10^{-4}0.01300.0320.019
2182^{18}3.7×10−53.7\times 10^{-5}0.00800.0100.002
Appendix
Table 10: Learned surrogate models: median predictive divergence of the fitted model (nearly identical in the two families) and probability that the plug-in value errs by more than 20%20\% (100 random networks per family, 64 fits per network and sample size).
UnitsFamilyNetworksScansEdgesEdge within distance 1Median jumpJumps ≥25%\geq 25\%
4excitatory200/2001,26242426%25%100%
4mixed-sign117/20071442544%50%99%
5excitatory60/601437333%33%99%
5mixed-sign37/601167648%45%97%
Appendix
Table 11: Edges in random four- and five-unit logistic networks. Each scan follows one random direction through a state with φ>0\varphi>0 over a parameter distance of one on either side (two directions per state for four units, one for five); every discontinuity is confirmed by bisection. Networks: those with at least one state with φ>0\varphi>0 , of the number drawn. Jumps are relative to the larger side.
More in this topic

Causal Identifiability

Causal identifiability asks when data and stated assumptions uniquely determine causal structures, effects, or counterfactuals.

See all 133 papers

Explore similar work

CardsList
Jul 9, 2026

Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretability

Mechanistic interpretability often evaluates explanations by intervening on a model: swapping hidden states, patching activations, ablating components, or comparing a compressed model to the original one. These experiments are usually summarized by a point estimate, even though the evaluation may be monitored while it runs or adapted toward suspected failures. This makes it hard to tell whether a reported fidelity or patching effect is a stable causal claim or a consequence of finite sampling and evaluation choices. We introduce Certified Interventional Fidelity (CIF), a statistical layer for interventional interpretability evaluations. CIF first writes the quantity being reported as a causal estimand: an expectation of a bounded score over a stated input distribution and a stated intervention distribution. It then provides confidence intervals and anytime-valid confidence sequences for this estimand, including under adaptive intervention sampling via bounded mixture importance weighting. We instantiate CIF with Hoeffding-style sequences and variance-adaptive betting sequences, the latter reducing certification cost by 10-30x in our experiments. On MNIST abstractions and GPT-2 Small IOI circuits, CIF certifies high-fidelity claims, shows when apparent method differences are not statistically supported, and makes sensitivity to the intervention distribution explicit.

Amir Asiaee

Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN 37232, USA

Mechanistic InterpretabilityCausal Interventions in Language ModelsExplainability EvaluationCausal Intervention
AskPDF
May 20, 2026

From Circuit Evidence to Mechanistic Theory: An Inductive Logic Approach

Mechanistic interpretability produces circuit-level causal analyses of neural network behaviour, but discovered circuits often remain isolated experimental artefacts: there is no shared formal representation for what circuits compute, how they relate, or when two findings provide evidence for the same mechanism. This work provides a formal infrastructure for cumulative mechanistic science by treating circuit interpretation as inductive theory construction. Each circuit is characterised at two levels: a Causal Functional Signature (CFS), which grounds component behaviour in causal attribution evidence and token role profiles, and an architectural signature τarchτ_{\mathrm{arch}}, learned by inductive logic programming (ILP) from scale-invariant structural predicates. Together, these constitute a formal coherence layer that makes mechanistic claims explicit, comparable via θθ-subsumption, and portable across model scales. CFS reveals qualitatively distinct computational strategies across task types, including attention-mediated copying versus MLP-mediated binding. ILP signatures achieve substantially better structural separation than graph kernel and feature-vector baselines, and support principled transfer across model scales and architecture families.

Nura Aljaafari, Danilo S. Carvalho, Andre Freitas

Department of Computer Science, University of Manchester, United Kingdom · CRUK National Biomarker Centre, University of Manchester, United Kingdom · Idiap Research Institute, Switzerland

Circuit DiscoveryMechanistic InterpretabilityInductive Logic ProgrammingCausal Interpretability
AskPDF
Sep 26, 2026

Certifying Interventional Agreement Among Observationally Equivalent Causal Models

Observationally equivalent causal models can still disagree about what happens under intervention, because interventions create inputs that never occur in observational data. We introduce Interventional Separation Selection (ISS), which repeatedly queries the true system with an admissible intervention on which the surviving candidate models disagree, discards the candidates the outcome contradicts, and stops once no intervention within a cost bound separates the survivors. If the true system is among the candidates, this stopping condition certifies that every survivor agrees with it on every admissible intervention within the bound, a guarantee that no observational learner can give, however much data it sees. The stopping condition depends only on the survivors, so it can be checked without knowing the truth. For continuous variables the candidates form an infinite version space, and mixed-integer linear programs decide the stopping condition exactly over all of it, with agreement holding up to a tolerance. On a three-digit colored MNIST causal abstraction task in which ink hue tracks digit size, plain convolutional networks trained on examples reach zero held-out error, yet disagree with shape-based labels on 26% of single-digit edits, as often as hue-based labels do. Auditing the causal abstractions of networks observed only on such images, ISS certifies what each network perceives with 13.6 interventions per image on average, and each certificate, checked against every admissible intervention, holds whenever the network's true abstraction is among the candidates. When a network bypasses a unit that every candidate abstraction relies on, certificates covering interventions on that unit can be silently void, and twenty random validation interventions refute 69% of them.

Sourena Khanzadeh, Daniel Platnick, Marjan Alirezaie +1

Toronto Metropolitan University · Flybits Labs, Creative AI Hub · MIT Media Lab, Massachusetts Institute of Technology

Causal InferenceCausal InterventionCausal InterpretabilityCausal IdentifiabilityCausal Abstraction
AskPDF