Intervene, Calibrate, Discover: Learning Causal Structure from Pixels

Ziyi Ding; Xiaodi Huang; Xianxin Lai; Haijier Chen; Zhang Zhang; Xiao-Ping Zhang; Jiayu Chen

cs.AIOct 26, 2026

Intervene, Calibrate, Discover: Learning Causal Structure from Pixels

Authors: Ziyi Ding, Xiaodi Huang, Xianxin Lai, Haijier Chen, Zhang Zhang, Xiao-Ping Zhang, Jiayu Chen

Organizations: Tsinghua University · INFIFORCE Intelligent Technology, China · Institute of Automation, Chinese Academy of Sciences · The University of Hong Kong ·

Abstract

Learning high-level causal variables from low-level observations is a central problem for artificial intelligence. With known node semantics but an unknown graph, visual readout errors can misalign model settings with environment intervention values. We address this mismatch by pairing image readouts with verified values attained through fixed round-robin interventions. With encoder and decoder frozen, a dimension-wise invertible calibration map puts visual responses and target settings in shared numerical units. After freezing this map, the agent selects interventions to resolve graph directions and fits local mechanisms to predict consequences; inverse calibration returns counterfactual states to the decoder's original coordinates. Across Domino, Pick-and-Place, Balance, Gears, CAUSAL3D, and CausalWorld, we evaluate graph discovery, intervention consequences, and counterfactual images. Controlled domino experiments show that interventions resolve orientation ambiguity left by equal additional observations. In a matched-data domino control, calibration reduces the mean error of predicted changes in other variables within the same factual context, relative to an uncalibrated model.

Figures & tables

Panel of Figure 1 — Figure 1: Left: simulator scenes before/after releasing domino 2 from a $20^{\circ}$ tilt; the robot arm is fixed. Right: a separate seven-node structural causal model (SCM): pusher, four dominoes, bell, and light. $\doop(s_{3}=u)$ replaces the target mechanism and removes incoming edges. The physical preview is neither a prediction nor validation of this graph.

Panel of Figure 2 — Figure 2: Positioning intervention-value queries. Sets summarize inputs, acquisition, and structure learning; their intersection denotes a joint setting, not a ranking. We align visual estimates with verified values. Numbered work and classification conditions: Appendix H.1 .

Figure 3: Learning and evaluation. The visual model and calibration freeze before active targeting, orientation, and mechanism fitting. $E,T,G,f$ : encoder, calibration, graph, mechanisms. Gray/blue/green: inputs/outputs, computation, calibration. OOD excludes A/B-only tests lacking defined shift axes. Curves denote continuous commands (query coordinates: Appendix Figure A2 ).

Figure 4: Example environments. Top: controlled MEC scenes and generating graphs for PnP, Balance, and Gears. Bottom: original CAUSAL3D Spring and Seesaw images and a native CausalWorld Pushing illustration. Generating graphs are not estimates; scene illustrations do not represent task-table configurations or model predictions. Sources appear in Appendix A.7 .

	Domino		Other MEC tasks			CAUSAL3D		Causal World
Method	SCM	Physical	PnP	Balance	Gears	Spring	Seesaw
Ours	4 .846	0 .982	0 .862	6 .413	7 .793	9 .242	5 .587	4 .276
CausalVAE	6 .313	3 .284	3 .516	8 .475	8 .522	11 .213	7 .342	11 .013
ILCM	6 .194	2 .294	5 .232	4 .178	5 .177	17 .084	8 .231	16 .007
Completed/planned runs $\uparrow$ (not graph-recovery rates)
Ours	8/8	7/8	8/8	7/8	8/8	8/8	6/8	7/8

Table 1: Graph discovery: each cell reports SHD

\downarrow

/AUPRC

\uparrow

on its upper/lower line. Bold marks the best point estimate; light blue marks our method. Bottom: completed/planned runs, not graph-recovery rates. See Appendix G.1 for comparison scope.

	ID readout	Non-target CF effect NMSE $\downarrow$		RGB MSE $\downarrow$
Configuration	MSE $\downarrow$	ID	Amp. shift	ID	Amp. shift
Uncalibrated	$22.367\pm 3.941$	$5.463\pm 0.946$	$6.858\pm 1.290$	$1.085\pm 0.066$	$0.874\pm 0.046$
Fully fitted affine	$14.721\pm 1.177$	$\mathbf{4.763}\pm 0.142$	$6.656\pm 0.531$	$\mathbf{1.043}\pm 0.065$	$\mathbf{0.856}\pm 0.034$
Nonlinear (ours)	$\mathbf{12.864}\pm 0.830$	$4.968\pm 0.123$	$\mathbf{6.515}\pm 0.418$	$1.065\pm 0.063$	$0.878\pm 0.036$
Ordinary joint updating (ref.)	$11.361\pm 5.044$	$9.985\pm 1.611$	$20.449\pm 3.725$	$1.058\pm 0.080$	$0.851\pm 0.053$

Table 2: Readouts, intervention effects, and images in the same paired three-seed cohort. Errors are multiplied by

10^{3}

; entries are means

\pm

sample SD across seeds. The first three rows share each seed’s frozen encoder, data, learned graph, and ID-refitted decoder

D^{\prime}

, fitting mechanisms in their respective coordinates. Ordinary joint updating additionally trains a readout head and decoder and is reference-only. All seeds 5/6/7 are scored; bold marks the lowest mean in the first three rows. Paired differences appear in Appendix D.4 .

	Domino		Other MEC tasks			CAUSAL3D		Causal World
Method	SCM	Physical †	PnP	Balance	Gears	Spring †	Seesaw †	$\dagger$
Ours	8.573 10.341 0.829	6.234 8.291 0.752	1.771 3.461 0.512	2.085 4.101 0.508	.729 1.091 0.668	5.323 8.064 0.660	4.276 6.786 0.630	4.817 5.378 0.896
CausalVAE	123.799 807.120 0.153	228.713 791.706 0.289	51.105 318.545 0.160	130.996 852.155 0.154	75.124 503.839 0.149	186.127 609.486 0.305	156.435 987.345 0.158	293.854 974.245 0.302
ILCM	123.713 874.392 0.141	162.410 515.288 0.315	58.106 394.825 0.147	104.107 713.652 0.146	101.258 699.789 0.145	234.144 772.011 0.303	95.245 312.261 0.305	124.215 332.211 0.374
No graph	34.144 245.391 0.139	51.205 421.363 0.122	57.011 393.035 0.145	102.146 700.012 0.146	93.292 663.595 0.141	204.114 402.341 0.507	98.701 244.512 0.404	41.124 408.946 0.101

Table 3: Conditional consequence-prediction errors, excluding blocked queries. Unmarked columns retain the reported ID/amplitude-shift labels; entries are NMSEs (

\times 10^{3}

) and ratio

r

. Columns marked

\dagger

report test sets A, B, and

E_{A}/E_{B}

, without inferring an OOD type. Bold marks the smallest listed error summary; light blue marks our method. Lower errors are better;

r

alone does not define a ranking. Scope and distribution qualifications: Appendices A.2 and A.8 .

	Domino		Other MEC tasks			CAUSAL3D		Causal World
Method	SCM	Physical	PnP	Balance	Gears	Spring	Seesaw
Ours	1.473	.880	1.118	2.180	.938	2.384	8.264	5.261
CausalVAE	2.525	1.287	3.563	4.603	1.859	6.345	11.745	7.761
ILCM	9.009	5.707	7.025	15.477	8.187	3.256	5.234	9.664
ICM-VAE (given graph)	–	–	–	–	–	7.342	9.753	–
No graph	4.215	2.314	2.161	5.014	1.859	15.234	32.219	19.281

Table 4: Counterfactual RGB generation: pixel MSE

\downarrow

(

\times 10^{-3}

) for executed queries only, excluding blocked queries. Bold marks the smallest listed summary; light blue marks our method; “–” means not applicable. Physical ILCM is a single-seed reference with different permissions and budget, excluded from unified ranking. Appendix A.2 specifies the comparison scope.

Figure 5: (A) Schematic of learned calibration, factual-residual propagation, and inverse-calibrated decoding. (B) Physical comparison for making the first domino fall ( $F_{1}=1$ ). The blue Ours column denotes our conditional-mean comparison branch, not the full calibration-and-residual path in A. Factual images, paired GT, and model outputs are above RGB mean absolute errors with a shared scale. Images are clipped to $[0,1]$ for display only. ILCM is a single-seed reference with different training permissions (Appendix A.9 ).

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Result location	Configuration and record	Queries, permissions, and scope
Physical ILCM in Table 5.5	run_s42_v1 , seed 42; native implicit mechanisms and its own decoder; epoch 120 selected by ID validation.	1280 training pairs, 256 validation pairs; 32 contexts $\times$ 4 targets $\times$ 2 commands, 256/256 queries. Paired GT matches archived seed-42 queries, not necessarily the retained sets of other task-table summaries. Permissions and budget differ (Appendix A.9 ).
MEC mechanism controls in Appendix C.4	Seeds 300/301, learned $T$ ; frozen representation, calibration, and graph mechanisms; a shared decoder trained on 2400 independent ID images.	PnP/Gears retain 2/2 seeds and Balance 1/2; respectively 384/384/320 CF queries per valid seed. Command ranges and 2048-sample outcome evaluation apply only to this cohort.
Uncalibrated physical control (aggregate)	No numerical calibration; aggregate full-image pixel MSE is 0.930 ( $\times 10^{-3}$ ).	Reported separately from the paired control below; its seed count, SD, query count, and training budget are not transferred. This aggregate is not used with the main-table learned- $T$ result to estimate paired calibration gains.
Paired conditional-mean control in Appendix A.9	No numerical calibration or addition of factual residuals to descendants; common completed seeds 42 and 314.	256 common queries per seed; our method completes 2/3 seeds and CausalVAE 3/3, with more concept labels for the latter. Disaggregated errors are reported in Appendix A.9 .

Appendix

Table A1: Physical and MEC cohort configurations. System-table comparison rules are in Appendix A.2 ; denominators below belong only to their named cohorts.

Benchmark	Task variables	Operation and test grouping
Domino SCM	7 continuous states: pusher, four dominoes, bell, lamp	Single-state setting; reported ID/amplitude groups
Physical Domino	4 binary final toppling events	Fix upright / tilt and release; test sets A/B *
PnP	6 SCM variables	Single-state setting; reported ID/amplitude groups
Balance	5 SCM variables	Single-state setting; reported ID/amplitude groups
Gears	6 SCM variables	Single-state setting; reported ID/amplitude groups
CAUSAL3D Spring	Scored variables not reported	Test sets A/B; targets and ranges not reported

Appendix

Table A2: Protocol coverage of the system-table summaries. Completed/planned run counts in Table 5.2 are not prediction seed counts

n

, executed queries

Q

, or calibration-inclusive totals

B

. Those fields are not reported for the aggregate cells and are not inferred from the controls below.

Configuration	Full-image MSE	Keep upright	Make fall
Our method (conditional-mean propagation)	$0.880\pm 0.016$	1.282	0.478
Factual-residual propagation control	$0.924\pm 0.026$	0.997	0.850
Target-only latent editing	$1.373\pm 0.001$	1.064	1.682
CausalVAE, 512 images, original decoder	$1.617\pm 0.153$	1.305	1.929
CausalVAE, 1280 images, original decoder	$1.287\pm 0.039$	1.051	1.523
CausalVAE, 512 images, ID refit	$1.946\pm 0.077$	1.640	2.252

Appendix

Table A3: Disaggregated errors for the paired rigid-body dominoes control, in units of

10^{-3}

. Full-image results are means

\pm

sample standard deviations over common completed seeds 42 and 314; they do not describe the uncalibrated physical aggregate in Table A.3 or the main-table learned-calibration summary.

Panel of Figure 13 — Figure A1: The same seed 42 and factual initial states as in the main text, with the command changed to $F_{1}=0$ . The second row shows conditional-mean propagation changing a descendant that fell independently in the factual scene. Predicted pixels retain the original output.

Figure A2: Coordinate transformations for a factual-conditioned counterfactual query (schematic). An image passes through

E,T

to produce calibrated readouts, while the verified value passes through

\alpha_{j}

. Both paths use the same observational normalization to yield

h(x)

and

q_{j}(u)

, respectively; the latter does not pass through

T

again. After setting the target, retaining factual residuals, and updating descendants, normalization and

T

are inverted before decoding with

D

E,D,G,f

abbreviate the encoder, decoder, estimated graph, and local mechanisms, respectively. All mappings and statistics are frozen. Population prediction without a factual input is described in Section 4.3 .

Seed	Original-rule SHD	Bilateral SHD	Bilateral unoriented edges	Nonlinear increment SD
5	0	6	6	0.0343
6	0	0	0	0.0311
7	0	0	0	0.0306

Appendix

Table A4: Frozen diagnostics. Graph columns use the final acquisition prefix; all three coordinate systems give the same results, and each cell is unchanged across 32 traversal orders. SHD counts each unoriented true edge as one unresolved relation and therefore differs from the number of wrong arrows. The last column is the SD of transformed increments for fixed original-coordinate residuals under nonlinear calibration, averaged equally over non-root nodes and three training-residual quantiles, in training-label-standardized coordinates. It is a coordinate diagnostic, not prediction error.

Acquisition policy	Seed	New records	Added-observation SHD
Coverage targeting	5	1800	6
Fixed round-robin	5	1800	6
Coverage targeting	6	3600	6
Fixed round-robin	6	1800	6
Coverage targeting	7	3600	6
Fixed round-robin	7	1800	6

Appendix

Table A5: Graph-evidence sources under the same frozen visual and calibration interface. Each row compares equal numbers of new records; both sides include the same 896 calibration records and 6000 initial observations. Intervention and additional-observation graphs are scored at matching prefixes; records not acquired after early stopping are not treated as available evidence.

Full sampled range
Calibration	Direct intervention	Natural observation	Other-node intervention
No calibration	$23.636\pm 2.994$	$36.850\pm 7.400$	$39.063\pm 6.405$
Fully fitted affine	$15.224\pm 0.399$	$\mathbf{33.258\pm 3.868}$	$\mathbf{36.365\pm 2.890}$
Nonlinear calibration	$\mathbf{12.998\pm 0.233}$	$35.144\pm 3.376$	$38.296\pm 2.306$
Fixed common bins
No calibration	$22.953\pm 3.222$	$30.711\pm 7.441$	$29.878\pm 5.482$

Appendix

Table A6: Cross-source readout MSE, multiplied by

10^{3}

, reported as mean

\pm

sample SD over 3 seeds with equal weighting of 7 nodes per seed. Common bins are fixed in training-label units and require at least 8 records per source, node, and bin; common bins then receive equal weight. This controls coarse common support, not full distribution matching. Bold denotes the lowest point estimate in each panel, not statistical significance.

Calibration	Command	CF-state NMSE	Population-mean NMSE	Image MSE
No calibration	ID	$37.303\pm 8.593$	$3.366\pm 2.889$	$2.257\pm 0.025$
Fully fitted affine	ID	$\mathbf{32.810\pm 3.066}$	$\mathbf{1.758\pm 0.025}$	$\mathbf{2.232\pm 0.023}$
Nonlinear calibration	ID	$34.875\pm 1.268$	$2.045\pm 0.176$	$2.237\pm 0.019$
No calibration	Amplitude shift	$38.396\pm 8.949$	$3.983\pm 3.330$	$1.699\pm 0.039$
Fully fitted affine	Amplitude shift	$\mathbf{35.102\pm 2.667}$	$3.756\pm 0.321$	$\mathbf{1.688\pm 0.043}$
Nonlinear calibration	Amplitude shift	$36.703\pm 0.869$	$\mathbf{3.492\pm 0.510}$	$1.692\pm 0.048$

Appendix

Table A7: Query results with frozen graphs, mechanisms, and decoders; errors are multiplied by

10^{3}

. State columns use true simulator states as GT and exclude the direct target; the image column reports MSE of each model’s own decoder output. Values are means

\pm

sample SD over common valid seeds 5 and 7 out of 3 planned seeds; seed 6 is blocked because its frozen graph is not sufficiently oriented. Bold denotes the lowest point estimate within each command group; amplitude shift is not strict out-of-support extrapolation.

Method	Domino	PnP	Balance	Gears
Our method	$0.1745$ $[0.1650,0.1855]$	$0.1345$ $[0.1251,0.1452]$	$0.0886$ $[0.0771,0.1041]$	$0.1021$ $[0.0995,0.1045]$
GIES	$0.2716$ $[0.1750,0.3939]$	$0.1319$ $[0.1235,0.1413]$	$0.0894$ $[0.0781,0.1049]$	$0.1018$ $[0.0998,0.1037]$
Passive graph extension	$0.6975$ $[0.6841,0.7094]$	$0.1345$ $[0.1252,0.1448]$	$0.5966$ $[0.4478,0.6782]$	$0.7199$ $[0.7024,0.7374]$
Shuffled graph	$0.4824$ $[0.4183,0.5600]$	$0.8287$ $[0.7532,0.9065]$	$0.6069$ $[0.5283,0.6883]$	$0.6478$ $[0.5312,0.7558]$
No graph propagation	$0.6975$ $[0.6841,0.7091]$	$0.9662$ $[0.9486,0.9865]$	$0.9418$ $[0.9290,0.9552]$	$0.9081$ $[0.8879,0.9315]$
True graph (structural reference)	$0.1745$ $[0.1650,0.1858]$	$0.1345$ $[0.1253,0.1448]$	$0.0886$ $[0.0770,0.1043]$	$0.1021$ $[0.0995,0.1045]$

Appendix

Table A8: Static factor-level counterfactual NMAE, lower is better. Means and 95% bootstrap intervals come from the corresponding per-seed records. Metrics with different normalization definitions are not ranked together.

Panel of Figure 20 — Figure A3: Fixed queries in three controlled MEC scenes. Cases are prespecified by test index, not filtered by prediction quality. PnP uses $64\times 96$ images and Balance/Gears use $64\times 64$ ; this group is not pooled with rigid-body domino or RGB128 image protocols.

Configuration	Target MSE (3/3)	SHD	CF pixel MSE (2/3)
Uncalibrated	$23.090\pm 3.183$	0/1/0	$2.1894\pm 0.0183$
Restricted positive affine	$15.036\pm 0.773$	0/1/0	$\mathbf{2.1643}\pm 0.0192$
Nonlinear calibration	$\mathbf{12.849}\pm 0.798$	0/1/0	$2.1707\pm 0.0204$

Appendix

Table A9: Restricted affine and nonlinear calibration with a frozen decoder. ID readout and CF errors are multiplied by

10^{3}

, with mean

\pm

sample SD across seeds. The graph column lists SHD of each configuration’s frozen graphs for seeds 5/6/7. Seed 6 is blocked by the graph gate, so no CF error is entered.

Configuration	Uncalibrated	Fully fitted affine	Nonlinear calibration	Ordinary joint updating
Target-readout MSE	$22.683\pm 4.703$	$14.377\pm 1.253$	$12.190\pm 1.201$	$\mathbf{8.075}\pm 0.860$

Appendix

Table A10: Matched-record controls with sequential direction updates in the seven-node domino scene (

64\times 64

RGB images). Errors are multiplied by

10^{3}

and reported as mean

\pm

sample SD across three encoder seeds; all configurations have 3/3 valid seeds and 896 CF queries per seed and split. The top panel reports target-readout MSE in standardized label units. Updated modules are, respectively, none, affine

T

, nonlinear

T

, and readout heads plus

D

; joint updating shares supervision but differs in parameter count. The bottom panel excludes the direct target from state, effect, and population-mean NMSE. Bold marks the lowest comparable point estimate; blue marks nonlinear calibration.

Configuration	Command	Target	Desc.	Non-desc.	Desc. change	Non-desc. change
Uncalibrated	ID	$219.4\pm 4.7$	$156.6\pm 5.1$	$88.6\pm 2.4$	$123.9\pm 0.5$	$42.4\pm 3.1$
Fully fitted affine	ID	$209.8\pm 4.5$	$155.0\pm 6.0$	$92.8\pm 3.1$	$122.1\pm 1.4$	$47.0\pm 3.1$
Nonlinear calibration	ID	$212.0\pm 4.0$	$157.2\pm 6.2$	$92.5\pm 3.0$	$125.5\pm 1.6$	$46.5\pm 2.8$
Ordinary joint updating	ID	$\mathbf{28.7}\pm 2.5$	$\mathbf{61.8}\pm 4.7$	$\mathbf{17.9}\pm 0.4$	$\mathbf{61.3}\pm 2.4$	$\mathbf{6.1}\pm 0.4$
Uncalibrated	Amplitude	$77.1\pm 2.9$	$86.5\pm 2.1$	$174.2\pm 5.7$	$149.3\pm 1.9$	$144.0\pm 6.5$
Fully fitted affine	Amplitude	$70.2\pm 3.2$	$82.9\pm 3.9$	$181.2\pm 3.6$	$146.8\pm 3.3$	$151.7\pm 4.8$

Appendix

Table A11: Visible-state MAE measured by an independent image readout model, multiplied by

10^{3}

; mean

\pm

SD across three encoder seeds. Target, descendant, and non-descendant readouts are scored against true simulator activations; changes use each model’s factual reconstruction as the reference. GT-image readout errors are reported separately as a measurement-error reference, not as a substitute for ground truth. Bold marks the lowest point estimate within each split and column.

Seed	Configuration	Readout MSE	ID mean NMSE	Amplitude mean NMSE	ID image MSE
5	Uncalibrated	18.898	1.027	1.969	2.260
5	Fully fitted affine	13.975	1.596	3.826	2.239
5	Nonlinear calibration	11.982	1.972	3.803	2.247
5	Ordinary joint updating	7.996	3.446	15.700	1.031
6	Uncalibrated	21.204	2.496	4.065	2.287
6	Fully fitted affine	13.374	2.065	3.860	2.276

Appendix

Table A12: Per-seed results for matched-record controls with sequential direction updates; errors are multiplied by

10^{3}

. The four configurations within each seed share acquired pixels and test contexts. All graphs have SHD=0, and all three prespecified seeds are scored.

Comparison (first minus second)	Query	Non-target effect NMSE difference	RGB MSE difference
Affine $-$ uncalibrated	ID	$-0.700\pm 0.930$ (2/3)	$-0.042\pm 0.021$ (3/3)
Affine $-$ uncalibrated	Amplitude shift	$-0.202\pm 1.583$ (2/3)	$-0.018\pm 0.020$ (2/3)
Nonlinear $-$ uncalibrated	ID	$-0.495\pm 1.022$ (2/3)	$-0.020\pm 0.027$ (2/3)
Nonlinear $-$ uncalibrated	Amplitude shift	$-0.343\pm 1.643$ (2/3)	$0.004\pm 0.023$ (2/3)
Nonlinear $-$ affine	ID	$0.204\pm 0.259$ (1/3)	$0.022\pm 0.006$ (0/3)
Nonlinear $-$ affine	Amplitude shift	$-0.141\pm 0.261$ (2/3)	$0.022\pm 0.004$ (0/3)

Appendix

Table A13: Paired differences for the same three-seed cohort as main-text Table 5.3 . Errors and SDs are multiplied by

10^{3}

; parentheses give the number of improved seeds/3. SD is computed from per-seed differences, not by subtracting the two groups’ SDs.

Calibration	Query	ID readout MSE	Non-target effect NMSE	Population mean NMSE	RGB MSE
Uncalibrated	ID	$7.517\pm 1.327$	$1.239\pm 0.229$	$0.455\pm 0.152$	$0.923\pm 0.011$
Uncalibrated	Amplitude shift	$7.517\pm 1.327$	$1.556\pm 0.384$	$0.845\pm 0.632$	$1.141\pm 0.007$
Affine	ID	$7.295\pm 1.599$	$1.347\pm 0.287$	$0.615\pm 0.323$	$0.926\pm 0.017$
Affine	Amplitude shift	$7.295\pm 1.599$	$2.001\pm 0.751$	$1.363\pm 1.135$	$1.144\pm 0.016$
Nonlinear	ID	$7.298\pm 1.364$	$1.396\pm 0.369$	$0.742\pm 0.471$	$0.928\pm 0.019$
Nonlinear	Amplitude shift	$7.298\pm 1.364$	$2.324\pm 1.153$	$1.724\pm 1.581$	$1.159\pm 0.010$

Appendix

Table A14: Calibration transfer on Gears-MEC with fixed graphs, seeds 300/301. All errors are multiplied by

10^{3}

and reported as mean

\pm

sample SD across seeds; readouts use independent ID verification. All three configurations share

D

;

n=2

permits only descriptive comparison.

Configuration	ID image MSE $\downarrow$	Amplitude shift image MSE $\downarrow$	Amplitude shift target visible MAE $\downarrow$	Amplitude shift non-descendant effect MAE $\downarrow$
Uncalibrated, frozen $D$	$2.259\pm 0.038$	$1.698\pm 0.069$	$79.256\pm 3.823$	$143.379\pm 9.280$
Uncalibrated, ID-refitted $D$	$1.082\pm 0.007$	$0.854\pm 0.012$	$16.212\pm 1.226$	$15.145\pm 1.766$
Fully fitted affine, frozen $D$	$2.241\pm 0.044$	$1.690\pm 0.076$	$72.223\pm 4.315$	$150.861\pm 8.913$
Fully fitted affine, ID-refitted $D$	$1.037\pm 0.021$	$0.836\pm 0.029$	$15.810\pm 1.201$	$16.719\pm 2.287$
Nonlinear calibration, frozen $D$	$2.246\pm 0.044$	$1.695\pm 0.076$	$66.577\pm 3.543$	$155.635\pm 8.886$
Nonlinear calibration, ID-refitted $D$	$1.058\pm 0.019$	$0.859\pm 0.028$	$15.394\pm 1.384$	$18.058\pm 2.758$

Appendix

Table A15: Paired decoder diagnostics; errors are multiplied by

10^{3}

, with mean

\pm

sample SD across three seeds and 896 paired queries per seed/split. The upper panel compares frozen

D

and ID-refitted

D^{\prime}

on the same images with encoder, calibration, graph, mechanisms, and state predictions fixed. Joint updating changes heads and decoder and is only a reference. Visible-activation MAE uses an independent frozen readout against simulator activations; GT-image readout error is only a measurement reference. The lower panel reports

10^{3}(\mathrm{MSE}_{D^{\prime}}-\mathrm{MSE}_{D})

: negative is better, and parentheses count improved seeds for each pair. Its SD is computed from per-seed differences, not subtracted group SDs.

Figure A4: Counterfactual queries in the seven-node domino scene with $64\times 64$ RGB images, at fixed seed 5, factual index 0, target 1, and ID command $u=\mu_{1}^{s}-0.75\sigma_{1}^{s}$ . All three configurations share factual context/decoder and retain factual residuals in their own coordinates; Learned $T$ is nonlinear calibration. Bottom-row absolute RGB errors share a color scale. This index-prespecified case illustrates an independent frozen diagnostic cohort, not a single-image method ranking.

A. Target-calibration error under fixed readout conditions
Condition	Seeds	Uncalibrated	Restricted positive affine $T$	Nonlinear $T$
No added distortion	3	$23.337\pm 5.417$	$15.333\pm 2.244$	$\mathbf{12.852\pm 1.690}$
Affine distortion	3	$114.354\pm 20.126$	$15.421\pm 2.135$	$\mathbf{12.950\pm 1.909}$
Cubic distortion	3	$61.438\pm 15.002$	$22.149\pm 3.796$	$\mathbf{17.781\pm 2.835}$
B. Counterfactual image error within each condition Jointly executable seeds
Condition	Valid/planned	Uncalibrated	Restricted positive affine $T$	Nonlinear $T$

Appendix

Table A16: Domino component and budget diagnostics. Errors in A/B are in units of

10^{-3}

, reported as mean

\pm

sample standard deviation. A reports target-readout MSE and B reports ID counterfactual RGB MSE; values are not compared across panels. Light blue marks nonlinear calibration, and bold marks the lowest error in each row. No standard deviation is reported for a single seed. C gives records used per strategy, including 896 calibration records.

\dagger

denotes failure to recover after exhausting the budget, not a successful-recovery cost.

A. Calibration objective and replay 3/3 seeds, all values multiplied by $10^{3}$
Configuration	Target-readout MSE $\downarrow$	Paired difference (ablation $-$ full)
Full configuration	$13.327\pm 0.648$	$0.0000\pm 0.0000$
Without observation preservation	$\mathbf{13.320}\pm 0.647$	$-0.0071\pm 0.0016$
Without cumulative sample replay	$15.245\pm 1.377$	$1.9174\pm 1.3410$
B. Post-intervention orientation propagation 3/3 seeds, identical fixed evidence
Configuration	SHD $\downarrow$	Complete valid graphs

Appendix

Table A17: Module controls with a fixed visual backbone. Values are mean

\pm

sample SD across training seeds. Bold marks only the lowest point estimate in each error column; light blue marks the full configuration. A uses independent ID validation, while B/C reuse fixed evidence and queries.

Configuration	Target MSE	CF image MSE	SHD
Uncalibrated	$25.232\pm 3.391$	$2.1892\pm 0.1470$	0/1/0
Restricted positive affine	$16.278\pm 1.553$	$\mathbf{2.1651}\pm 0.1441$	0/1/0
Fully fitted positive affine	$16.278\pm 1.553$	$\mathbf{2.1651}\pm 0.1441$	0/1/0
Nonlinear calibration	$\mathbf{14.146}\pm 0.850$	$2.1726\pm 0.1428$	0/1/0

Appendix

Table A18: Calibration-mapping controls with a frozen decoder; errors are multiplied by

10^{3}

. The upper panel reports mean

\pm

sample SD and SHD in seed order 5/6/7; the lower panel gives every seed value. State estimation uses 3 seeds, CF the jointly valid 2/3. Seed 6 retains one unoriented edge in every configuration: CF is blocked (–), not zero.

Configuration	Complete true-graph recovery	Actual calls	SHD
Nonlinear calibration + coverage targeting	2/3	$2696/11696^{*}/4496$	0/1/0
Nonlinear calibration + random targeting	3/3	$6296/6296/2696$	0/0/0
Nonlinear calibration + round-robin targeting	3/3	$2696/2696/2696$	0/0/0
Uncalibrated + coverage targeting	2/3	$1800/11696^{*}/3600$	0/1/0

Appendix

Table A19: Equal-total-budget controls. Call counts include calibration. An asterisk denotes exhaustion of the budget without full orientation, not the cost of successful recovery. Seed order is 5/6/7 throughout.

Task	Method	PnP (2/2)	Balance (1/2)	Gears (2/2)
CF-image MSE	Our method	$1.118\pm.134$	2.180	$.938\pm.021$
	No-graph ablation	$2.161\pm.078$	5.014	$1.859\pm.093$
ID NMSE	Our method	$1.771\pm.433$	2.085	$.729\pm.228$
	No-graph ablation	$57.011\pm 2.039$	102.146	$93.292\pm.179$
OOD NMSE	Our method	$3.461\pm 1.367$	4.101	$1.091\pm.341$
	No-graph ablation	$393.035\pm 10.084$	700.012	$663.595\pm 5.185$

Appendix

Table A20: Disaggregated statistics for the MEC mechanism-comparison group, with errors multiplied by

10^{3}

. Two-seed results give mean

\pm

sample standard deviation; parentheses show executable/planned seed counts.

Metric	Dominoes (64, 8/10)	Dominoes (128, 9/10)
Counterfactual-image MSE	$2.082\pm.073$	$2.116\pm.068$
ID original-coordinate MSE	$19.382\pm 11.244$	$16.208\pm 2.870$
OOD original-coordinate MSE	$32.297\pm 40.164$	$18.480\pm 2.970$
ID mean-image MSE	$.370\pm.025$	$.399\pm.046$
OOD mean-image MSE	$.314\pm.065$	$.331\pm.050$

Appendix

Table A21: Supplementary results for dominoes (RGB images) with our method; errors are multiplied by

10^{3}

. Neither original-coordinate MSE nor mean-image MSE substitutes for physical-readout NMSE in the main text.

Scene	Method	ID node- $R^{2}$ range	Selected/trained epochs
Domino-MEC	CausalVAE	[0.849, 0.984]	86/100
Domino-MEC	ILCM	[0.220, 0.797]	120/120
PnP-MEC	CausalVAE	[0.919, 0.986]	86/100
PnP-MEC	ILCM	[-0.005, 0.044]	120/120
Balance-MEC	CausalVAE	[0.920, 0.995]	98/100
Balance-MEC	ILCM	[0.118, 0.804]	120/120

Appendix

Table A22: Available ID validation and model-selection records for the MEC representation-model comparison group; each row uses seed 300,

n=1

R^{2}

is the range across nodes, not a seed-level confidence interval. Selection uses the native ID validation objective over the full mechanism stage. Diagnostics apply only to these checkpoints; coinciding values do not justify extending them to other main-table runs.

Work	Input/samples	Node or variable supervision	Intervention information and key conditions
CausalVAE (2021)	Images	Concept labels	Causal layer and latent-space editing; no active environment acquisition
ICF (2018)	Pixel interaction trajectories	Independently controllable factors	Selectivity feedback jointly trains representations and policies
Causal Curiosity (2021)	Interaction trajectories	Learned environment factors	Executable experimental behaviors; not standard node-setting supervision
CITRIS (2022)	Temporal images	Learned latent variables	Known target labels; actual values not required
iCITRIS (2023)	Temporal images	Learned latent variables	Known targets, including instantaneous effects
BISCUIT (2023)	Interactive temporal images	Learned variable/interaction correspondence	Binary interaction mechanisms

Appendix

Table A23: Assumptions and information available to related methods. A “concept anchor” is supervision that associates a visual coordinate with a known physical concept during training. More information does not imply a stronger method; these conditions determine whether a comparison is fair.

Work	Repr. learn.	Active exp.	Graph output/task	Representation constraints
CausalVAE	$\checkmark$	—	Static causal graph	Concept supervision and causal layer
ICF	$\triangle$	$\checkmark$	No explicit graph-recovery task	Independent controllability and intervention selectivity
Causal Curiosity	$\triangle$	$\checkmark$	No general explicit graph recovery	Experiments distinguish environment factors
CITRIS	$\checkmark$	—	$\triangle$ Subsequent temporal graph	Intervention-target constraints
iCITRIS	$\checkmark$	—	Instantaneous and temporal graphs	Intervention-target constraints
BISCUIT	$\checkmark$	—	$\triangle$ Subsequent temporal graph	Binary interaction-mechanism correspondence

Appendix

Table A24: Capabilities and representation constraints of related methods.

\checkmark

: explicitly included in the original method;

\triangle

: a different form or a subsequent module; —: not a component of the method, without ruling out extensions. The last column states what is constrained, rather than treating different tasks as a single type of calibration.

Causal Representation Learning

Causal representation learning seeks latent features that reveal underlying causal variables and mechanisms for robust prediction, intervention, and transfer.

See all 101 papers

Partial Causal Structure Learning for Valid Selective Conformal Inference under Interventions
Mar 2, 2026Amir Asiaee, Kavey Aryan, James P. LongSelective Inference Causal Structure Learning
Department of Biostatistics, Vanderbilt University Medical Center, TN 37203, USA · Department of Informatics, King’s College London, London, UK · Department of Biostatistics, MD Anderson Cancer Center, Houston, TX, USA
CausaLab: A Scalable Environment for Interactive Causal Discovery Toward AI Scientists
May 25, 2026Junlin Yang, Dylan Zhang, Xiangchen Song +7Structural Causal Models LLM Agent Evaluation
Tsinghua University · University of Illinois Urbana-Champaign · Carnegie Mellon University +2
From Prompts to Tokens: Internalizing Causal Supervision in Vision-Language Model for Multi-Image Causal Reasoning
Jun 10, 2026Haoping Yu, Yuanxi Li, Jing MaVisual Reasoning Causal Structure Learning
Department of Computer & Data Sciences, Case Western University, Cleveland, OH, US.

Mar 2, 2026J/K move · Enter open · S save

Partial Causal Structure Learning for Valid Selective Conformal Inference under Interventions

Amir Asiaee, Kavey Aryan, James P. Long

Department of Biostatistics, Vanderbilt University Medical Center, TN 37203, USA · Department of Informatics, King’s College London, London, UK · Department of Biostatistics, MD Anderson Cancer Center, Houston, TX, USA

Selective Inference Causal Structure Learning Conformal Prediction Genomics Causal Intervention

Open paper AskPDF arXiv

Selective conformal prediction can yield substantially tighter uncertainty sets when we can identify calibration examples that are exchangeable with the test example. In interventional settings, such as perturbation experiments in genomics, exchangeability often holds only within subsets of interventions that leave a target variable "unaffected" (e.g., non-descendants of an intervened node in a causal graph). We study the practical regime where this invariance structure is unknown and must be estimated from data. Our main result quantifies how coverage degrades when the estimated safe calibration set accidentally includes interventions that affect the target, and gives a conservative correction when an upper bound on this error is available. Rather than learning a full causal graph, we learn only the intervention-target relationships needed to choose calibration interventions. We give algorithms for this partial learning task and evaluate them on synthetic structural equation models and Replogle K562 CRISPR-interference data, where the experiments illustrate synthetic gains from selective calibration and finite-sample tradeoffs on real perturbation screens.

Abstract

Causal Representation Learning

Explore similar work

Partial Causal Structure Learning for Valid Selective Conformal Inference under Interventions

CausaLab: A Scalable Environment for Interactive Causal Discovery Toward AI Scientists

From Prompts to Tokens: Internalizing Causal Supervision in Vision-Language Model for Multi-Image Causal Reasoning