Organizations: Equal contribution · Technische Universität Berlin · Department of Computer Science, University of Potsdam · German Research Center for Artificial Intelligence (DFKI)
We introduce structural causal bottleneck models (SCBMs), a novel class of structural causal models in which causal effects between high-dimensional variables are mediated by low-dimensional summary statistics, or bottlenecks. SCBMs provide a flexible framework for mechanism-specific, target-dependent dimension reduction while remaining estimable via standard learning algorithms. We prove that bottleneck variables are identifiable up to bijection from observational data, and validate this experimentally. We then study causal effect estimation in linear SCBMs under three standard adjustment strategies. For instrumental variables, we identify a population-level non-identifiability of naïve two-stage least squares when all variables are high-dimensional, and show that bottleneck representations restore identification. For mediation and backdoor adjustment, we prove that conditioning on bottleneck variables yields valid effect estimates with explicit finite-sample efficiency gains over conditioning on full high-dimensional variables. We argue that SCBMs provide a principled alternative to existing causal dimension reduction frameworks such as causal representation learning and causal abstractions.
Figures & tables
Figure 1: A factored SCBM.
Figure 2: Left: Mean ± std of avg. R2 for bottleneck recovery across settings; top row linear, bottom row nonlinear SCBMs. High scores confirm recovery up to bijection. Right: Learned bottleneck spaces Z^ vs. ground-truth Z , related by an affine map (linear) or smooth bijection (nonlinear).
Figure 3: Mean R2± 95% CI for varying assumed bottleneck dimension dZ^ ; true dZ indicated by dashed line.
Figure 4: Left: Structural equations of a three-node SCBM with shared root X1∈R2 . Right: Input-space partitions (left column) and output spaces (right column) induced by b^12 (top) and b^13 (bottom). Colors denote bottleneck-equivalent, hence causally equivalent, states. The two bottlenecks induce fundamentally different partitions of the same X1 —concentric rings vs. quadrants—invariant to bijective reparametrizations (Lemma 3.2 ). In the output spaces, only transitions across partition boundaries produce changes in the downstream distribution.
Figure 5: Left: Mediator adjustment; Right: backdoor adjustment. MSE of MXY vs. ntrain (log-log) for conditioning on the full high-dimensional variable ( blue , d=100 ) vs. the bottleneck ( red , dZ=5 ). Bottleneck conditioning yields a consistent finite-sample advantage, with both estimators converging for large ntrain . Results of additional parameter configurations provided in Appendix D.2 .
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Visualizations of maps from ground truth bottleneck space Z (first plot) to different learned bottleneck spaces Z^nonlin . The second and third plots show examples of successfully identified bottlenecks, as visualized by the smooth, structure preserving bijections from ground-truth to learned bottleneck spaces. We additionally show the identifiability metric, which in both cases is close to one. The third and fourth plots show examples where identifiability is not achieved (by not training to convergence). For such non-identified cases, the learned map is not a bijection and the local structure of the ground-truth bottleneck space is not preserved. The low R2 metric in these failure cases further reflects non-identification of the ground-truth bottleneck.
Figure 7: Results of the mediator adjustment experiment across various parameter configurations. Bottlenecks are preferable as conditioning variables over high-dimensional variables across settings. As dX/dZ→1 , the finite sample benefits of bottlenecks diminish, with the notable exception of dX=500 .
Figure 8: Results of the backdoor adjustment experiment across various parameter configurations. Bottlenecks are preferable as conditioning variables over high-dimensional variables across settings. As dX/dZ→1 , the finite sample benefits of bottlenecks diminish. Interestingly, for intermediate values of dX/dZ (middle two columns) the gap in estimation error of MXY when using the bottleneck vs. the high-dimensional confounder persists, even for large ntrain .
npilot
Oracle b-2SLS
Estimated b-2SLS
Ridge 2SLS
Pilot-only OLS
100
1.2×10−5
8.0
1×103
1.1
500
1.2×10−5
7.9×10−2
1×103
1.3×10−1
2000
1.2×10−5
4.9×10−3
1×103
2.6×10−2
Appendix
Table 1: Median MSE of MXY over 10 seeds for varying pilot sizes ( nobs=30000 , d=50 , r=2 ).
Figure 9: Backdoor adjustment under approximately low-rank MCY . MSE of MXY vs. misspecification ρ (log-log; ρ>0 shown) for conditioning on the full confounder C ( blue ) vs. the rank- dZ bottleneck ZCY ( red ), for dZ=2 (solid) and dZ=5 (dashed). Rows: dX ; columns: ntrain . Lines show medians and bands interquartile ranges over 10 seeds. Full conditioning is unaffected by ρ , while the bottleneck loses its advantage at ρ of order 10−7 – 10−5 .
Figure 10: Crossover ρ∗ above which conditioning on the bottleneck is worse than conditioning on the full confounder. Left: ρ∗ vs. ntrain ; ρ∗ decreases with sample size because the misspecification bias does not vanish. Right: ρ∗ vs. outcome noise variance σY2 ( dX=100 , dZ=5 , ntrain=5000 ); ρ∗ scales linearly with σY2 . Markers show medians and error bars interquartile ranges of the per-seed crossover over 10 seeds.
N
Full
Split
Cross-fit
Oracle
Mediator
103
3.3±0.64
6.2±1.4
3.1±0.52
2.8±0.47
5⋅103
0.52±0.046
0.98±0.060
0.50±0.053
0.49±0.049
3⋅104
0.087±0.014
0.17±0.027
0.085±0.013
0.084±0.013
105
0.025±0.0046
0.048±0.0075
0.025±0.0039
0.025±0.0038
3⋅105
0.0090±0.0020
0.017±0.0030
0.0087±0.0020
0.0087±0.0020
Appendix
Table 2: MSE of MXY under a fixed sample budget N (mean ± std over 10 seeds).
Figure 11: Three-node, fully connected graph.
Hyperparameter
Value
Encoder hidden layer sizes
[256,128,64,64,32]
Decoder hidden layer sizes
[32,64,64,128,256]
Epochs
500
Batch size
1024
Optimizer
AdamW [ Loshchilov and Hutter, 2019 ]
Learning rate scheduler
Cosine decay w/ warmup
Appendix
Table 3: Hyperparameters for identifiability and misspecification experiments.
Structural Causal Models (SCMs) provide a principled framework for observational, interventional, and counterfactual reasoning, all of which rely on probabilistic inference. Existing frameworks either transform SCMs into alternative tractable representations or rely on approximate inference with expressive generative models. We instead perform inference using a message passing scheme directly on the SCM by representing structural mechanisms and probability densities in a common finite-dimensional function space. Forward propagation combines a numerical realization of the Law of the Unconscious Statistician, while propagated densities are recovered by spline projection from LOTUS moments. The same representation naturally yields backward message passing for conditioning, providing a unified framework for causal inference. Using tensor-product B-spline bases, the resulting algorithm expresses inference through numerical quadrature and linear algebra without explicitly constructing conditional distributions. Experiments on randomly generated continuous SCMs demonstrate accurate approximation of Monte Carlo reference solutions and systematic improvement with increasing basis resolution.
Causal foundation models (CFMs) pre-trained on data generated from various structural causal models (SCMs) have been proposed for estimating causal effects from observational data. However, differences in pre-training environments and evaluation protocols make it difficult to assess how their performance depends on the information available for causal identification. To enable controlled comparisons, we introduce CausalIDView, a multi-view benchmark that holds fixed SCM realization and target estimand while varying only the observational view available to the estimator. Each observational view corresponds to a distinct identification regime under the benchmark's maintained causal assumptions. Across these matched views, no CFM consistently performs best and model rankings vary substantially. Under controlled structural changes, CFMs exhibit model-specific failures to maintain stable estimates when true effects are unchanged and to track genuine effect changes. We also examine whether combining explicit identification with strong predictive estimation is effective. A modular approach that pairs a predictive tabular foundation model with regime-specific identification procedures is competitive with CFMs and outperforms several of them. These findings motivate cross-regime comparisons to assess the empirical value of CFMs.
Heejin Jung, Gyeongdeok Seo, Hoyoon Byun +2
Yonsei University · University of Illinois Urbana–Champaign
Causal abstractions formalize when a high-level structural causal model (SCM) captures the interventional behavior of a lower-level SCM. Existing applications of this notion largely follow a hypothesis-testing paradigm: an expert proposes a candidate high-level model and then evaluates if the low-level system implements it. We study the complementary problem of learning a high-level model directly from low-level measurements. Our contributions leverage hypotheses from low-rank causal discovery, and can be summarized as follows: (1) we show that observations generated by a low-rank graph induce latents that form a causal abstraction, (2) we provide identifiability results about these latents, and (3) we propose a practical objective to learn this high-level SCM.
Théo Saulus, Simon Lacoste-Julien, Dhanya Sridhar
Mila - Quebec AI Institute · Université de Montréal · Canada CIFAR AI Chair