ACTIVA: Amortized Causal Effect Estimation via Variational Autoencoders
Authors: Andreas Sauter, Saber Salehkaleybar, Frank van Harmelen, Aske Plaat, Erman Acar
Organizations: Learning and Reasoning Group, Vrije Universiteit Amsterdam · Leiden Institute of Advanced Computer Science (LIACS), Leiden University · Informatics Institute & Institute for Logic, Language and Computation, University of Amsterdam
Predicting post-intervention distributions from observational data is central to many scientific and decision-making problems, but remains challenging due to causal ambiguity, restrictive modeling assumptions, and the lack of amortization across tasks. We introduce ACTIVA, a transformer-based conditional variational autoencoder for amortized estimation of full interventional distributions from observational data and intervention queries. ACTIVA learns a conditional latent prior that amortizes causal knowledge across diverse training tasks, so that a new causal model drawn from the training distribution is handled in a single forward pass, without interventional data, a causal graph, or per-task fitting. Beyond the training distribution, only a parametric shift of the simulator is tested. We provide a population-level characterization of ACTIVA's estimand, showing that, under idealized expressivity and infinite samples assumptions, ACTIVA's learning objective targets a simulator-weighted mixture over the interventional distributions of causal models that are observationally compatible with the input. Empirically, on synthetic datasets and biologically realistic gene-expression simulations, ACTIVA substantially outperforms a correlational baseline, reduces spurious non-descendant effects, and achieves competitive performance relative to strong amortized baselines. On a pair of observationally equivalent models sharing a single observational dataset, ACTIVA recovers the predicted mixture, validating the theory in a finite-capacity model. Our results show that ACTIVA is a promising approach for estimating interventional distributions from observational data.
Figures & tables
Figure 1: Causal models and their corresponding interventional distributions are generally ambiguous from observational data alone. Given observational data and an intervention query, ACTIVA encodes this ambiguity via a prior that represents the mixture over ambiguous models in the latent space. Through ACTIVA’s decoder, the latent prior distribution is then transformed into a mixture over the interventional distributions of the ambiguous causal models. Section 7.4 demonstrates this behavior empirically on a pair of observationally equivalent models.
Figure 2: Overview of the proposed model architecture. The orange blocks represent the observational stream, which processes observational data alongside the intervention representation to parameterize both the prior and the posterior. The green blocks represent the interventional stream, which processes interventional samples exclusively during training to inform the posterior. Finally, the blue blocks represent the decoder stream. In a brief walkthrough: inputs are embedded, processed via alternating MHSA blocks, and mean-pooled over the sample dimension to yield intermediate representations ( h′ and h′′ ). These representations parameterize the GMM prior and MVN posterior. A latent variable z is then sampled and passed through the transformer-based decoder to predict the final post-interventional mixture parameters ( bγ , μγ , Σγ ).
Figure 3: Samples from the observational and interventional distributions of the ground-truth model (first column), the baselines (columns 2-4), and our model (last column) for the Beta 8 data. Each row corresponds to a randomly sampled SCM from the test data. The colors indicate which variables have been intervened on ( do(Vi=5) ). All samples are projected on the V0 vs. V1 plane. The closer the distributions are to those in the first column, the more accurate the prediction is.
All
All ∖Vt
Desc.
Non-Desc.
Data
Approach
ERG ↓
NLL ↓
ERG ↓
NLL ↓
ERG ↓
NLL ↓
ERG ↓
NLL ↓
Gauss 2
Linear Causal (O)
—
—
0.09
0.90
0.32
1.39
0.02
0.73
ACTIVA (ours)
0.14
-1.57
0.14
0.90
0.48
1.41
0.03
0.78
MACE-TNP
0.16
-0.73
0.17
1.09
0.47
1.39
0.06
0.99
Gaussian Cond.
—
—
1.16
8.55
0.33
1.39
1.43
11.00
Beta 2
Linear Causal (PI)
—
—
0.16
0.82
0.31
1.44
0.01
0.19
Table 1: Average inference performance of our trained model and the baseline on held-out test according to various marginals explained in Section 6.3 . The Data column indicates which dataset has been used with how many variables. We report the average energy distance and average negative log likelihood over the causal models in the corresponding test sets. The Linear Causal and Nonlinear Causal approaches have privileged information (PI) by having access to the causal graph; the Linear Causal approach is additionally an asymptotic oracle (O) for the Gaussian datasets. Entries marked ∗ are obtained from a density fitted to the baseline’s own interventional samples, since the g-formula factorization admits no closed-form marginal over the descendant subset. Lower is better for all metrics.
Figure 4: Mean absolute error of the ATE of the predicted distributions w.r.t. the ground-truth sample ATE. The left panel shows the error on the full joint distribution (excluding the intervened variable), the middle panel shows the error for the causal descendants, and the right panel for the causal non-descendants of the intervention target. The different approaches and baselines are color-coded. Lower is better.
Figure 5: Behavior under provable non-identifiability, given only the shared observational input and the query do(X=5) . (a) Samples from the full C=20 component conditional prior (green). Dashed curves (orange and blue) are the two posteriors scaled by their branch rate, so they sum to the aggregated posterior. (b) Samples from the decoded prior (green) and ground-truth interventional samples scaled by their simulator weight, so they sum to the target in Equation 15 (dashed, orange and blue).
Counterfactual prediction estimates an individual's outcome under an alternative intervention given their factual observations. Such outcomes are generally not identifiable from observational data without additional assumptions. Even within the class of fully observed additive noise models (ANMs), different causal graphs can generate the same observational distribution yet imply different individual counterfactual outcomes. Predictions based on a single estimated graph ignore this structural uncertainty. We therefore target a Bayesian counterfactual posterior predictive distribution that combines predictions from plausible SCMs. We introduce CAFE (CounterfActual Prediction via Fast Posterior Estimation), an amortized inference framework that directly approximates the Bayesian counterfactual posterior predictive distribution. We pretrain a transformer-based model on synthetic counterfactual tasks generated from a diverse prior over ANMs. Given an observational dataset, an individual's factual observations, and an intervention, CAFE approximates the corresponding posterior predictive distribution in a single forward pass. Experiments show that CAFE accurately predicts individual counterfactual outcomes in identifiable settings and approximates the posterior predictive distribution when structural uncertainty induced by observationally indistinguishable causal graphs exists. Strong performance in realistic manufacturing and viticulture settings further demonstrates its empirical robustness beyond the assumptions of the training prior.
Deep generative models reproduce the observational distribution of their training data, inheriting any spurious associations it contains. A common source is an unobserved confounder that shapes both an attribute the user wants to control at sampling time and an attribute expected to vary in response. Existing causal generative approaches resolve the resulting ambiguity by imposing structural assumptions strong enough to single out one interventional distribution; in image domains, such assumptions are rarely warranted, and the data is generally consistent with a set of distinct causal mechanisms -- a feasible region of interventional distributions. We propose CauVaDE (Causal Variational Deep Embedding), built on a canonical augmented SCM in which the unobserved confounder collapses, without loss of generality, into a discrete latent cluster of bounded support while continuous variation is absorbed into independent noises. We prove that this canonical class is dense, in both observational and interventional Wasserstein distance, in the class of augmented SCMs compatible with a given causal diagram, and instantiate it as a mixture variational autoencoder whose cluster variable plays the role of the canonical confounder. An entropy regularizer with weight γ on the cluster posterior then traces a family of candidate causal effects that fit the observational data to comparable likelihood while spanning the feasible region. Experiments on image data benchmarks show that CauVaDE produces diverse interventional samples and improves FID against an unconfounded reference.
Jingyuan Chen, Kangrui Ruan, Junzhe Zhang
Department of EECS Syracuse University · AWS AI Lab
Mean-reverting dynamics are pervasive in finance, and the Cox--Ingersoll--Ross (CIR) process is a standard model for the time series they produce, from short rates to credit default swap (CDS) spreads. Yet CIR models capture only correlated co-movement, not causal influence between series, so they cannot answer the system's response when one series is externally shocked, which observational conditionals confound with historical co-movement. We make two contributions. First, an amortized model for distributional causal effect estimation that frames trajectories as time-stamped observations and predicts the calibrated multi-horizon shock response without retraining per scenario. Second, a causal multivariate CIR data-generating process that supplies the paired observational and interventional ground truth that real markets cannot. We instantiate and calibrate the framework on CDS spreads as a testbed. CIR-ACTIVA's validity is established on synthetic ground truth, independent of how well the simulator matches reality, while practical grounding is assessed by backtesting the generated traces against real CDS data. Against observational and amortized causal-inference baselines, CIR-ACTIVA leads on both causal selectivity in the joint distribution and horizon-resolved calibration, retaining its selectivity once the interventional law varies over the horizon, with gains concentrating at short horizons. This opens up a class of what-if queries on coupled spread systems, CDS stress testing among them, that observational forecasters cannot answer.
Andreas Sauter, Sumit Sourabh, Drona Kandhai +1
Vrije Universiteit Amsterdam · ING · University of Amsterdam / ING +1