ACTIVA: Amortized Causal Effect Estimation via Variational Autoencoders

Andreas Sauter; Saber Salehkaleybar; Frank van Harmelen; Aske Plaat; Erman Acar

cs.LGMar 3, 2025

ACTIVA: Amortized Causal Effect Estimation via Variational Autoencoders

Authors: Andreas Sauter, Saber Salehkaleybar, Frank van Harmelen, Aske Plaat, Erman Acar

Organizations: Learning and Reasoning Group, Vrije Universiteit Amsterdam · Leiden Institute of Advanced Computer Science (LIACS), Leiden University · Informatics Institute & Institute for Logic, Language and Computation, University of Amsterdam

Abstract

Predicting post-intervention distributions from observational data is central to many scientific and decision-making problems, but remains challenging due to causal ambiguity, restrictive modeling assumptions, and the lack of amortization across tasks. We introduce ACTIVA, a transformer-based conditional variational autoencoder for amortized estimation of full interventional distributions from observational data and intervention queries. ACTIVA learns a conditional latent prior that amortizes causal knowledge across diverse training tasks, so that a new causal model drawn from the training distribution is handled in a single forward pass, without interventional data, a causal graph, or per-task fitting. Beyond the training distribution, only a parametric shift of the simulator is tested. We provide a population-level characterization of ACTIVA's estimand, showing that, under idealized expressivity and infinite samples assumptions, ACTIVA's learning objective targets a simulator-weighted mixture over the interventional distributions of causal models that are observationally compatible with the input. Empirically, on synthetic datasets and biologically realistic gene-expression simulations, ACTIVA substantially outperforms a correlational baseline, reduces spurious non-descendant effects, and achieves competitive performance relative to strong amortized baselines. On a pair of observationally equivalent models sharing a single observational dataset, ACTIVA recovers the predicted mixture, validating the theory in a finite-capacity model. Our results show that ACTIVA is a promising approach for estimating interventional distributions from observational data.

Figures & tables

Figure 1: Causal models and their corresponding interventional distributions are generally ambiguous from observational data alone. Given observational data and an intervention query, ACTIVA encodes this ambiguity via a prior that represents the mixture over ambiguous models in the latent space. Through ACTIVA’s decoder, the latent prior distribution is then transformed into a mixture over the interventional distributions of the ambiguous causal models. Section 7.4 demonstrates this behavior empirically on a pair of observationally equivalent models.

Figure 2: Overview of the proposed model architecture. The orange blocks represent the observational stream, which processes observational data alongside the intervention representation to parameterize both the prior and the posterior. The green blocks represent the interventional stream, which processes interventional samples exclusively during training to inform the posterior. Finally, the blue blocks represent the decoder stream. In a brief walkthrough: inputs are embedded, processed via alternating MHSA blocks, and mean-pooled over the sample dimension to yield intermediate representations (

h^{\prime}

and

h^{\prime\prime}

). These representations parameterize the GMM prior and MVN posterior. A latent variable

z

is then sampled and passed through the transformer-based decoder to predict the final post-interventional mixture parameters (

b_{\gamma}

\mu_{\gamma}

\Sigma_{\gamma}

Figure 3: Samples from the observational and interventional distributions of the ground-truth model (first column), the baselines (columns 2-4), and our model (last column) for the Beta 8 data. Each row corresponds to a randomly sampled SCM from the test data. The colors indicate which variables have been intervened on ( $do(V_{i}=5)$ ). All samples are projected on the $V_{0}$ vs. $V_{1}$ plane. The closer the distributions are to those in the first column, the more accurate the prediction is.

		All		All $\setminus\bm{V_{t}}$		Desc.		Non-Desc.
Data	Approach	ERG $\downarrow$	NLL $\downarrow$	ERG $\downarrow$	NLL $\downarrow$	ERG $\downarrow$	NLL $\downarrow$	ERG $\downarrow$	NLL $\downarrow$
Gauss 2	Linear Causal (O)	—	—	0.09	0.90	0.32	1.39	0.02	0.73
	ACTIVA (ours)	0.14	-1.57	0.14	0.90	0.48	1.41	0.03	0.78
	MACE-TNP	0.16	-0.73	0.17	1.09	0.47	1.39	0.06	0.99
	Gaussian Cond.	—	—	1.16	8.55	0.33	1.39	1.43	11.00
Beta 2	Linear Causal (PI)	—	—	0.16	0.82	0.31	1.44	0.01	0.19

Table 1: Average inference performance of our trained model and the baseline on held-out test according to various marginals explained in Section 6.3 . The Data column indicates which dataset has been used with how many variables. We report the average energy distance and average negative log likelihood over the causal models in the corresponding test sets. The Linear Causal and Nonlinear Causal approaches have privileged information (PI) by having access to the causal graph; the Linear Causal approach is additionally an asymptotic oracle (O) for the Gaussian datasets. Entries marked ∗ are obtained from a density fitted to the baseline’s own interventional samples, since the g-formula factorization admits no closed-form marginal over the descendant subset. Lower is better for all metrics.

Figure 4: Mean absolute error of the ATE of the predicted distributions w.r.t. the ground-truth sample ATE. The left panel shows the error on the full joint distribution (excluding the intervened variable), the middle panel shows the error for the causal descendants, and the right panel for the causal non-descendants of the intervention target. The different approaches and baselines are color-coded. Lower is better.

Panel of Figure 5 — Figure 5: Behavior under provable non-identifiability, given only the shared observational input and the query $do(X\!=\!5)$ . (a) Samples from the full $C=20$ component conditional prior (green). Dashed curves (orange and blue) are the two posteriors scaled by their branch rate, so they sum to the aggregated posterior. (b) Samples from the decoded prior (green) and ground-truth interventional samples scaled by their simulator weight, so they sum to the target in Equation 15 (dashed, orange and blue).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

parameter	in-distribution	out-of-distribution
genes	8	11
b	Uniform(0, 1)	Uniform(0.5, 2.0)
k_param	Uniform(1, 5)	Uniform(3, 7)
k_sign	Beta(1, 1)	Beta(0.5, 0.5)
hill	$\in$ { 1.9, 2.0, 2.1}	$\in$ { 1.5, 2.5}
decays	$\in$ { 0.7, 0.8, 0.9}	$\in$ { 0.5, 1.5}

Appendix

Table 2: SERGIO data generation parameters.

Causal Effect Estimation

Causal effect estimation measures how outcomes would change under interventions, using experiments or observational data and explicit assumptions.

See all 264 papers

CAFE: Counterfactual Prediction via Fast Posterior Estimation
Sep 26, 2026Xinyan Han, Xiaoyu Lin, Hao Zou +3Counterfactual Prediction Structural Causal Models
Tsinghua University · Stable AI
Causal Variational Deep Embedding: A Family of Interventional Generators for Confounded Images
Jun 19, 2026Jingyuan Chen, Kangrui Ruan, Junzhe ZhangVariational Autoencoders Structural Causal Models
Department of EECS Syracuse University · AWS AI Lab
Amortized Interventional Forecasting for Multivariate CIR Processes
Aug 4, 2026Andreas Sauter, Sumit Sourabh, Drona Kandhai +1Causal Effect Estimation Quantitative Finance
Vrije Universiteit Amsterdam · ING · University of Amsterdam / ING +1

Sep 26, 2026J/K move · Enter open · S save

CAFE: Counterfactual Prediction via Fast Posterior Estimation

Xinyan Han, Xiaoyu Lin, Hao Zou, Xingxuan Zhang+2

Tsinghua University · Stable AI

Counterfactual Prediction Structural Causal Models

Open paper AskPDF arXiv

Counterfactual prediction estimates an individual's outcome under an alternative intervention given their factual observations. Such outcomes are generally not identifiable from observational data without additional assumptions. Even within the class of fully observed additive noise models (ANMs), different causal graphs can generate the same observational distribution yet imply different individual counterfactual outcomes. Predictions based on a single estimated graph ignore this structural uncertainty. We therefore target a Bayesian counterfactual posterior predictive distribution that combines predictions from plausible SCMs. We introduce CAFE (CounterfActual Prediction via Fast Posterior Estimation), an amortized inference framework that directly approximates the Bayesian counterfactual posterior predictive distribution. We pretrain a transformer-based model on synthetic counterfactual tasks generated from a diverse prior over ANMs. Given an observational dataset, an individual's factual observations, and an intervention, CAFE approximates the corresponding posterior predictive distribution in a single forward pass. Experiments show that CAFE accurately predicts individual counterfactual outcomes in identifiable settings and approximates the posterior predictive distribution when structural uncertainty induced by observationally indistinguishable causal graphs exists. Strong performance in realistic manufacturing and viticulture settings further demonstrates its empirical robustness beyond the assumptions of the training prior.

Abstract

Causal Effect Estimation

Explore similar work

CAFE: Counterfactual Prediction via Fast Posterior Estimation

Causal Variational Deep Embedding: A Family of Interventional Generators for Confounded Images

Amortized Interventional Forecasting for Multivariate CIR Processes