When Reconstruction Is Not Enough: Doubly Robust Counterfactual and Interventional Mediation Analysis with Learned Multimodal Representations

Cong Cao

cs.LGOct 21, 2026

When Reconstruction Is Not Enough: Doubly Robust Counterfactual and Interventional Mediation Analysis with Learned Multimodal Representations

Authors: Cong Cao

Organizations: Department of Biostatistics, Yale School of Public Health, New Haven, CT 06520, USA · Department of Surgery, Stanford University School of Medicine, Stanford, CA 94305, USA

Abstract

Learned representations of high-dimensional multimodal health records are increasingly used to adjust for confounding, yet they are usually trained to reconstruct the data rather than preserve the information that causal identification requires. We study counterfactual outcomes under alternative treatments, including phenotype-specific effects and patient-level counterfactual risk profiles of the kind digital twins provide, and interventional direct and indirect effects under exposure-induced intermediate confounding. We give conditions under which a representation preserves identification and show that interventional mediation is more demanding than counterfactual comparison. We prove that no covariate-only training objective, whether reconstruction, contrastive, or physics-informed, can certify that a representation is sufficient, and that, in a linear structural model, a representation can lose arbitrarily little reconstruction accuracy yet induce arbitrarily large bias. We decompose the error of cross-fitted doubly robust estimators into a first-order representation bias, which neither double robustness nor cross-fitting removes, and a second-order remainder of nuisance-estimation errors, so coverage worsens as the sample size grows when the representation is insufficient. We propose a workflow combining modality-specific variational autoencoders trained outside the estimation sample, cross-fitted doubly robust estimators, locked phenotypes, select-then-confirm pathway analysis, portable representations, and diagnostics for representation insufficiency, exposure leakage, and sensitivity. In simulations, discarding a rare, moderately strong confounder gave indirect-effect intervals with 2% coverage at n = 3,000; reconstruction loss missed the problem, and a simple predictive diagnostic flagged it in every replicate.

Figures & tables

$\kappa=0$
	$n=1,000$			$n=3,000$
Method	Bias	RMSE	Cov.	Bias	RMSE	Cov.
Oracle $(U,R)$	0.011	0.082	0.935	$-$ 0.000	0.046	0.955
Raw covariates $W$	0.020	0.076	0.900	0.002	0.043	0.940
VAE (R1), $d=4$	0.011	0.084	0.935	$-$ 0.001	0.047	0.950
VAE (R1), $d$ by nuisance loss	0.010	0.085	0.935	$-$ 0.001	0.047	0.950

Table 1: Estimation of the IIE across methods, confounding strength of the lost code (

\kappa

), and sample size.

Figure 1: Coverage of nominal 95% intervals for the IIE by sample size, for each confounding strength

\kappa

of the lost code. The dashed line marks 0.95; 200 replicates per setting.

			$n=1,000$			$n=3,000$
$\kappa$	$d$	Recon. loss	Bias	Cov.	Chosen	Bias	Cov.	Chosen
0	1	31.00	$-$ 0.097	0.830	0%	$-$ 0.107	0.515	0%
0	2	25.19	$-$ 0.045	0.920	2%	$-$ 0.056	0.805	0%
0	4	18.55	0.011	0.935	98%	$-$ 0.001	0.950	100%
0	8	18.49	0.007	0.940	0%	$-$ 0.000	0.945	0%
0.5	1	31.00	$-$ 0.063	0.880	0%	$-$ 0.059	0.800	0%

Table 2: Latent dimension, reconstruction loss, and estimation of the IIE (regime R1). “Chosen” is the share of replicates in which the nuisance-loss rule D4 selected that dimension; the reconstruction rule selects the dimension with the lowest reconstruction loss in every replicate.

Figure 2: Absolute bias of the IIE against validation reconstruction loss across latent dimensions

d

of the (R1) VAE, by confounding strength

\kappa

and sample size

n

$\kappa$	$n$	D1, $d=1$	D1, $d=4$	D2 AUC, VAE	D2 AUC, raw	D3 range
0	1,000	100%	2%	0.678	0.683	0.127
0	3,000	100%	0%	0.683	0.685	0.110
0.5	1,000	100%	28%	0.671	0.678	0.123
0.5	3,000	100%	99%	0.683	0.687	0.109
1	1,000	100%	100%	0.670	0.685	0.127
1	3,000	100%	100%	0.681	0.696	0.112

Table 3: Diagnostics. D1: share of replicates in which adding the raw covariates significantly improved a nuisance model (Holm-adjusted, 5% level). D2: cross-validated area under the curve for predicting

A

. D3: mean range of IIE estimates across latent dimensions and PCA within a replicate.

		$n=1,000$					$n=3,000$
$\kappa$	Method	ATE bias	ATE cov.	$\tau(s)$ cov.	Risk RMSE	Agree	ATE bias	ATE cov.	$\tau(s)$ cov.	Risk RMSE	Agree
0	Oracle $(U,R)$	0.014	0.960	0.955	0.053	0.893	$-$ 0.001	0.935	0.953	0.030	0.936
0	Raw covariates $W$	0.033	0.950	0.942	0.066	0.834	0.006	0.930	0.953	0.042	0.906
0	VAE (R1), $d=4$	0.016	0.975	0.953	0.079	0.841	0.000	0.930	0.955	0.046	0.901
0	VAE (R1) $+$ key covariates	0.017	0.970	0.957	0.089	0.826	0.000	0.930	0.955	0.051	0.891
0.5	Oracle $(U,R)$	$-$ 0.012	0.955	0.948	0.053	0.901	0.001	0.940	0.960	0.030	0.943

Table 4: Counterfactual comparisons: average effect (bias, coverage), phenotype-specific effects (mean coverage across phenotypes), counterfactual risk profiles (root mean squared error against the true risks), and agreement on which failure is most likely under each option.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Specification
Latent factors	$U\sim N_{4}(0,I)$
Laboratory modality	20 variables, $W^{(1)}=\Lambda_{1}U+e_{1}$ , $e_{1}\sim N(0,0.1^{2}I)$
Diagnosis modality	30 binary codes, logistic in $U$ , plus $R\sim\mathrm{Bernoulli}(0.1)$
Survey modality	10 four-level items, thresholded $\Lambda_{3}U+e_{3}$ , $e_{3}\sim N(0,0.7^{2}I)$
Loadings	entries $N(0,0.8^{2})$ , drawn once (seed 20261020); diagnosis intercepts $U(-2,-0.5)$
Exposure $A$	logit: $\gamma=(0.5,\ -0.4,\ 0.3,\ 0.2)$ , $\gamma_{R}=1$

Appendix

Table S1: Fixed parameters of the data-generating mechanism.

$\kappa=0$
	$n=1,000$			$n=3,000$
Method	Bias	RMSE	Cov.	Bias	RMSE	Cov.
Oracle $(U,R)$	0.014	0.092	0.960	$-$ 0.001	0.053	0.935
Raw covariates $W$	0.033	0.095	0.950	0.006	0.052	0.930
VAE (R1), $d=4$	0.016	0.093	0.975	0.000	0.052	0.930
VAE (R1), $d$ by nuisance loss	0.017	0.094	0.975	0.000	0.052	0.930

Appendix

Table S2: Estimation of the ATE across methods, confounding strength of the lost code (

\kappa

), and sample size.

$\kappa=0$
	$n=1,000$			$n=3,000$
Method	Bias	RMSE	Cov.	Bias	RMSE	Cov.
Oracle $(U,R)$	0.003	0.086	0.975	0.000	0.059	0.935
Raw covariates $W$	0.019	0.080	0.940	0.007	0.055	0.910
VAE (R1), $d=4$	0.005	0.089	0.990	0.003	0.059	0.940
VAE (R1), $d$ by nuisance loss	0.007	0.091	0.980	0.003	0.059	0.940

Appendix

Table S3: Estimation of the IDE across methods, confounding strength of the lost code (

\kappa

), and sample size.

$\kappa$	$n$	Procedure	FWER	Power	Selection bias
0	1,000	Same-sample ranking	52%	96%	0.044
0	1,000	Select-then-confirm	0%	47%	0.030
0	3,000	Same-sample ranking	56%	100%	0.024
0	3,000	Select-then-confirm	0%	99%	0.006
1	1,000	Same-sample ranking	100%	94%	0.358
1	1,000	Select-then-confirm	21%	68%	0.329

Appendix

Table S4: Pathway selection among 20 candidate mediators (two true). FWER: share of replicates with at least one null pathway declared significant; power: share of the two true pathways declared significant; selection bias: mean of

|\hat{\theta}|-|\theta|

over selected pathways.

$\kappa$	$n$	$\Delta_{\mathrm{port}}$	0 in CI	Source bias	Target bias	Target cov.
0	1,000	$-$ 0.001	95%	$-$ 0.002	0.003	0.965
0	3,000	0.000	97%	$-$ 0.002	0.001	0.970
1	1,000	0.000	95%	0.185	0.180	0.300
1	3,000	0.000	97%	0.182	0.176	0.020

Appendix

Table S5: Portable representation (lab and diagnosis encoders only).

\Delta_{\mathrm{port}}

: mean difference in the IIE between the full and portable representations in the source; “0 in CI”: share of replicates whose interval for

\Delta_{\mathrm{port}}

contains zero. Target: shifted population without the survey modality.

Causal Effect Estimation

Causal effect estimation measures how outcomes would change under interventions, using experiments or observational data and explicit assumptions.

See all 264 papers

Jul 30, 2026

Doubly Robust Functional Representation Learning for Longitudinal Causal Inference with Irregular Histories

Longitudinal causal studies often record histories as irregular functional fragments: laboratory values, physiologic signals, sensor streams, and image-derived summaries measured at unequal and informative times. Standard doubly robust estimators usually require scalar summaries, whereas sequence learners optimize prediction losses that need not stabilize the efficient influence function. We propose Doubly Robust Functional Representation Learning (DR-FRL), a cross-fitted workflow that turns irregular histories into estimand-targeted states for observed-history regimes. Functional and temporal encoders map point clouds and prior histories into states; nuisance heads estimate outcome, treatment, and censoring functions; and EIF-targeted validation, calibration, overlap, tail, and ablation diagnostics assess whether the state supports the estimating equation. If the selected state preserves the nuisance information needed by the EIF, representation error enters the same second-order product remainder as ordinary nuisance error, and the mean estimator is asymptotically linear under explicit rate, overlap, calibration, and stability conditions. Catoni aggregation is treated separately as a bounded-influence point estimator, not a replacement for Wald inference. Simulations show gains when functional confounding is high-dimensional, measurement is informative, support is weak, or pseudo-outcomes are heavy-tailed. A VitalDB audit shows that DR-FRL can use irregular laboratory point clouds and deliver a useful negative finding: for this ICU-disposition endpoint, scalar laboratory summaries already carry much endpoint-relevant information.

Mengfei Ran, Yifeng Shen, Ruijie Guan

Academy of Life and Natural Sciences, Xi’an Jiaotong-Liverpool University, Suzhou, China · Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, China

Causal Representation Learning Causal Inference Doubly Robust Estimation

Ask PDF

Jun 5, 2026

MediEncoder: Nonlinear Representation Learning for High-Dimensional Causal Mediation Analysis

Causal mediation analysis decomposes a treatment effect into indirect pathways through mediators and direct pathways not operating through them. Modern biomedical studies often involve high-dimensional covariates and mediators that are noisy proxies for lower-dimensional latent biological processes. Existing methods typically rely on sparsity, linear factor models, or ignore the connection among variables in the learned representations, which can be restrictive when measurements are nonlinear and covariate and mediator factors are structurally dependent. We propose MediEncoder, a representation-learning framework for nonlinear high-dimensional mediation analysis. MediEncoder jointly learns low-dimensional covariate and mediator representations using a coupled encoder-decoder architecture with a cross-factor network that links treatment and covariate representations to mediator representations. The learned features are then used in a cross-fitted efficient influence function-based estimator of natural direct and indirect effects. The resulting estimator is multiply robust and asymptotically normal under suitable regularity conditions. Simulations show that MediEncoder improves estimation accuracy over competing dimension-reduction approaches, and an application to Alzheimer's Disease Neuroimaging Initiative data illustrates its utility in high-dimensional biomedical causal mediation analysis.

Shi Bo, Debarghya Mukherjee, AmirEmad Ghassami

Department of Mathematics and Statistics, Boston University

Causal Effect Estimation Representation Learning Causal Representation Learning Causal Mediation Analysis Causal Inference

Ask PDF

May 3, 2026

Distributional Causal Mediation via Conditional Generative Modeling

Mediation analysis has traditionally focused on outcome-level summary contrasts, such as mean effects, which may obscure substantial distributional changes induced by complex and nonlinear causal mechanisms. We propose Distributional Causal Mediation Analysis (DCMA), a generative learning framework for identifying and estimating treatment effects on entire outcome distributions transmitted through multiple mediators. DCMA learns conditional generative models for the mediators and the outcome, recovering the relevant conditional distributions from observational data. Leveraging the identification formulas, it reconstructs interventional outcome distributions via Monte Carlo forward simulation by noise resampling, enabling the capture of both classical summary effects and rich distributional contrasts such as energy distance and the Wasserstein distance. Analytical error bounds are derived to decompose how estimation errors in the learned conditional models propagate to the reconstructed interventional outcome distributions. The empirical effectiveness of DCMA is demonstrated through numerical experiments and real-world data applications.

Jinlun Zhang, Haoneng Huang, Zishu Zhan +1

State Key Laboratory of Multi-organ Injury Prevention and Treatment, Department of Biostatistics, Guangdong Provincial Key Laboratory of Tropical Disease Research, School of Public Health, Southern Medical University, Guangzhou, China

Causal Effect Estimation Causal Mediation Analysis Causal Inference

Ask PDF

Abstract

Causal Effect Estimation

Explore similar work

Doubly Robust Functional Representation Learning for Longitudinal Causal Inference with Irregular Histories

MediEncoder: Nonlinear Representation Learning for High-Dimensional Causal Mediation Analysis

Distributional Causal Mediation via Conditional Generative Modeling