Beyond Distributional Fidelity: Causal-Penalized Diffusion for Synthetic Tabular Data

Lan Tao; Yongxian He; Shirong Xu; Yidong Ouyang; Guang Cheng

stat.MLOct 8, 2026

Beyond Distributional Fidelity: Causal-Penalized Diffusion for Synthetic Tabular Data

Authors: Lan Tao, Yongxian He, Shirong Xu, Yidong Ouyang, Guang Cheng

Organizations: Department of Statistics & Data Science, University of California, Los Angeles · Herbert Wertheim School of Public Health and Human Longevity Science, University of California, San Diego · Wang Yanan Institute for Studies in Economics, Xiamen University

Abstract

Synthetic tabular generators are commonly optimized for distributional fidelity, but statistical similarity alone does not guarantee preservation of causal effects. In this paper, we study whether causal fidelity can be improved directly within a fully generative tabular model. Causal Fidelity is defined with respect to a target estimand as the discrepancy between inferential distributions obtained from real and synthetic data, and theoretical results show that high statistical fidelity does not generally imply high causal fidelity. We then propose a causal-fidelity-aware training framework which adds a causal discrepancy penalty to the generative objective. The framework is instantiated with a causal-penalized TabDDPM and optimized using an on-policy score-function estimator. We further establish conditions under which causal regularization improves expected causal fidelity. Experiments across diverse treatment-effect simulations and two benchmark datasets evaluate the ability of our method to improve causal fidelity while preserving competitive statistical fidelity.

Figures & tables

Figure 1: ATE posterior preservation for CTGAN, TVAE, and TabDDPM. For each generator, 10 independent models are trained on the same reference dataset and 10 synthetic datasets are sampled from each model, yielding 100 synthetic datasets per generator. CausalPFN is used to infer the ATE posterior from both the reference and synthetic datasets. Red denotes the reference posterior and blue denotes the synthetic-data posteriors.

Figure 2: ATE posterior distributions estimated from real and synthetic data in the linear simulation setting with true ATE $\tau=-2$ . Synthetic datasets are generated using vanilla TabDDPM (left) and the proposed causal-penalized TabDDPM (right). The causal-fidelity penalty shifts the synthetic-data posterior toward the posterior estimated from the real data and the true ATE.

Setting	Method	Num. W 1 $\downarrow$	Cat. JS $\downarrow$	T TVD $\downarrow$	Y W 1 $\downarrow$	Corr. L2 $\downarrow$	Post. W 1 $\downarrow$	Mean Gap $\downarrow$	ATE RMSE $\downarrow$
		Distributional fidelity					Causal fidelity
Linear	TabDDPM	$0.877_{\pm 0.021}$	$0.065_{\pm 0.003}$	$0.016_{\pm 0.007}$	$1.899_{\pm 0.048}$	$5.115_{\pm 0.159}$	$0.336_{\pm 0.086}$	$0.336_{\pm 0.086}$	0.302
Linear	Causal-TabDDPM	$\mathbf{0.737}_{\pm 0.017}$	$\mathbf{0.044}_{\pm 0.003}$	$\mathbf{0.006}_{\pm 0.005}$	$\mathbf{1.596}_{\pm 0.042}$	$\mathbf{4.967}_{\pm 0.140}$	$\mathbf{0.123}_{\pm 0.093}$	$\mathbf{0.122}_{\pm 0.094}$	0.179
Nonlinear	TabDDPM	$\mathbf{0.688}_{\pm 0.016}$	$0.043_{\pm 0.002}$	$\mathbf{0.068}_{\pm 0.010}$	$\mathbf{2.120}_{\pm 0.061}$	$\mathbf{3.604}_{\pm 0.095}$	$0.213_{\pm 0.107}$	$0.212_{\pm 0.108}$	0.233
Nonlinear	Causal-TabDDPM	$0.759_{\pm 0.020}$	$\mathbf{0.040}_{\pm 0.004}$	$0.086_{\pm 0.009}$	$2.254_{\pm 0.071}$	$6.404_{\pm 0.135}$	$\mathbf{0.066}_{\pm 0.060}$	$\mathbf{0.061}_{\pm 0.063}$	0.086
Heterogeneous	TabDDPM	$\mathbf{0.372}_{\pm 0.012}$	$\mathbf{0.024}_{\pm 0.002}$	$\mathbf{0.013}_{\pm 0.008}$	$\mathbf{0.780}_{\pm 0.035}$	$\mathbf{2.629}_{\pm 0.125}$	$0.125_{\pm 0.068}$	$0.124_{\pm 0.069}$	0.136

Table 1: Comparison of vanilla TabDDPM and causal-penalized TabDDPM across three simulation settings: linear constant treatment effect (

\tau=-2

), nonlinear constant treatment effect (

\tau=1

), and heterogeneous treatment effects (

\tau=1.55

IHDP dataset
	Distributional fidelity					Causal fidelity
Method	Num. WD $\downarrow$	Cat. JS $\downarrow$	T TVD $\downarrow$	Y $W_{1}$ $\downarrow$	Corr. $L_{2}$ $\downarrow$	Post. $W_{1}$ $\downarrow$	Mean Gap $\downarrow$
CTGAN	$0.092_{\scriptscriptstyle\pm 0.001}$	$0.023_{\scriptscriptstyle\pm 0.001}$	$0.015_{\scriptscriptstyle\pm 0.006}$	$0.931_{\scriptscriptstyle\pm 0.031}$	$3.056_{\scriptscriptstyle\pm 0.014}$	$3.928_{\scriptscriptstyle\pm 0.131}$	$3.928_{\scriptscriptstyle\pm 0.131}$
TVAE	$0.035_{\scriptscriptstyle\pm 0.001}$	$0.159_{\scriptscriptstyle\pm 0.001}$	$0.107_{\scriptscriptstyle\pm 0.005}$	$0.464_{\scriptscriptstyle\pm 0.029}$	$3.060_{\scriptscriptstyle\pm 0.101}$	$0.110_{\scriptscriptstyle\pm 0.065}$	$0.107_{\scriptscriptstyle\pm 0.069}$
TabSyn	$0.027_{\scriptscriptstyle\pm 0.001}$	$0.028_{\scriptscriptstyle\pm 0.002}$	$0.016_{\scriptscriptstyle\pm 0.007}$	$0.290_{\scriptscriptstyle\pm 0.027}$	$0.486_{\scriptscriptstyle\pm 0.025}$	${\color[rgb]{0,0.4453,0.6992}\mathbf{0.056}}_{\scriptscriptstyle\pm 0.022}$	${\color[rgb]{0,0.4453,0.6992}\mathbf{0.043}}_{\scriptscriptstyle\pm 0.030}$
TabDDPM	${\color[rgb]{0,0.4453,0.6992}\mathbf{0.010}}_{\scriptscriptstyle\pm 0.001}$	${\color[rgb]{0,0.4453,0.6992}\mathbf{0.011}}_{\scriptscriptstyle\pm 0.001}$	${\color[rgb]{0,0.4453,0.6992}\mathbf{0.013}}_{\scriptscriptstyle\pm 0.007}$	${\color[rgb]{0,0.4453,0.6992}\mathbf{0.118}}_{\scriptscriptstyle\pm 0.027}$	${\color[rgb]{0,0.4453,0.6992}\mathbf{0.430}}_{\scriptscriptstyle\pm 0.051}$	$0.133_{\scriptscriptstyle\pm 0.059}$	$0.131_{\scriptscriptstyle\pm 0.060}$

Table 2: Comparison of synthetic tabular data generators on the IHDP and IST datasets with respect to distributional and causal fidelity. The best and second-best results for each metric are highlighted in blue and orange , respectively.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Num. WD $\downarrow$	Cat. JS $\downarrow$	Corr. $L_{2}$ $\downarrow$	Y $W_{1}$ $\downarrow$	T TVD $\downarrow$
CTGAN	$0.067_{\scriptscriptstyle\pm 0.009}$	$0.043_{\scriptscriptstyle\pm 0.010}$	$2.030_{\scriptscriptstyle\pm 0.058}$	$0.824_{\scriptscriptstyle\pm 0.222}$	$0.043_{\scriptscriptstyle\pm 0.033}$
TVAE	$0.016_{\scriptscriptstyle\pm 0.001}$	$0.048_{\scriptscriptstyle\pm 0.007}$	$2.530_{\scriptscriptstyle\pm 0.176}$	$0.194_{\scriptscriptstyle\pm 0.094}$	$0.016_{\scriptscriptstyle\pm 0.013}$
TabDDPM	$\mathbf{0.007}_{\scriptscriptstyle\pm 0.001}$	$\mathbf{0.017}_{\scriptscriptstyle\pm 0.004}$	$\mathbf{1.083}_{\scriptscriptstyle\pm 0.221}$	$\mathbf{0.104}_{\scriptscriptstyle\pm 0.028}$	$\mathbf{0.009}_{\scriptscriptstyle\pm 0.007}$

Appendix

Table 3: Distributional fidelity of CTGAN, TVAE, and TabDDPM under Setting 1 with

\tau=1

. Each entry reports the mean and standard deviation across 100 synthetic datasets (10 fitted models

\times

10 datasets).

True $\tau$	CTGAN	TVAE	TabDDPM
$-2$	2.102	0.952	0.109
$1$	0.896	0.328	0.085
$5$	4.806	0.715	0.199

Appendix

Table 4: Absolute discrepancy between the synthetic- and reference-derived posterior mean ATEs under Setting 1.

Figure 3: Causal-fidelity discrepancy of vanilla TabDDPM across simulation regimes and dimensions. Each cell shows the mean absolute synthetic–reference ATE posterior mean gap, averaged over treatment-effect configurations.

Figure 4: Representative ATE posterior comparisons for vanilla TabDDPM. Red and blue denote reference and synthetic posteriors; dashed lines indicate the corresponding posterior means and the true ATE.

Setting	True ATE	Method	ATE mean $\pm$ SD	MAE $\downarrow$	RMSE $\downarrow$	Post. W 1 $\downarrow$
Linear	$-2$	TabDDPM	$-1.7099\pm 0.0855$	$0.2901$	$0.3018$	$0.3361$
Linear	$-2$	Causal-TabDDPM	$\mathbf{-2.1165}\pm 0.1388$	$\mathbf{0.1454}$	$\mathbf{0.1786}$	$\mathbf{0.1233}$
Linear	$1$	TabDDPM	$\mathbf{0.9729}\pm 0.0614$	$\mathbf{0.0499}$	$\mathbf{0.0657}$	$\mathbf{0.0532}$
Linear	$1$	Causal-TabDDPM	$0.9517\pm 0.0618$	$0.0609$	$0.0772$	$0.0546$
Sin mix	$1$	TabDDPM	$1.2005\pm 0.0738$	$0.2005$	$0.2130$	$0.2370$
Sin mix	$1$	Causal-TabDDPM	$\mathbf{0.9728}\pm 0.0680$	$\mathbf{0.0578}$	$\mathbf{0.0716}$	$\mathbf{0.0575}$

Appendix

Table 5: Causal fidelity comparison of vanilla TabDDPM and Causal-TabDDPM across multiple simulation settings and treatment-effect values. The better result within each setting is shown in bold.

Method	Dataset	Key parameters
CTGAN	IHDP	lr= $10^{-4}$ , batch=500, epochs=300
CTGAN	IST	lr= $10^{-4}$ , batch=200, epochs=300
TVAE	IHDP	dims=(128,128), batch=200, epochs=300
TVAE	IST	dims=(256,256), batch=500, epochs=300
TabSyn	IHDP	$\beta_{\max}=0.05$ , batch=4096
TabSyn	IST	$\beta_{\max}=0.01$ , batch=1024

Appendix

Table 6: Final generator configurations used in the real-application experiments. Only the main tuned parameters are shown.

Figure 5: Detailed comparison of vanilla TabDDPM and Causal-TabDDPM on IHDP. Panels (a)-(b) show feature-wise distributional discrepancies, panel (c) compares ATE posteriors, and panel (d) reports posterior Wasserstein distance and absolute posterior mean gap across synthetic datasets.

Figure 6: Detailed comparison of vanilla TabDDPM and Causal-TabDDPM on IST. Panels (a)-(b) show feature-wise distributional discrepancies, panel (c) compares ATE posteriors, and panel (d) reports posterior Wasserstein distance and absolute posterior mean gap across synthetic datasets.

Explore similar work

Generative Synthetic Data for Causal Inference: Pitfalls, Remedies, and Opportunities
Apr 26, 2026Yichen XuCausal Effect Estimation Synthetic Data Generation
University of California, Berkeley
Improving TabPFN's Synthetic Data Generation by Integrating Causal Structure
Mar 10, 2026Davide Tugnoli, Andrea De Lorenzo, Marco Virgolin +1Tabular Foundation Models Synthetic Data Generation
Department of Mathematics, Informatics and Geosciences, University of Trieste, Trieste, Italy · InSilicoTrials Technologies, Riva Grumula 2, 34123 Trieste, Italy · Department of Engineering and Architecture, University of Trieste, Trieste, Italy +3
Causal-Aware Tabular GANs with Reinforcement Learning
Oct 28, 2025Tu Anh Hoang Nguyen, Dang Nguyen, Tri-Nhan Vo +3Synthetic Data Generation Synthetic Tabular Data Generation
Applied Artificial Intelligence Initiative (A2I2), Deakin University, Australia · School of Computer Science and Information Technology, Adelaide University, Australia

Apr 26, 2026, stat.MEJ/K move · Enter open · S save

Generative Synthetic Data for Causal Inference: Pitfalls, Remedies, and Opportunities

Yichen Xu

University of California, Berkeley

Causal Effect Estimation Synthetic Data Generation Synthetic Tabular Data Generation Synthetic Data Evaluation Causal Intervention

Open paper AskPDF arXiv

Synthetic tabular data are often evaluated by distributional similarity, privacy distance, or train-on-synthetic-test-on-real predictive performance, but these criteria do not ensure validity for causal inference. We show that fully generative tabular synthesizers, including GAN- and LLM-based models, can preserve predictive utility while distorting average treatment effect (ATE) estimates. The failure is structural: ATE preservation requires both a realistic covariate law and an accurate treatment-effect contrast, whereas prediction loss penalizes treatment-effect error only through an overlap-weighted term. Thus, under imbalance or limited overlap, a generator may reproduce dominant observed outcomes while underlearning intervention-relevant contrasts. We formalize this mismatch through sensitivity and loss-decomposition results. Motivated by this causal analysis and intuition, we propose a hybrid synthetic-data framework for causal inference that generates covariates while modeling treatment and outcome mechanisms separately. We evaluate the framework in three settings: ATE preservation under fully generative versus hybrid synthesis, augmentation for practical positivity problems, and diagnostic simulation engines for comparing OR, IPW, AIPW, and TMLE before real-data analysis. We also stress-test the hybrid construction across settings that vary overlap, covariate dimension, seed sample size, and treatment-effect complexity, including a logistic outcome-model misspecification check. Across controlled simulation experiments, hybrid synthesis improves causal fidelity relative to fully generative baselines; the ACTG application shows improved predictive fidelity and potential for finite-sample estimator benchmarking. LLM-based hybrid synthesis is often more faithful than CTGAN in settings where causal fidelity can be assessed.

Abstract

Explore similar work

Generative Synthetic Data for Causal Inference: Pitfalls, Remedies, and Opportunities

Improving TabPFN's Synthetic Data Generation by Integrating Causal Structure

Causal-Aware Tabular GANs with Reinforcement Learning