Statistical Inference via Generative Models: Flow Matching and Causal Inference

Shinto Eguchi

stat.MLMar 9, 2026

Statistical Inference via Generative Models: Flow Matching and Causal Inference

Authors: Shinto Eguchi

Abstract

Generative AI has achieved remarkable empirical success, but from the perspective of statistics it often remains opaque: its predictions may be accurate, yet the underlying mechanism is difficult to interpret, analyze, and trust. This book reinterprets generative AI in the language of statistics, using flow matching as a central example. The key idea is that generative models should be understood not merely as devices for producing plausible data, but as methods for the nonparametric learning of high-dimensional probability distributions. From this viewpoint, missing-data imputation becomes principled sampling from learned conditional distributions, counterfactual analysis becomes the estimation of intervention distributions, and distributional dynamics become statistically analyzable objects. Mathematically, flow matching represents distributional deformation through the continuity equation and a time-dependent velocity field, thereby extending score matching from the learning of static score fields to the learning of transport paths themselves. Building on this foundation, the book develops a statistical framework in which generative models are used to estimate nuisance components while inferential validity is maintained through orthogonalization and cross-fitting in the spirit of double/debiased machine learning. Applications to survival analysis, censoring, missingness, and causal inference show how generative models can be integrated into statistical inference for structured high-dimensional problems.

Figures & tables

Figure 2.1 : Quartic potential model $p_{\theta}(x)$ with $\theta=(0,2,-0.5)$ .

Figure 2.2: Vector field induced by a James–Stein-type shrinkage term.

Panel of Figure 3 — Figure 3.1: Pairings induced by different couplings (illustration).

Figure 4.1: Stabilization and robustness induced by Lipschitz constraints on the velocity field. The color intensity visualizes the sensitivity ratio ${\|\Delta X_{1}\|}/{\|\Delta X_{0}\|}$ of the final state to a small perturbation in the initial state; red indicates instability. (Left) standard training. (Right) training with a Lipschitz constraint via spectral normalization.

Figure 4.2: Neyman orthogonality: parameter score vs nuisance score directions.

\widehat{\Psi}_{n}(\theta)=\frac{1}{n}\sum_{k=1}^{K}\sum_{i\in I_{k}}\Psi\bigl(O_{i};\theta,\widehat{\eta}^{(-k)}\bigr).

Algorithm 2 Cross-fitted DDML with an orthogonal estimating equation

Figure 5.1: Flow-copula learning by FM. Left: target samples exhibiting an S-shaped dependence. Right: samples from the learned model; KDE-based copula-density contours are shown as filled levels, overlaid with the learned vector field (velocity at $t=0.5$ , arrows).

dataset	$t_{0}$	PH $p$	tv	mean $\|\Delta R\|$	Br.(Cox)	Br.(Flow)
lung	210.0	0.545	ph.ecog	0.00348	0.21587	0.21578
pbc	1180.5	0.118	bili	0.00341	0.06763	0.06828
veteran	93.5	2.17e-4	karno	0.01807	0.16758	0.16476

Table 5.1: PH diagnostics and Cox vs Cox+TV summary (evaluation data).

Figure 5.2: Calibration curves for a PH-good case ( lung ) and a PH-bad case ( veteran ). The horizontal axis is predicted risk at horizon $t_{0}$ ; the vertical axis is observed risk estimated by Kaplan–Meier within bins defined by Cox-predicted risk. Each label indicates the bin sample size $n$ . When PH holds, Cox and Cox+TV nearly coincide; when PH is violated, Cox+TV moves toward the ideal line.

Metric	MICE	FM
Missingness rate $P(M_{3}=1)$	$0.359$
RMSE (mean imputation)	$0.5387$	$0.4955$
$W_{1}$ (distribution distance)	$0.2628$	$0.0553$
Brier ( $\mathbb{I}\{X_{3}>0\}$ )	$0.0674$	$0.0624$
$\widehat{\beta}_{0}$ (SE)	$-0.0750\ (0.0200)$	$-0.0685\ (0.0192)$
$\widehat{\beta}_{1}$ (SE)	$0.4600\ (0.0242)$	$0.4452\ (0.0237)$

Table 5.2: Reconstruction of

X_{3}

for missing rows and regression inference (Rubin combination). Since

X

is standardized before analysis, the “true” coefficients refer to standardized values.

Panel of Figure 5 — Figure 5.3: Distribution of $X_{3}$ among missing rows ( $M_{3}=1$ ). Top: true $X_{3}$ (bimodal). Middle: pooled MI distribution from MICE (collapses toward unimodal). Bottom: pooled MI distribution from FM-based imputations (bimodality preserved).

Figure 6.1: A basic causal DAG example. (A) Conditioning on observed confounders $X$ closes the backdoor path, and the interventional distribution is identified by the $g$ -formula. (B) If an unobserved confounder $U$ is a common cause of $A$ and $Y$ , conditioning on $X$ does not close the backdoor path, and $p(y\,|\,\mathrm{do}(A=a))$ is not uniquely identified from the observational distribution alone.

Figure 6.2: QQ plots for interventional distributions (truth vs estimate). Left: do $(A=0)$ ; right: do $(A=1)$ . RF (regression + pooled standardized residuals) underestimates the upper tail, with a larger discrepancy for do $(1)$ . FM (conditional sampler) reproduces the distributional shape including the tail more faithfully.

Y_{\mathrm{rf}}^{(a,\ell)}\leftarrow\widehat{\mu}_{a}(X^{(\ell)})+\widehat{s}_{a}(X^{(\ell)})\,\widehat{\varepsilon}^{(\ell)}.

Algorithm 3 Estimating interventional distributions (simple comparison: RF vs FM)

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

	RMSE_MLE	CT_MLE (sec)	RMSE_SM	CT_SM (sec)
mean	0.182256	1.595136	0.051682	0.127432
std	0.001820	1.511921	0.001156	0.016656

Appendix

Table A.1: (Example) Monte Carlo mean and standard deviation of estimation accuracy (RMSE) and computation time (CT).

d=200,n=120

$\mathbb{R}^{d}$	The $d$ -dimensional Euclidean space.
$P,Q$	Probability measures.
$p(x),q(x),\rho(x)$	Probability density functions.
$X\sim P$	A random variable $X$ follows distribution $P$ .
$\mathbb{E}[f(X)]$	Expectation. When the underlying distribution $P$ is explicit, we write $\mathbb{E}_{X\sim P}[f(X)]$ or $\mathbb{E}_{P}[f]$ .
$\widehat{P}_{n}$	Empirical measure: $\frac{1}{n}\sum_{i=1}^{n}\delta_{X_{i}}$ .
$\mathcal{N}(\mu,\Sigma)$	Gaussian distribution with mean $\mu$ and covariance matrix $\Sigma$ .

Appendix

Table 16

$\nabla_{x}f(x)$	Gradient of $f$ with respect to $x$ .
$\nabla\cdot v(x)$	Divergence of a vector field $v$ : $\sum_{j=1}^{d}\partial_{x_{j}}v_{j}(x)$ .
$\Delta f(x)$	Laplacian: $\nabla\cdot\nabla f(x)$ .
$v_{t}(x)$	Vector field (velocity field) at time $t$ ; also written as $v(t,x)$ .
$s_{p}(x)$	Score function of density $p$ : $\nabla_{x}\log p(x)$ .
$T_{\#}P$	Pushforward of measure $P$ by a map $T$ . If $X\sim P$ , then $T(X)\sim T_{\#}P$ .
$\Phi_{t}(x)$	Flow map of an ODE: the position at time $t$ of a particle starting at $x$ at time $0$ .

Appendix

Table 17

$t$	Pseudo-time or interpolation parameter, typically $t\in[0,1]$ .
$\pi(x)$	Reference distribution, typically a known distribution such as the standard Gaussian.
$\rho_{t}(x)$	Density of the intermediate distribution (probability path) at time $t$ .
$u_{t}(x\mid x_{1})$	Conditional vector field (target velocity) with endpoint $x_{1}$ fixed.
$X_{0},X_{1}$	Random variables representing the start point (noise) and the end point (data), respectively.

Appendix

Table 18

$A$	Treatment (intervention) variable, typically $A\in\{0,1\}$ .
$Y$	Observed outcome variable.
$Y(a)$	Potential outcome under treatment $A=a$ .
$\mathrm{do}(A=a)$	Intervention operator that sets $A$ to $a$ in a structural model.
$\tau(x)$	Conditional average treatment effect (CATE): $\mathbb{E}[Y(1)-Y(0)\mid X=x]$ .
$e(x)$	Propensity score: $P(A=1\mid X=x)$ .
$\mu_{a}(x)$	Outcome regression: $\mathbb{E}[Y\mid A=a,X=x]$ .

Appendix

Table 19

Causal Inference

Causal inference studies how interventions and counterfactuals reveal cause-and-effect relationships beyond observed associations.

See all 316 papers

Harnessing Synthetic Data from Generative AI for Statistical Inference
Mar 5, 2026Ahmad Abdel-Azim, Ruoyu Wang, Xihong LinSynthetic Data Generation Synthetic Data Evaluation
Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts 02115, USA · Harvard University, Boston, Massachusetts 02115, USA
Generative Augmented Inference
Apr 16, 2026Cheng Lu, Mengxin Wang, Dennis J. Zhang +1LLM-Assisted Annotation Causal Inference
Olin Business School, Washington University in St. Louis, USA · Naveen Jindal School of Management, University of Texas at Dallas, USA · W. P. Carey School of Business, Arizona State University, USA
Debiased Counterfactual Generation via Flow Matching from Observations
May 8, 2026Hugh Dance, Johnny Xi, Peter Orbanz +1Flow Matching Causal Counterfactual Generation
Gatsby Computational Neuroscience Unit University College London · Department of Statistics University of British Columbia

Mar 5, 2026J/K move · Enter open · S save

Harnessing Synthetic Data from Generative AI for Statistical Inference

Ahmad Abdel-Azim, Ruoyu Wang, Xihong Lin

Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts 02115, USA · Harvard University, Boston, Massachusetts 02115, USA

Synthetic Data Generation Synthetic Data Evaluation Generative Modeling

Open paper AskPDF arXiv

The emergence of generative AI models has dramatically expanded the availability and use of synthetic data across scientific, industrial, and policy domains. While these developments open new possibilities for data analysis, they also raise fundamental statistical questions about when synthetic data can be used in a valid, reliable, and principled manner. This paper reviews the current landscape of synthetic data generation and use from a statistical perspective, with the goal of clarifying the assumptions under which synthetic data can meaningfully support downstream discovery, inference, and prediction. We survey major classes of modern generative models, their intended use cases, and the benefits they offer, while also highlighting their limitations and characteristic failure modes. We additionally examine common pitfalls that arise when synthetic data are treated as surrogates for real observations, including biases from model misspecification, attenuated uncertainty, and difficulties in generalization. Building on these insights, we discuss emerging frameworks for the principled use of synthetic data. We conclude with practical recommendations, open problems, and cautions intended to guide both method developers and applied researchers.

Abstract

Causal Inference

Explore similar work

Harnessing Synthetic Data from Generative AI for Statistical Inference

Generative Augmented Inference

Debiased Counterfactual Generation via Flow Matching from Observations