Statistical Inference via Generative Models: Flow Matching and Causal Inference

Shinto Eguchi

stat.MLMar 9, 2026

Statistical Inference via Generative Models: Flow Matching and Causal Inference

Authors: Shinto Eguchi

Abstract

Generative AI has achieved remarkable empirical success, but from the perspective of statistics it often remains opaque: its predictions may be accurate, yet the underlying mechanism is difficult to interpret, analyze, and trust. This book reinterprets generative AI in the language of statistics, using flow matching as a central example. The key idea is that generative models should be understood not merely as devices for producing plausible data, but as methods for the nonparametric learning of high-dimensional probability distributions. From this viewpoint, missing-data imputation becomes principled sampling from learned conditional distributions, counterfactual analysis becomes the estimation of intervention distributions, and distributional dynamics become statistically analyzable objects. Mathematically, flow matching represents distributional deformation through the continuity equation and a time-dependent velocity field, thereby extending score matching from the learning of static score fields to the learning of transport paths themselves. Building on this foundation, the book develops a statistical framework in which generative models are used to estimate nuisance components while inferential validity is maintained through orthogonalization and cross-fitting in the spirit of double/debiased machine learning. Applications to survival analysis, censoring, missingness, and causal inference show how generative models can be integrated into statistical inference for structured high-dimensional problems.

Figures & tables

Figure 2.1 : Quartic potential model $p_{\theta}(x)$ with $\theta=(0,2,-0.5)$ .

Figure 2.2: Vector field induced by a James–Stein-type shrinkage term.

Panel of Figure 3 — Figure 3.1: Pairings induced by different couplings (illustration).

Figure 4.1: Stabilization and robustness induced by Lipschitz constraints on the velocity field. The color intensity visualizes the sensitivity ratio ${\|\Delta X_{1}\|}/{\|\Delta X_{0}\|}$ of the final state to a small perturbation in the initial state; red indicates instability. (Left) standard training. (Right) training with a Lipschitz constraint via spectral normalization.

Figure 4.2: Neyman orthogonality: parameter score vs nuisance score directions.

\widehat{\Psi}_{n}(\theta)=\frac{1}{n}\sum_{k=1}^{K}\sum_{i\in I_{k}}\Psi\bigl(O_{i};\theta,\widehat{\eta}^{(-k)}\bigr).

Algorithm 2 Cross-fitted DDML with an orthogonal estimating equation

Figure 5.1: Flow-copula learning by FM. Left: target samples exhibiting an S-shaped dependence. Right: samples from the learned model; KDE-based copula-density contours are shown as filled levels, overlaid with the learned vector field (velocity at $t=0.5$ , arrows).

dataset	$t_{0}$	PH $p$	tv	mean $\|\Delta R\|$	Br.(Cox)	Br.(Flow)
lung	210.0	0.545	ph.ecog	0.00348	0.21587	0.21578
pbc	1180.5	0.118	bili	0.00341	0.06763	0.06828
veteran	93.5	2.17e-4	karno	0.01807	0.16758	0.16476

Table 5.1: PH diagnostics and Cox vs Cox+TV summary (evaluation data).

Figure 5.2: Calibration curves for a PH-good case ( lung ) and a PH-bad case ( veteran ). The horizontal axis is predicted risk at horizon $t_{0}$ ; the vertical axis is observed risk estimated by Kaplan–Meier within bins defined by Cox-predicted risk. Each label indicates the bin sample size $n$ . When PH holds, Cox and Cox+TV nearly coincide; when PH is violated, Cox+TV moves toward the ideal line.

Metric	MICE	FM
Missingness rate $P(M_{3}=1)$	$0.359$
RMSE (mean imputation)	$0.5387$	$0.4955$
$W_{1}$ (distribution distance)	$0.2628$	$0.0553$
Brier ( $\mathbb{I}\{X_{3}>0\}$ )	$0.0674$	$0.0624$
$\widehat{\beta}_{0}$ (SE)	$-0.0750\ (0.0200)$	$-0.0685\ (0.0192)$
$\widehat{\beta}_{1}$ (SE)	$0.4600\ (0.0242)$	$0.4452\ (0.0237)$

Table 5.2: Reconstruction of

X_{3}

for missing rows and regression inference (Rubin combination). Since

X

is standardized before analysis, the “true” coefficients refer to standardized values.

Panel of Figure 5 — Figure 5.3: Distribution of $X_{3}$ among missing rows ( $M_{3}=1$ ). Top: true $X_{3}$ (bimodal). Middle: pooled MI distribution from MICE (collapses toward unimodal). Bottom: pooled MI distribution from FM-based imputations (bimodality preserved).

Figure 6.1: A basic causal DAG example. (A) Conditioning on observed confounders $X$ closes the backdoor path, and the interventional distribution is identified by the $g$ -formula. (B) If an unobserved confounder $U$ is a common cause of $A$ and $Y$ , conditioning on $X$ does not close the backdoor path, and $p(y\,|\,\mathrm{do}(A=a))$ is not uniquely identified from the observational distribution alone.

Figure 6.2: QQ plots for interventional distributions (truth vs estimate). Left: do $(A=0)$ ; right: do $(A=1)$ . RF (regression + pooled standardized residuals) underestimates the upper tail, with a larger discrepancy for do $(1)$ . FM (conditional sampler) reproduces the distributional shape including the tail more faithfully.

Y_{\mathrm{rf}}^{(a,\ell)}\leftarrow\widehat{\mu}_{a}(X^{(\ell)})+\widehat{s}_{a}(X^{(\ell)})\,\widehat{\varepsilon}^{(\ell)}.

Algorithm 3 Estimating interventional distributions (simple comparison: RF vs FM)

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

	RMSE_MLE	CT_MLE (sec)	RMSE_SM	CT_SM (sec)
mean	0.182256	1.595136	0.051682	0.127432
std	0.001820	1.511921	0.001156	0.016656

Appendix

Table A.1: (Example) Monte Carlo mean and standard deviation of estimation accuracy (RMSE) and computation time (CT).

d=200,n=120

$\mathbb{R}^{d}$	The $d$ -dimensional Euclidean space.
$P,Q$	Probability measures.
$p(x),q(x),\rho(x)$	Probability density functions.
$X\sim P$	A random variable $X$ follows distribution $P$ .
$\mathbb{E}[f(X)]$	Expectation. When the underlying distribution $P$ is explicit, we write $\mathbb{E}_{X\sim P}[f(X)]$ or $\mathbb{E}_{P}[f]$ .
$\widehat{P}_{n}$	Empirical measure: $\frac{1}{n}\sum_{i=1}^{n}\delta_{X_{i}}$ .
$\mathcal{N}(\mu,\Sigma)$	Gaussian distribution with mean $\mu$ and covariance matrix $\Sigma$ .

Appendix

Table 16

$\nabla_{x}f(x)$	Gradient of $f$ with respect to $x$ .
$\nabla\cdot v(x)$	Divergence of a vector field $v$ : $\sum_{j=1}^{d}\partial_{x_{j}}v_{j}(x)$ .
$\Delta f(x)$	Laplacian: $\nabla\cdot\nabla f(x)$ .
$v_{t}(x)$	Vector field (velocity field) at time $t$ ; also written as $v(t,x)$ .
$s_{p}(x)$	Score function of density $p$ : $\nabla_{x}\log p(x)$ .
$T_{\#}P$	Pushforward of measure $P$ by a map $T$ . If $X\sim P$ , then $T(X)\sim T_{\#}P$ .
$\Phi_{t}(x)$	Flow map of an ODE: the position at time $t$ of a particle starting at $x$ at time $0$ .

Appendix

Table 17

$t$	Pseudo-time or interpolation parameter, typically $t\in[0,1]$ .
$\pi(x)$	Reference distribution, typically a known distribution such as the standard Gaussian.
$\rho_{t}(x)$	Density of the intermediate distribution (probability path) at time $t$ .
$u_{t}(x\mid x_{1})$	Conditional vector field (target velocity) with endpoint $x_{1}$ fixed.
$X_{0},X_{1}$	Random variables representing the start point (noise) and the end point (data), respectively.

Appendix

Table 18

$A$	Treatment (intervention) variable, typically $A\in\{0,1\}$ .
$Y$	Observed outcome variable.
$Y(a)$	Potential outcome under treatment $A=a$ .
$\mathrm{do}(A=a)$	Intervention operator that sets $A$ to $a$ in a structural model.
$\tau(x)$	Conditional average treatment effect (CATE): $\mathbb{E}[Y(1)-Y(0)\mid X=x]$ .
$e(x)$	Propensity score: $P(A=1\mid X=x)$ .
$\mu_{a}(x)$	Outcome regression: $\mathbb{E}[Y\mid A=a,X=x]$ .

Appendix

Table 19

Causal Inference

Causal inference studies how interventions and counterfactuals reveal cause-and-effect relationships beyond observed associations.

See all 316 papers

Mar 5, 2026

Harnessing Synthetic Data from Generative AI for Statistical Inference

The emergence of generative AI models has dramatically expanded the availability and use of synthetic data across scientific, industrial, and policy domains. While these developments open new possibilities for data analysis, they also raise fundamental statistical questions about when synthetic data can be used in a valid, reliable, and principled manner. This paper reviews the current landscape of synthetic data generation and use from a statistical perspective, with the goal of clarifying the assumptions under which synthetic data can meaningfully support downstream discovery, inference, and prediction. We survey major classes of modern generative models, their intended use cases, and the benefits they offer, while also highlighting their limitations and characteristic failure modes. We additionally examine common pitfalls that arise when synthetic data are treated as surrogates for real observations, including biases from model misspecification, attenuated uncertainty, and difficulties in generalization. Building on these insights, we discuss emerging frameworks for the principled use of synthetic data. We conclude with practical recommendations, open problems, and cautions intended to guide both method developers and applied researchers.

Ahmad Abdel-Azim, Ruoyu Wang, Xihong Lin

Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts 02115, USA · Harvard University, Boston, Massachusetts 02115, USA

Synthetic Data Generation Synthetic Data Evaluation Generative Modeling

Ask PDF

Apr 16, 2026

Generative Augmented Inference

Large language models enable inexpensive AI-generated annotations, but using them reliably for causal inference remains challenging. Naively pooling AI and human data induces bias, while existing methods such as Prediction-Powered Inference (PPI; Angelopoulos et al., 2023a) treat AI outputs as proxies of true labels -- an assumption often violated for generative model outputs in practice. We propose Generative Augmented Inference (GAI), a framework that treats AI outputs as general, potentially high-dimensional informative features for learning human labels rather than as surrogates. GAI flexibly models this relationship using nonparametric methods, enabling consistent estimation and valid inference from combined human and AI data. We establish asymptotic normality and show that, under random labeling, GAI strictly improves asymptotic efficiency over human-data-only estimation whenever AI outputs are informative for true labels. Empirical studies on real-world datasets demonstrate that GAI significantly reduces estimation error and improves confidence interval quality across diverse generative data sources relative to human-only and PPI-based estimation.

Cheng Lu, Mengxin Wang, Dennis J. Zhang +1

Olin Business School, Washington University in St. Louis, USA · Naveen Jindal School of Management, University of Texas at Dallas, USA · W. P. Carey School of Business, Arizona State University, USA

LLM-Assisted Annotation Causal Inference

Ask PDF

May 8, 2026

Debiased Counterfactual Generation via Flow Matching from Observations

Estimating counterfactual distributions under interventions is central to treatment risk assessment and counterfactual generation tasks. Existing approaches model the counterfactual distribution as a standalone generative target, without exploiting its relationship to the observational data. In this work, we show that under standard assumptions, observational and counterfactual outcome distributions are tightly linked: they have identical support and tail behavior, remain statistically close under weak confounding, and share any features of high-dimensional outcomes which are invariant to confounders. These properties motivate learning counterfactual distributions not from scratch, but via a deconfounding flow from the observational distribution. We formulate this problem via flow-matching and derive a semiparametrically efficient estimator based on a novel efficient influence function correction. We subsequently extend our estimator to target minimal-energy flows in high-dimensions, which we show can be especially simple targets between observational and counterfactual distributions. In experiments, deconfounding flows outperform existing debiased counterfactual distribution estimators, while also mitigating known failure modes of flow-based methods.

Hugh Dance, Johnny Xi, Peter Orbanz +1

Gatsby Computational Neuroscience Unit University College London · Department of Statistics University of British Columbia

Flow Matching Causal Counterfactual Generation Causal Inference Counterfactual Learning

Ask PDF

Abstract

Causal Inference

Explore similar work

Harnessing Synthetic Data from Generative AI for Statistical Inference

Generative Augmented Inference

Debiased Counterfactual Generation via Flow Matching from Observations