Organizations: Department of Applied Mathematics and Statistics, Johns Hopkins University. · Amazon · Department of Statistics, University of California, Davis.
Counterfactual generation seeks to sample outcomes under a hypothetical intervention or decision using observational data collected under the factual assignment mechanism. We develop a flow-matching approach that combines a sample-split, doubly robust training objective with a learned coupling between observed source outcomes and target outcomes drawn from a fitted conditional outcome model. To enable finite-step generation, we leverage a score-corrected stochastic sampler based on a Gaussian-smoothed interpolation. Our main theoretical contribution is a coupling-sensitive KL bound for constant-step Euler discretization: the error is controlled by moments of the source--target displacement under the chosen coupling, rather than by global uniform regularity of the velocity field, and has near-linear dependence on the ambient dimension. We also establish finite-sample non-parametric guarantees for the learned velocity and score fields when both the conditional outcome model and the source-target coupling are estimated from data. These bounds separate approximation, coupling-replacement, nuisance-estimation, generalization, and Monte Carlo errors and, combined with the sampler analysis, yield an end-to-end guarantee for counterfactual generation. Experiments on synthetic and semi-synthetic image benchmarks support the coupling-dependent theory and show that, at finite discretization budgets, the stochastic sampler can outperform the corresponding deterministic ODE sampler.
Figures & tables
Figure 1: Angular-mode experiment. (a)–(b) Velocity and score errors across couplings ( σ=0.12 ). (c) Exact-field 8 -step KL versus D5 , using 2,000 samples and τ=0.5σD5/d . (d) Mode-mass allocation for 4 and 64 steps (EOT εOT=0.5 , σ=0.02 ): 4 -step ODE/SDE misallocate mass, while 64 -step ODE/SDE match the target modes.
Figure 2: ColorMNIST 1–6. (a) Clean-target SW2 for σ∈{0.01,0.015} ( 5,000 samples, five seeds; dashed: pure ODE). (b)–(d) Color evolution along the σ=0.01 SDE at t=0,0.5,1 ( 5,000 samples; dashed: clean target).
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Fashion-MNIST shirts experiment. (a) Clean-target SW2 as a function of τ for the two smoothing levels; shaded regions show variation across five evaluation seeds, and the dashed line denotes the pure ODE baseline. (b)–(d) Color distributions at t=0,0.5,1 along the σ=0.005 SDE trajectory; dashed distributions denote the clean target.
Figure 5: Qualitative examples from the observed source distribution (top), samples generated by the σ=0.005 SDE (bottom). The generated samples preserve the boots shape and structure while showing clear, coherent color changes.
Figure 6: Qualitative examples from the observed source distribution (top), samples generated by the σ=0.005 SDE (bottom). The generated samples preserve the shirt shape and structure while showing clear, coherent color changes.
Work
Setting / Method
Metric
Dim.
Assumption
Coupling control
Stochastic interpolants and flow matching
This work Main theorem
constant- smoothing SI SDE, Euler
KL
O(d)
L2 error
D5
( Liu et al., 2025 ) Thm. 4.3; Sec. 5
SI SDE, Euler
KL
O(d2)
L2 error
Partial
( Liu et al., 2026 ) Thm. 4.5
SI ODE, Euler
TV
O(d2)
drift/div. errors; unif. reg.
No
Diffusion-model baselines
( Benton et al., 2024 ) Cor. 1
diffusion exp. integrator
KL
O(d)
L2 error
–
Appendix
Table 1: Comparison with related finite-step sampling results. Dimension dependence suppresses logarithmic factors and holds accuracy and the other problem parameters fixed; for our result this includes σ,D5 , with τ2=σD5/d and controlled field error.“Partial” means that source–target displacement enters the bound, but coupling-independent score terms remain.
Notation
Meaning
Notation
Meaning
(C,A,Y)
Observed covariate, treatment, and outcome.
Vn,Sn
Clipped sparse-ReLU velocity class and score class induced by clipped sparse-ReLU denoisers.
a
Fixed target treatment value.
R,ε
Network input-clipping radius and approximation tolerance; field-specific choices appear in Proposition 1 .
PY∣C=c,A=a
Conditional outcome law at covariate value c under treatment a .
εvel,εscore
Integrated estimation errors of the learned velocity and score fields.
μ1
Target law ∫PY∣C=c,A=adPC(c) .
τ
Noise level of the score-corrected SDE sampler.
PY∣A=a
Observed outcome law among samples with treatment a .
N,h
Number of sampling steps and Euler step size, with h=1/N .
Dtr,Daux
Training and auxiliary splits, each containing n observations.
tk
Sampling grid point tk=kh .
Appendix
Table 2: Summary of notation used throughout the paper.
Estimating counterfactual distributions under interventions is central to treatment risk assessment and counterfactual generation tasks. Existing approaches model the counterfactual distribution as a standalone generative target, without exploiting its relationship to the observational data. In this work, we show that under standard assumptions, observational and counterfactual outcome distributions are tightly linked: they have identical support and tail behavior, remain statistically close under weak confounding, and share any features of high-dimensional outcomes which are invariant to confounders. These properties motivate learning counterfactual distributions not from scratch, but via a deconfounding flow from the observational distribution. We formulate this problem via flow-matching and derive a semiparametrically efficient estimator based on a novel efficient influence function correction. We subsequently extend our estimator to target minimal-energy flows in high-dimensions, which we show can be especially simple targets between observational and counterfactual distributions. In experiments, deconfounding flows outperform existing debiased counterfactual distribution estimators, while also mitigating known failure modes of flow-based methods.
Hugh Dance, Johnny Xi, Peter Orbanz +1
Gatsby Computational Neuroscience Unit University College London · Department of Statistics University of British Columbia
Generative models for counterfactual outcomes have great potential to support decision-making under complex interventions, but existing approaches are limited by unstable estimation, poor generalization across environments, and bias from nuisance model misspecification. We introduce ADIGen, a framework for automatic, debiased, and invariant counterfactual generation under general interventions, including high-dimensional interventions and outcomes. ADIGen combines Riesz regression to avoid unstable density-ratio estimation, causal invariance to improve generalization under distribution shift, and orthogonal statistical learning to obtain doubly robust guarantees against nuisance model misspecification. We provide excess-risk bounds showing that ADIGen controls counterfactual risk under general interventions, with a product-bias nuisance remainder and an invariant risk bound across environments.
Raphael C Kim, Jingsen Zhu, Ramin Zabih +1
Cornell Tech, Cornell University, New York, NY · Department of Biostatistics, Department of Population Health, New York University Grossman School of Medicine, New York, NY
Flow matching models generate samples by numerically integrating a learned velocity field, with each integration step requiring a neural network evaluation. Fast generation therefore requires using a small fixed evaluation budget effectively: the key question is not only how to integrate the flow, but where the sampler should spend its steps. We propose SharpEuler, a training-free sampler that profiles a pretrained model offline by estimating where the learned velocity field changes most rapidly along calibration trajectories. This finite-difference estimate defines a solver-aware sharpness profile, which is smoothed and converted by a quantile transform into a timestep grid for any desired inference budget. At test time, sampling remains ordinary Euler integration with the same number of model evaluations as a uniform schedule. We justify SharpEuler using three principles: a numerical principle identifying trajectory acceleration as the leading source of Euler discretization error, a variational principle deriving sharpness-based power-law timestep densities, and a statistical guarantee showing that the finite-sample calibrated sampler is stable at the terminal distribution level. Our experiments show that SharpEuler improves sample quality at fixed budgets, reducing inter-mode leakage and increasing mode coverage.
Aditi Gupta, Soon Hoe Lim, Annan Yu +1
Lawrence Berkeley National Laboratory · International Computer Science Institute · 1Lawrence Berkeley National Laboratory, 2International Computer Science Institute +4