Two-Sample Testing via Generative Processes

Eshant English; Kenji Fukumizu; Taiji Suzuki

stat.MLOct 6, 2026

Two-Sample Testing via Generative Processes

Authors: Eshant English, Kenji Fukumizu, Taiji Suzuki

Organizations: The University of Tokyo, Tokyo, Japan · Hasso Plattner Institute for Digital Engineering, Germany · Institute of Statistical Mathematics, Japan · RIKEN Centre for Advanced Intelligence Project (AIP), Tokyo, Japan

Abstract

Deciding whether two samples come from the same distribution is a classical problem in statistics, and generative transport offers a new way to approach it. We build a stochastic interpolant directly between the two samples and observe that, for a symmetric schedule, its law is invariant under the time reflection $t \mapsto 1-t$ whenever the two distributions coincide. We therefore test whether the marginals at times t and 1-t agree by computing their Jensen--Shannon divergence. Both marginals are explicit mixtures over all cross-pairs of observations, so nothing is learned, and permutation calibration gives an exact finite-sample level. For Gaussian noise, this divergence equals a time integral that pairs the reflection defects of the velocity field and of the score, so the test compares transport dynamics rather than endpoints alone. With a narrow-plus-broad noise design, the test attains the minimax separation rate n^{-2s/(4s+d)} over bounded, compactly supported densities whose difference has Sobolev smoothness s > 3d/4, with no lower bound on the densities. Fusing a dyadic grid of noise scales through their permutation ranks, without sample splitting, preserves exact level and adapts to unknown s at an iterated-logarithmic cost. Empirically, the test matches or outperforms state-of-the-art kernel two-sample tests.

Figures & tables

Figure 1: Time reflection of a stochastic interpolant built directly between $\rho_{0}$ and $\rho_{1}$ using our construction with the linear schedule $x_{t}=(1-t)\,x_{0}+t\,x_{1}+\sigma t(1-t)\,z$ , $\sigma=1.5$ and Gaussian $z$ . Slices show the exact marginals $\rho_{t}$ . Solid curves are probability-flow paths from six starting points in $\rho_{0}$ , coloured by their starting mode; dashed curves mirror the first half of each path under $t\mapsto 1-t$ , and dotted contours on the slices $t=3/4$ and $t=1$ outline the mirror frames $\rho_{1/4}$ and $\rho_{0}$ . Top: under $\mathcal{H}_{0}$ , $\rho_{t}=\rho_{1-t}$ ( Proposition 3.2 ), each mirrored distribution coincides with the actual one. Bottom: under $\mathcal{H}_{1}$ , the mirrored paths end in the outline of $\rho_{0}$ while the actual paths end in $\rho_{1}$ . All densities and velocity fields are exact Gaussian-mixture expressions; nothing is learned.

Figure 2: Power (rejection rate under the alternative) over 1000 repetitions at level

\kappa=0.05

. (a) Blob, as a function of the sample size

n

per distribution. (b) MNIST with

n=200

, under alternatives

Q_{1}

–

Q_{5}

of increasing difficulty. Shaded regions are 95% Wilson confidence intervals for Ours, MMDAgg and MMD-Fuse. Solid and dashed lines denote Gaussian and Laplace kernels, respectively, for the single-kernel MMD tests. C2ST-RF (random-forest classifier two-sample test) and AutoTST (non-neural tabular models) train on half of the sample and test on the other half. † Oracle methods use

n

additional samples for kernel selection.

Figure 3: Type I error (rejection rate under the null) over 1000 repetitions at level

\kappa=0.05

. (a) Blob, as a function of the sample size

n

per distribution. (b) MNIST with

n=200

. Error bars are 95% Wilson confidence intervals. The dotted line marks

\kappa

, and the grey band is the range of rejection rates consistent with an exact level-

\kappa

test over 1000 repetitions (95% Monte Carlo interval). Filled and hollow markers denote Gaussian and Laplace kernels, respectively, for the single-kernel MMD tests (and C2ST-RF and AutoTST, respectively, for the classifier-based tests). † Oracle methods use

n

additional samples for kernel selection.

	Type I error (Blob-S)				Power (Blob-D)
Method	$n=90$	$n=180$	$n=270$	$n=360$	$n=90$	$n=180$	$n=270$	$n=360$
Ours	0.036 ∗	0.043	0.060	0.053	0.141	0.526	0.934	0.999
MMDAgg	0.047	0.041	0.053	0.057	0.123	0.405	0.767	0.935
MMD-Fuse	0.050	0.039	0.050	0.061	0.149	0.403	0.763	0.931
C2ST-RF ‡	0.038	0.038	0.038	0.042	0.058	0.181	0.343	0.559
AutoTST ‡	0.041	0.058	0.056	0.059	0.054	0.136	0.315	0.492

Table 1: Blob experiment: full results. Rejection rates over 1000 repetitions at

\kappa=0.05

. The 95% Wilson interval half-width is at most

\pm 0.014

near

0.05

and at most

\pm 0.031

elsewhere. Bold: highest power among methods that do not use extra data. ∗ : 95% interval excludes

\kappa

(above: liberal; below: conservative). † Oracle methods use

n

additional samples for kernel selection. ‡ Classifier-based tests (random-forest C2ST; AutoTST with non-neural tabular models) train on half of the sample and test on the other half.

	Type I error	Power
Method	Null	$Q_{1}$	$Q_{2}$	$Q_{3}$	$Q_{4}$	$Q_{5}$
Ours	0.054	1.000	0.993	0.769	0.411	0.122
MMDAgg	0.050	1.000	0.956	0.585	0.252	0.087
MMD-Fuse	0.056	1.000	0.930	0.622	0.309	0.104
C2ST-RF ‡	0.034 ∗	0.971	0.788	0.433	0.238	0.089
AutoTST ‡	0.035 ∗	0.989	0.871	0.530	0.256	0.096

Table 2: MNIST experiment (

n=200

): full results. Rejection rates over 1000 repetitions at

\kappa=0.05

. The 95% Wilson interval half-width is at most

\pm 0.014

near

0.05

and at most

\pm 0.031

elsewhere. Bold: highest power among methods that do not use extra data. ∗ : 95% interval excludes

\kappa

(above: liberal; below: conservative). † Oracle methods use

n

additional samples for kernel selection. ‡ Classifier-based tests (random-forest C2ST; AutoTST with non-neural tabular models) train on half of the sample and test on the other half.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Selection	Testing	Additional	Total
Our test, MMDAgg, MMD-FUSE	$0$	$n$	$0$	$n$
MMD, median	$0$	$n$	$0$	$n$
MMD, split	$\lfloor n/2\rfloor$	$\lceil n/2\rceil$	$0$	$n$
C2ST, AutoTST	$\lfloor n/2\rfloor$	$\lceil n/2\rceil$	$0$	$n$
MMD, oracle	$n$	$n$	$n$	$2n$

Appendix

Table 3: Sample allocation per distribution in one repetition. Selection includes classifier training. Additional data are independent of the common evaluation sample.

Alternative	Retained digits
$Q_{1}$	$\{1,3,5,7,9\}$
$Q_{2}$	$\{0,1,3,5,7,9\}$
$Q_{3}$	$\{0,1,2,3,5,7,9\}$
$Q_{4}$	$\{0,1,2,3,4,5,7,9\}$
$Q_{5}$	$\{0,1,2,3,4,5,6,7,9\}$

Appendix

Table 4: MNIST alternatives, ordered from fewer to more retained digit classes. Each

Q_{j}

is uniform over images in its indicated empirical pool.

Schedule	$\alpha_{t}$	$\gamma_{t}$
Linear	$1-t$	$\bar{\gamma}_{t}$
Endpoint power, $2$	$(1-t)^{2}$	$\bar{\gamma}_{t}$
Power ratio, $2$	$\dfrac{(1-t)^{2}}{(1-t)^{2}+t^{2}}$	$\bar{\gamma}_{t}$
Quintic smoothstep	$1-S_{5}(t)$	$\bar{\gamma}_{t}$
Normalized sigmoid, $10$	$\dfrac{1}{2}\!\left[1-\dfrac{\tanh(5(t-1/2))}{\tanh(5/2)}\right]$	$\bar{\gamma}_{t}$
$\ell_{2}$ normalized	$\dfrac{1-t}{\sqrt{(1-t)^{2}+t^{2}}}$	$\bar{\gamma}_{t}$

Appendix

Table 5: Gaussian channel schedules;

\beta_{t}=\alpha_{1-t}

in every row.

Explore similar work

Oct 5, 2026stat.ML

Two-Sample Testing via Path-based Inference

Modern deep generative models are primarily studied for their ability to generate realistic samples, yet the generative dynamics they learn can also serve as objects of statistical inference. We develop this idea for two-sample testing, the problem of deciding whether the same distribution generated two finite datasets. Using stochastic interpolants, we connect both distributions to a shared Gaussian bottleneck, so that each half of the resulting path is a Gaussian channel acting on a single population. We prove that the null hypothesis holds if and only if the population denoiser, or equivalently, the velocity fields of the two halves, coincide at any single noise level, which amounts to a reflection symmetry of the path about the bottleneck. Deviations from this symmetry yield a continuum of two-sample witnesses, which we estimate via held-out regression risks on learned denoisers and velocities and aggregate along the path; under an information-theoretic weighting, the aggregated discrepancy equals the Jeffreys divergence between the noise-smoothed distributions. Calibrating the resulting statistics by permutation yields tests that are valid in finite samples for any trained networks and consistent when the fields are learned accurately. On a synthetic benchmark and three image benchmarks, the proposed tests improve power over the strongest baseline by up to 33 percentage points at an equal total sample budget, with the best choice of regression representation and path weighting depending on the data modality. These results show that generative paths provide a principled representation for statistical testing, extending stochastic-interpolant models beyond generation.

Eshant English, Wei-Cheng Lai, Yanfeng Yang +3

The University of Tokyo, Tokyo, Japan · Hasso Plattner Institute for Digital Engineering, Germany · Institute of Statistical Mathematics, Japan +3

Two-Sample Testing

Ask PDF

Jul 23, 2026cs.LG

Zero-Flow Two-Sample Tests

Motivated by the success of modern flow-based generative models in modeling complex data, we study two-sample testing through the lens of flow-based methods. We propose the Zero-Flow Two-Sample Test (ZF2ST), built on the zero-flow criterion, which characterizes distributional equality through a time-reversal antisymmetry of a learnable velocity field. We extend this criterion and further develop the Zero-Flow Discrepancy, an identifying discrepancy that controls the Wasserstein distance, and derive a variational representation in terms of a witness function. This representation naturally leads to a witness-based test whose power is governed by the signal-to-noise ratio (SNR), allowing direct power maximization for witness learning. ZF2ST learns the witness on one data split and performs testing on held-out samples, thereby maintaining Type-I error control and admitting a simple asymptotic null distribution. Experimentally, ZF2ST performs competitively across various synthetic and real-world benchmarks, while showing particularly strong performance in distinguishing image distributions from different sources.

Yakun Wang, Leyang Wang, Song Liu +1

University of Bristol · RIKEN AIP · University of Tokyo & RIKEN AIP

Two-Sample Testing

Ask PDF

Dec 16, 2025stat.ML

Maximum Mean Discrepancy with Unequal Sample Sizes via Generalized U-Statistics

Existing two-sample testing techniques, particularly those based on choosing a kernel for the Maximum Mean Discrepancy (MMD), often assume equal sample sizes from the two distributions. Applying these methods in practice can require discarding valuable data, unnecessarily reducing test power. We address this long-standing limitation by extending the theory of generalized U-statistics and applying it to the usual MMD estimator, resulting in new characterization of the asymptotic distributions of the MMD estimator with unequal sample sizes (particularly outside the proportional regimes required by previous partial results). This generalization also provides a new criterion for optimizing the power of an MMD test with unequal sample sizes. Our approach preserves all available data, enhancing test accuracy and applicability in realistic settings. Along the way, we give much cleaner characterizations of the variance of MMD estimators, revealing something that might be surprising to those in the area: while zero MMD implies a degenerate estimator, it is sometimes possible to have a degenerate estimator with nonzero MMD as well; we give a construction and a proof that it does not happen in common situations.

Aaron Wei, Milad Jalali, Danica J. Sutherland

University of British Columbia · Independent researcher, Vancouver, Canada · Alberta Machine Intelligence Institute

Maximum Mean Discrepancy Two-Sample Testing Sample Size Discrepancy

Ask PDF