Generalization Bounds for Flow-matching Generative Models for Intrinsically Low-dimensional Data
Authors: Saptarshi Chakraborty, Quentin Berthet, Peter L. Bartlett
Organizations: Department of Statistics, University of Michigan · Google DeepMind · Department of Statistics, University of California, Berkeley · Department of Electrical Engineering and Computer Sciences, UC Berkeley
Despite the remarkable empirical success of flow-matching models, their statistical generalization guarantees remain underdeveloped. Existing analyses often impose restrictive assumptions on the estimated velocity field and yield convergence rates that fail to reflect the intrinsic low-dimensional structure common in real data, such as natural images and molecular geometries. In this work, we study the statistical generalization of flow-matching models for learning an unknown distribution Pdata from finitely many samples. We derive finite-sample error bounds on the learned generative distribution, measured in the Wasserstein-p distance, for all p≥1. Specifically, given n i.i.d. samples from Pdata, we show that, for every d>dp∗(Pdata) and appropriately chosen network architectures and hyperparameters, the learned distribution PFM satisfies Wp(PFM,Pdata)≲n−1/d+n−1/(2p)(log(1/ξ))1/(2p) with probability at least 1−ξ, where dp∗(Pdata) denotes the Wasserstein-p dimension of the target measure. Our results demonstrate that flow matching naturally adapts to the intrinsic geometry of data and mitigates the curse of dimensionality, as the convergence exponent depends on the intrinsic rather than ambient dimension. These guarantees remain meaningful in high-dimensional regimes and provide a theoretical explanation for the empirical success of flow matching on structured data distributions under substantially more relaxed assumptions than those in existing analyses.
Figures & tables
Figure 1 : Average generalization error (in terms of FID scores) for different values of n for flow-matching. The error bars denote the standard deviation out of 10 replications.
Flow Matching (FM) learns a velocity field whose ODE transports a simple source distribution to a target law. Existing finite-sample theory largely treats ambient-space regularity or data supported exactly on low-dimensional sets. We study linear FM under a noisy latent-generator model, where a low-dimensional Hölder map is perturbed by nondegenerate ambient Gaussian noise, so the target law is full-dimensional despite its latent structure. We construct a spatially regular ReLU velocity class and establish non-asymptotic high-probability approximation and estimation bounds whose leading sample-size exponent is governed by the latent dimension rather than the ambient dimension, with ambient and noise dependence kept explicit. Fixed positive target noise keeps the interpolation nondegenerate over the full time interval. The same spatial regularity propagates the learned velocity error through the transport ODE, yielding a corresponding Wasserstein convergence guarantee. These results show that exact low-dimensional support is not necessary for Flow Matching to retain latent-dimensional statistical behavior.
Lifeng Hao, Shaolin Ji
Zhongtai Securities Institute for Financial Studies, Shandong University, Jinan 250100, Shandong, China.
Diffusion Flow Matching (DFM) has recently emerged as a versatile framework for generative modeling, yet its theoretical convergence properties remain only partially understood. In this work, we provide refined and novel convergence guarantees for Brownian motion based DFMs, focusing on the discretization error. Our analysis is conducted under the Kullback-Leibler (KL) divergence and the 2-Wasserstein distance. Under finite-moment conditions and a mild score integrability assumption, we derive KL convergence bounds with improved dimensional dependence compared to prior work, achieving, up to our knowledge, state-of-the-art scaling under minimal conditions. We further extend the analysis to the 2-Wasserstein distance: under an additional first-order score integrability assumption and a weak log-concavity condition, we obtain convergence guarantees with dimensional dependence consistent with the KL case.
Marta Gentiloni Silveri, Giovanni Conforti, Alain Durmus
Ecole Polytechnique, Massy Palaiseau, France · Università degli Studi di Padova, Padua, Italy
In this work, we develop theoretical foundation for flow matching with neural-network-parameterized conditional velocity fields. We establish convergence guarantees for gradient descent in the over-parameterized 2-layered ReLU neural network regime. We derive generalization bounds for the conditional velocity-field matching objective. Building on these results, we provide Wasserstein-distance guarantees for the samples generated by the induced flow. Our analysis is based on generalization bound for multi-task representation learning with unbounded losses, which may be of independent interest beyond flow-based generative modeling. These theoretical results are validated through extensive experiments on both synthetic and real-world image benchmarks.
Yihan He, Qishuo Yin, Yuan Cao +2
Princeton University, Princeton, NJ, USA. · The University of Hong Kong, Hong Kong. · Northwestern University, Evanston, IL, USA.