In recent years, Rectified flow (RF) has gained considerable popularity largely due to its generation efficiency and state-of-the-art performance. In this paper, we investigate how well RF automatically adapts to the intrinsic low dimensionality of the support of the target distribution to accelerate sampling. We show that, using a carefully designed \blue{end-point adapted} time-discretization scheme and with sufficiently accurate drift estimates, the RF sampler enjoys an iteration complexity of order O(k/ε) (up to log factors), where ε is the precision in total variation distance and k is the intrinsic dimension of the target distribution. In addition, we show that the denoising diffusion probabilistic model (DDPM) procedure is equivalent to a stochastic version of RF by establishing a novel connection between these processes and stochastic localization. Building on this connection, we further design a stochastic RF sampler that also adapts to the low-dimensionality of the target distribution under mild requirements on the accuracy of the drift estimates, and also with a specific time schedule. We illustrate the efficacy of newly designed time-discretization schedules with simulations on the synthetic data and text-to-image (T2I) data experiments.
Figures & tables
Figure 1: Trajectories and generated samples from deterministic (RF) and stochastic (STOC-RF) samplers targeting a two-component Gaussian mixture. Blue contours indicate target density, and highlighted endpoints lie in low-density regions. Purple arrows illustrate the score-driven component of the Langevin correction, which locally directs samples toward higher density and can counteract accumulated sampling errors—acting as a soft “guardrail.” The stochastic sampler exhibits fewer low-density endpoints in this example.
Figure 2: Stochastic localization (SL) provides a unifying bridge between DDPM and STOC-RF. Lemmas 3.1 and 3.2 establish the respective equivalences in law under suitable time reparameterizations and scalings, leading to Proposition 3.3, which connects the DDPM and STOC-RF samplers.
Figure 3: (a)-(b) Estimated TV distance between blurred low-rank Gaussian distribution and the generated samples by the RF sampler ( 5 ) under uniform and non-uniform time-grid under varying d . (c)-(d) Estimated TV distance between blurred low-rank Gaussian distribution and the generated samples by the RF and STOC-RF samplers under uniform and non-uniform time-grid for varying d .
Table 1: Comparison of sampling methods and time discretization schedules. (a) CLIP, Pick, and CER metrics for text-to-image (T2I) generation. (b) FID and MMD metrics on MNIST under different time grids. Best results are highlighted in bold.
Figure 4: Generated images using Flux for specific prompts via Euler’s scheme ( 5 ) under non-uniform time-grid ( 16 ) ( Red frame ) and uniform time-grid ( Blue frame ).
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Empirical Validation of TV Distance Scaling Laws and Bound Utilisation.
Method
CLIP ( ↑ )
PickScore ( ↑ )
CER ( ↓ )
Prompt: Robot Lecturer
ODE + Lin
31.420
0.225
2.574
ODE + Geom
30.520
0.228
2.957
SDE (vanilla)
33.710
0.211
4.939
STOC-RF
31.310
0.237
2.044
Prompt: T-shirt (“I love AI”)
Appendix
Table 2: Comparison of generative sampling methods across six T2I prompts from Appendix A.2 using CLIP, PickScore, and CER. Best results for each prompt are highlighted in bold.
Figure 6: Generated images using Flux for specific prompts via Euler’s scheme ( 5 ) under non-uniform time-grid ( 16 ) ( Columns 1 ), uniform time-grid (Column 2), STOC-RF sampler ( Column 3 ), and vanilla SDE sampler (Column 4). Red color indicates the proposed methods in this paper. The generation quality of Stoc-RF is generally better out of the four samplers.
Time Grid
CLIP ( ↑ )
PickScore ( ↑ )
CER ( ↓ )
Prompt: Cat with sign
Linear
35.54±1.93
0.2402±0.0068
0.6851±0.0454
Geometric
35.44±0.98
0.2401±0.0025
0.7064±0.0159
Cosine
35.27±2.33
0.2414±0.0033
0.7489±0.0834
Logit
35.24±0.94
0.2403±0.0023
0.7064±0.0085
Prompt: T-shirt (“I love AI”)
Appendix
Table 3: Ablation study of time discretization schedules on five T2I prompts from Appendix A.2. Best results for each prompt are highlighted in bold.
Deterministic RF (ODE + Geometric)
STOC-RF (SDE)
NFE
Time (s)
CLIP ( ↑ )
PickScore ( ↑ )
CER ( ↓ )
Time (s)
CLIP ( ↑ )
PickScore ( ↑ )
CER ( ↓ )
Prompt: Cat with sign
5
2.8
31.01
0.2124
0.8553
2.1
23.15
0.1910
1.0000
10
5.5
34.15
0.2326
0.7745
4.8
23.15
0.1910
1.0000
20
11.4
34.35
0.2399
0.7362
10.8
18.46
0.1671
1.0000
30
17.0
34.31
0.2403
0.7021
16.3
34.97
0.2294
0.8468
Appendix
Table 4: Quality–speed trade-off between deterministic RF (ODE) and STOC-RF (SDE) on two T2I prompts. Sampling time is measured on an NVIDIA L40S GPU. NFE denotes the number of function evaluations.
Rectified flow is a generative modeling method that learns near-straight transports from noise to data for efficient sampling and conditional generation.
Flow-based generative models have emerged as an efficient alternative to diffusion models. We study the sample complexity of learning a target distribution in Wasserstein distance via flow matching, which learns a velocity field transporting a base distribution to the target. Under exact empirical risk minimization over a fixed network class and standard assumptions on the architecture and data distribution, we show that any squared-loss flow-matching variant including rectified flow with bounded derivatives achieves O(ε−2) sample complexity for the statistical term with the Wasserstein guarantee taking the form W2≤K(ε+Cεapp), where εapp is the approximation error of the network class. This improves upon existing O(ε−4) guarantees and does not require the Polyak--Lojasiewicz condition in the statistical analysis. Instead, we exploit a structural property of the squared-loss objective - it induces an approximate Bernstein condition that directly ties the variance of the excess loss to the excess risk up to additive approximation-error. This condition enables a localized Rademacher complexity analysis yielding fast O(1/n) rates. The flow-matching schedule enters the leading statistical prefactor through the Lipschitz constant of the pointwise loss, which we characterize explicitly for standard schedules.
This paper investigates how diffusion generative models leverage (unknown) low-dimensional structure to accelerate sampling. Focusing on two mainstream samplers -- the denoising diffusion implicit model (DDIM) and the denoising diffusion probabilistic model (DDPM), we prove that their iteration complexities under exact score functions are at most the order of k/ε (up to log factor), where ε is the precision in total variation distance and k is some intrinsic dimension of the target distribution. We further extend these convergence guarantees to the setting in which the score functions are learned from data rather than known exactly, showing that the convergence performance degrades gracefully under suitable score estimation assumptions. We then show that these assumptions are attainable via kernel-based score estimators with finite-sample guarantees that also adapt to the low-dimensional structure. Our results apply to a broad family of target distributions without requiring smoothness or log-concavity. Our findings provide the first rigorous evidence for the adaptivity of the DDIM-type samplers to unknown low-dimensional structure, and improve over the state-of-the-art DDPM theory regarding total variation convergence.
Jiadong Liang, Zhihan Huang, Yuxin Chen
Department of Statistics and Data Science, the Wharton School, University of Pennsylvania
Diffusion models are known to exploit unknown low-dimensional structure to accelerate sampling. However, existing convergence theory under low-dimensional data structure has largely focused on update rules with narrowly prescribed coefficient choices. This raises a fundamental question: is adaptation to low-dimensional structure sensitive to the precise choice of update coefficients? In this paper, we show that such adaptation is a robust property of diffusion models. For a broad class of update coefficients, we prove that O(k/ε) iterations suffice to generate an ε-accurate sample in total variation (TV) distance, independently of the ambient dimension. Our framework substantially broadens the class of diffusion samplers known to enjoy low dimensional adaptation and applies to several commonly used methods in practice. These results provide a theoretical justification for the empirical effectiveness of diffusion samplers across different coefficient choices when applied to structured, high-dimensional data.
Changxiao Cai, Yuchen Jiao, Gen Li
Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, USA · Department of Statistics and Data Science, Chinese University of Hong Kong, Hong Kong