Curriculum Sampling: A Two-Phase Curriculum for Efficient Training of Flow Matching

Pengwei Sun

cs.LGMar 12, 2026

Curriculum Sampling: A Two-Phase Curriculum for Efficient Training of Flow Matching

Authors: Pengwei Sun

Organizations: Stanford University

Abstract

Timestep sampling $p(t)$ is a central design choice in Flow Matching models, yet common practice increasingly favors static middle-biased distributions (e.g., Logit-Normal). We show that this choice induces a speed--quality trade-off: middle-biased sampling accelerates early convergence but yields worse asymptotic fidelity than Uniform sampling. By analyzing per-timestep training losses, we identify a U-shaped difficulty profile with persistent errors near the boundary regimes, implying that under-sampling the endpoints leaves fine details unresolved. Guided by this insight, we propose Curriculum Sampling, a two-phase schedule that begins with middle-biased sampling for rapid structure learning and then switches to Uniform sampling for boundary refinement. On CIFAR-10, Curriculum Sampling improves the best FID from $3.85$ (Uniform) to $3.22$ while reaching peak performance at $100$ k rather than $150$ k training steps. Our results highlight that timestep sampling should be treated as an evolving curriculum rather than a fixed hyperparameter.

Figures & tables

Figure 1: Comparison of best achieved FID scores across non-curriculum sampling strategies. Methods are sorted from best (lowest FID) to worst.

Figure 2: Comparison of Fréchet Inception Distance (FID) scores over the first 100k training steps for Uniform sampling versus Logit-Normal (LN, $\mu\in\{-0.8,-0.4\},\sigma=1$ ) and Mode ( $s=1.0$ ) sampling.

Figure 3: Time-Dependent Training Loss Dynamics and Sampling Densities. Top row: Evolution of the loss over the diffusion time $t\in[0,1]$ throughout training (color gradient from 10k to 100k steps). Bottom row: Histograms of sampled time steps $t\sim p(t)$ from each distribution. Columns represent: (a) Uniform sampling, (b) Mode sampling ( $s=-0.5$ ), which skews towards $t=0$ and $t=1$ , (c) Logit-Normal sampling ( $\mu=0.8$ ), shifting focus towards the noise-dominant regime ( $t$ close to 1), and (d) Logit-Normal sampling ( $\mu=-0.4$ ), emphasizing the mid-trajectory.

$p_{1}(t)$	$p_{2}(t)$	$T_{s}$	Training Step @ Best FID	Best FID
LN ( $\mu=0.8$ )	Uniform	60k	100k	3.22
LN ( $\mu=-0.4$ )	Uniform	60k	100k	3.24
LN ( $\mu=0.8$ )	Uniform	50k	100k	3.34
LN ( $\mu=0.8$ )	Mode	60k	100k	3.42
LN ( $\mu=0.8$ )	Mode	90k	100k	3.43
LN ( $\mu=0.8$ )	Uniform	70k	100k	3.45

Table 1: Comparison of Best FID Scores with different curriculum settings. The results are ranked from low to high (lower is better). The last two entries are static sampling without curriculum.

Flow Matching

Flow matching learns continuous-time vector fields that transport simple noise distributions into data for efficient generative modeling.

See all 639 papers

Discrete Meanflow Training Curriculum
Apr 10, 2026Chia-Hong Hsu, Frank WoodFlow-Based Generative Modeling MeanFlow
Department of Computer Science, University of British Columbia, Vancouver, BC, Canada
Difficulty-Calibrated Interpolation Paths for Conditional Flow Matching
Aug 21, 2026Airin Akter Tania, Md Raihan KhanFlow Matching Conditional Flow Matching
Dept. of Electrical and Electronic Engineering Khulna University of Engineering & Technology Khulna, Bangladesh · Dept. of Electrical and Electronic Engineering North Western University Khulna, Bangladesh
Sharpen Your Flow: Sharpness-Aware Sampling for Flow Matching
May 12, 2026Aditi Gupta, Soon Hoe Lim, Annan Yu +1Flow Matching Flow-Based Generative Modeling
Lawrence Berkeley National Laboratory · International Computer Science Institute · 1Lawrence Berkeley National Laboratory, 2International Computer Science Institute +4

Apr 10, 2026J/K move · Enter open · S save

Discrete Meanflow Training Curriculum

Chia-Hong Hsu, Frank Wood

Department of Computer Science, University of British Columbia, Vancouver, BC, Canada

Flow-Based Generative Modeling MeanFlow One-Step Generative Modeling

Open paper AskPDF arXiv

Flow-based image generative models exhibit stable training and produce high quality samples when using multi-step sampling procedures. One-step generative models can produce high quality image samples but can be difficult to optimize as they often exhibit unstable training dynamics. Meanflow models exhibit excellent few-step sampling performance and tantalizing one-step sampling performance. Notably, MeanFlow models that achieve this have required extremely large training budgets. We significantly decrease the amount of computation and data budget it takes to train Meanflow models by noting and exploiting a particular discretization of the Meanflow objective that yields a consistency property which we formulate into a ``Discrete Meanflow'' (DMF) Training Curriculum. Initialized with a pretrained Flow Model, DMF curriculum reaches one-step FID 3.36 on CIFAR-10 in only 2000 epochs. We anticipate that faster training curriculums of Meanflow models, specifically those fine-tuned from existing Flow Models, drives efficient training methods of future one-step examples.

Abstract

Flow Matching

Explore similar work

Discrete Meanflow Training Curriculum

Difficulty-Calibrated Interpolation Paths for Conditional Flow Matching

Sharpen Your Flow: Sharpness-Aware Sampling for Flow Matching