Flow matching models, ODE-based generative models, generate samples by gradually morphing a simple source distribution into a target distribution. However, these models still face challenges in optimally balancing computational efficiency with generation fidelity. While predictor-corrector schemes have been proven to help address this issue in diffusion models effectively, their application to flow matching models remains under-explored. In this work we present a novel training-based predictor-corrector framework for flow matching, which exploits the fact that the ODE can be easily reversed. Our key idea, termed Marginal Matching, is to correct samples at a particular time step in the forward ODE towards a matching marginal distribution of the backward ODE. This scheme proves to reduce a bound on the distance between the data distribution and the learned distribution, motivating two different implementations for the correction function. We show that our proposed method improves sample quality on CIFAR-10 and ImageNet-64, with minimal overhead in computation time.
Figures & tables
Figure 1: Illustration of Marginals (a) Gaussian source distribution π0 (red), and target distribution π1 (blue). (b) Discrete forward marginals {ptnf}n=0N . (c) Discrete backward marginals {ptnb}n=0N . Note that ptnb=ptnf for every n , specifically p0b (= π^0 , black) differs from p0f (= π0 , red).
Figure 2: Marginals in Vector Field The black lines show two key trajectories in a vector field. The lower is the forward trajectory, starts at π0 and ends at p1f=π^1 . The upper is the backward trajectory, begins at p0b and ends at the data distribution π1 . Both are discretized at time-steps t and t′ . Our objective is to transition from forward to backward marginals trajectory.
Model
Sampler
NFE ↓
FID ↓
DDPM
Adaptive
274
7.48
Score Matching
Adaptive
242
19.94
OT-FM
Adaptive
142
6.35
OT-CFM
RK4
40
4.34
Adaptive
133.94
3.58
I-CFM
RK4
40
4.29
Table 1: CIFAR-10 performance. (Top) Leading baseline models. (Bottom) Our correction models (score and classifier) with OT-CFM and I-CFM base flow models. The base flow models are sampled with 10-step RK4 (40 NFE). Correction steps improve performance with minimal additional NFEs, and generally, increasing the number of applied corrections (C-NFE) enhances overall results. Additional comparisons are available in Appendix. G.6 .
Figure 3: CIFAR-10 sample quality comparison of score correction trained on the interpolated ( π1 and π0 ) vs. backward marginals ptb . The base flow is sampled with 10-step RK4.
Model
NFE ↓
Sample
FID ↓
DDPM
264
Adaptive
17.36
Score Matching
441
Adaptive
19.74
OT-FM
138
Adaptive
14.45
OT-CFM
20
RK4
17.99
24
RK4
16.60
28
RK4
15.71
Table 2: ImageNet-64 results for top generative models (Top) vs. score correction with OT-CFM base flow (Bottom), which is sampled with 5-step RK4 (20 NFE).
Figure 4: Correction models with OT-CFM as the base flow model on CIFAR-10 (Left) and ImageNet-64 (Right). The ”-Diff” shows the difference between corrected and uncorrected images. Corrected images show improved sharpness and better alignment with the dataset.
Figure 5: CIFAR-10 sample quality of score correction sampled with 50 and 100 step Langevin Dynamics versus a single correction step at the 10th time-step marginal. The base flow is sampled with 10-step RK4.
Figure 6: CIFAR-10 sample quality. The base flow model is OT-CFM sampled with 10-step RK4. Left: FID vs accumulative correction time-steps. Middle: FID vs single correction time-step. Right: Ablation study on classifier loss components sampled across different marginals.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Parallel Sampling The black arrows denote the path of first flowing with vθ and only then taking a correction step with hψ . In contrast, the dashed green arrow shows the approximation obtained by summing the direction of the flow model vθ (black arrow), and the correction model hψ (green arrow) on the current step.
OT-CFM
I-CFM
Correction Model
NFE ↓
C-NFE ↓
FID ↓
FID ↓
No Correction
40
0
4.34
4.29
Classifier
42
1 ( n=8 )
3.57
3.77
46
3 ( n=8,9,10 )
3.48
3.67
50
5 ( n=6,7,8,9,10 )
3.47
3.62
62
11 ( ∀n )
3.48
3.63
Appendix
Table 3: Comparison of two variants of classifier and score correction models on CIFAR-10. First, a correction model that runs in parallel with (”P-”) the flow model. Second, a correction model that learns an interpolation of the data and approximate source distribution. The base flow models are sampled with 10-step RK4 (40 NFE). The classifier succeeds in improving the FID even with the parallel approximation, while the score shows less improvement. Training with interpolated backward marginals improves the FID score, though less than using their exact version.
Correction Model
NFE ↓
C-NFE ↓
FID ↓
Score
21
1 ( n=5 )
15.91
22
2 ( n=0,5 )
15.53
26
6 ( n=0…5 )
15.57
Score (Backward Marginals Interpolation)
21
1 ( n=5 )
15.89
22
2 ( n=0,5 )
15.62
26
6 ( n=0…5 )
15.69
Appendix
Table 4: Comparison of backward marginals ptb and approximated marginals results on ImageNet-64 using OT-CFM as the base flow model, sampled with 5 -step RK4 ( 20 NFE). Both versions improve the original score, though the approximation’s benefit is smaller.
Figure 8: Percentage vs L2 difference. Histogram showing L2 differences of VGG features between samples generated using a flow model alone versus with a correction model. The samples are generated from identical Gaussian noise. The percentage indicates the proportion of images with the corresponding difference. The score and classifier correction models leave most images largely unchanged, with differences concentrated in a small range, while significantly altering only outliers. This suggests that there is an intersection between the forward and backward marginals.
Figure 9: TSNE of Classifier Features Red represents the forward marginals, while blue denotes the backward marginals. The classifier’s features can differentiate between the forward and backward marginals in most cases. However, in some instances, these marginals are indistinguishable from one another.
Figure 10: TSNE of Forward and Backward Marginals Red represents the forward marginals, while blue denotes the backward marginals. As time progresses, we observe a convergence between these two sets of marginals. However, it’s important to note that despite this increasing proximity, they never achieve perfect alignment or identity. n=0 compares the approximate source distribution distribution ( p0b ) and Gaussian noise while n=10 compares the model’s data approximation ( p1f ) and the real data.
Correction Model
NFE ↓
C-NFE ↓
FID ↓
0.001-Score
41
1
3.83
43
3
3.75
51
11
3.77
0.003-Score
41
1
3.51
43
3
3.44
51
11
3.45
Appendix
Table 5: Ablation study on the sigma value of score correction model sampled on n={[10],[0,5,10],∀n} marginals on CIFAR-10. The σ− Score represents the value of σ the score model was trained with. The base flow model is OT-CFM sampled with 10-step RK4 (40 NFE). The results demonstrate that the optimal performance is achieved when σ=0.005 (the value used in the main paper).
Figure 11: CIFAR-10 images generated by OT-CFM sampled with 10-step RK4, shown alongside their counterparts produced with class conditioned classifier correction. The classifier successfully directs the noise to the correct class.
Correction Model
NFE ↓
C-NFE ↓
FID ↓
No Correction
40
0
4.34
OT-CFM – 1st Marginal
48
2
9.74
64
6
6.31
120
20
6.33
OT-CFM – Nth Marginal
48
2
7.43
64
6
4.39
Appendix
Table 6: CIFAR-10 FID comparison of OT-CFM correction and flow model trained on first and last marginals. The correction score model was sampled with C-NFE RK4 steps, while the base flow model was sampled with 10-step RK4 (40 NFE). The correction flow model degrades the results.
Flow Model
NFE ↓
FID ↓
OT-CFM
40
6.43
44
6.04
48
5.80
52
5.70
Correction Model
NFE ↓
C-NFE ↓
FID ↓
Score
41
1 ( n=10 )
5.52
Appendix
Table 7: CIFAR-10 test set FID performance of OT-CFM flow model with and without our correction models (score and classifier). OT-CFM is sampled without correction using 10,20, and 50-step RK4. Our correction models use OT-CFM with 10-step RK4 (40 NFE). In general, adding correction steps (C-NFE) improves the results.
NFE
FID ↓
40
4.44
44
4.09
48
3.79
52
3.56
Appendix
Table 8: Results of residual networks combined with both the base flow model and the residual network OT-CFM flow models.
Model
NFE
FID ↓
OT-CFM ( Tong et al., 2024 )
40
4.34
44
3.96
48
3.73
52
3.59
56
3.52
60
3.47
Appendix
Table 9: CIFAR-10 FID scores for OT-CFM and I-CFM flow models sampled with RK4. The number of RK4 steps is 1/4 of the number of the NFEs.
CIFAR-10
ImageNet-64
Channels
128
192
Depth
2
3
Channels multiple
1,2,2,2
1,2,3,4
Heads
4
4
Heads Channels
64
64
Attention resolution
16
32,16,8
Appendix
Table 10: Hyper-parameters used for CIFAR-10 and ImageNet-64 flow models.
Classifier
Score
CIFAR-10
CIFAR-10
ImageNet-64
Channels
128
128
192
Depth
1
1
2
Channels multiple
1,2,1,2
1, 2, 1, 2
1, 2, 3, 2
Heads
4
4
4
Heads Channels
32
32
64
Appendix
Table 11: Correction models hyper-parameters.
Corr. Step
Classifier
10 Steps
5 Steps
3 Steps
1 Step
Step Size
Noise
Step Size
Noise
Step Size
Noise
Step Size
Noise
0
1.5
0.05
-
-
-
-
-
-
1
1.5
0.0
-
-
-
-
-
-
2
1.5
0.0
-
-
-
-
-
-
3
1.5
0.0
-
-
-
-
-
-
Appendix
Table 12: CIFAR-10 evaluation hyper-parameters for the classifier correction model.
Corr. Step
Score
10 Steps
3 Steps
1 Step
Step Size
Noise
Step Size
Noise
Step Size
Noise
0
0.4
0.03
0.35
0.03
-
-
1
0.3
0.0
-
-
-
-
2
0.3
0.0
-
-
-
-
3
0.3
0.0
-
-
-
-
Appendix
Table 13: CIFAR-10 evaluation hyper-parameters for score correction model.
Corr. Step
Score
5 Steps
2 Steps
1 Step
Step Size
Noise
Step Size
Noise
Step Size
Noise
0
0.4
0.1
0.4
0.01
-
-
1
0.2
0.05
-
-
-
-
2
0.2
0.0
-
-
-
-
3
0.2
0.0
-
-
-
-
Appendix
Table 14: ImageNet-64 evaluation hyper-parameters for score correction model.
Figure 12: ImageNet-64 image generation using OT-CFM alone and with score correction model. The corrected images demonstrate enhanced sharpness and definition compared to their uncorrected counterparts.
Figure 13: CIFAR-10 image generation using OT-CFM alone and with score correction and classifier correction models. Despite the distinct nature of these correction models, their suggested improvements often exhibit remarkable similarity. The corrected images demonstrate enhanced sharpness and definition compared to their uncorrected counterparts.
Figure 14: ImageNet-64 images generated by OT-CFM alone and with score correction model applied solely to the final step, accompanied by their difference (amplified for visibility). The noise-like appearance of the difference suggests the presence of residual noise in the model’s predictions.
Figure 15: CIFAR-10 images generated by OT-CFM alone and with score correction model applied solely to the final step, accompanied by their difference (amplified for visibility). The noise-like appearance of the difference suggests the presence of residual noise in the model’s predictions.
Department of Electrical and Computer Engineering, Ohio State Univeristy · Department of Computer Science and Information Systems, BITS Pilani · Department of Electrical and Computer Engineering, Tufts University +2