Current auto-regressive (AR) LLMs, diffusion-based text/image generative models, and recent flow matching (FM) algorithms are capable of generating premium quality text/image samples. However, the inference or sample generation in these models is often very time-consuming and computationally demanding, mainly due to large numbers of function evaluations corresponding to the lengths of tokens or the numbers of diffusion steps. This also necessitates heavy GPU resources, time, and electricity. In this work we propose a novel solution to reduce the sample generation time of flow matching algorithms by a guaranteed speed-up factor, without sacrificing the quality of the generated samples. Our key idea is to utilize computationally lightweight generative models whose generation time is negligible compared to that of the target AR/FM models. The draft samples from a lightweight model, whose quality is not satisfactory but fast to generate, are regarded as an initial distribution for a FM algorithm. Unlike conventional usage of FM that takes a pure noise (e.g., Gaussian or uniform) initial distribution, the draft samples are already of decent quality, so we can set the starting time to be closer to the end time rather than 0 in the pure noise FM case. This will significantly reduce the number of time steps to reach the target data distribution, and the speed-up factor is guaranteed. Our idea, dubbed {\em Warm-Start FM} or WS-FM, can essentially be seen as a {\em learning-to-refine} generative model from low-quality draft samples to high-quality samples. As a proof of concept, we demonstrate the idea on some synthetic toy data as well as real-world text and image generation tasks, illustrating that our idea offers guaranteed speed-up in sample generation without sacrificing the quality of the generated samples.
Figures & tables
Figure 1: Training and inference pipelines for our warm-start flow matching method.
Figure 2: Training algorithm comparison between DFM ( Gat et al., 2024 ) (LEFT) and our WS-DFM (RIGHT). Changes shown as red.
Figure 3: Inference algorithm comparison between DFM ( Gat et al., 2024 ) (LEFT) and our WS-DFM (RIGHT). Changes as red.
Figure 4: (Two moons) (a) Target distribution P1(x) . (b) Pure uniform noise distribution P0(x) used in DFM ( Gat et al., 2024 ) . (c–e) Three lightweight draft models adopted in our WS-DFM: (c) = pretty good, (d) = fair, (e) = poor.
t0
SKL ( ↓ )
NFE ( ↓ )
Original DFM ( Gat et al., 2024 )
0
0.62
20
WS-DFM + Pretty good draft model
0.95
0.74 ✘
1
0.9
0.54 ✔
2
0.8
0.37 ✔
4
WS-DFM + Fair draft model
0.8
0.86 ✘
4
0.5
0.51 ✔
10
Table 1: (Two moons) Sample generation results. The symmetric KL (SKL) divergences between generated and true samples, and the number of function evaluations (NFE) are shown. In our WS-DFM, those cases where the sample quality is no worse than original DFM are marked as ✔ .
Figure 5: (Two moons) (a) Generation steps in the original DFM ( Gat et al., 2024 ) with step size 0.05 ( NFE=20 steps). Only every other time steps are shown. (b–d) Our WS-DFM results for three different lightweight draft models: (b) = pretty good, (c) = fair, (d) = poor. Our WS-DFM models achieve sample quality no worse than DFM’s, but with significant speed-up in sample generation ( ×10 , ×2 , and ×1.5 depending on the draft model’s performance).
NLL ( ↓ )
Entropy ( ↑ )
Time ( ↓ )
LSTM
6.87
7.19
Negligible
Original DFM ( Gat et al., 2024 )
6.58
7.14
6.56
Our WS-DFM ( t0=0.8 )
6.54
7.11
1.36
Our WS-DFM ( t0=0.5 )
6.48
7.05
3.36
Refined by Gemma3 27B
6.54
7.18
−
Table 2: (Text-8) Text generation results. The NLL scores measured by GPT-J-6B LLM, the entropy in next token prediction, and the per-sentence generation time (in seconds) are shown.
Perplexity ( ↓ )
Entropy ( ↑ )
Time ( ↓ )
LSTM
171.23
7.56
Negligible
Original DFM ( Gat et al., 2024 )
69.06
7.42
8.33
Our WS-DFM ( t0=0.8 )
67.86
7.19
1.70
Our WS-DFM ( t0=0.5 )
64.68
7.16
4.20
Refined by Gemma3 27B
32.88
7.14
−
Table 3: (Wikitext-103) Text generation results. Perplexity scores are measured by GPT-J-6B LLM, and per-sentence generation time (in seconds) is also shown.
Figure 6: (CIFAR-10 Gray-scale) Some image samples generated by the competing methods.
Figure 7: (CIFAR-10 Gray-scale) Progress in sample generation in our WS-DFM with t0=0.5 . The left most column contains images generated by the lightweight DC-GAN model, which are progressively refined by our WS-DFM, and the rightmost column shows the finaloutput images of our WS-DFM.
Gray-scale
Color
FID ( ↓ )
Time ( ↓ )
FID ( ↓ )
Time ( ↓ )
DC-GAN
74.64
Negligible
80.91
Negligible
Original DFM
30.46
0.62
36.91
2.64
WS-DFM ( t0=0.8 )
23.59
0.13
37.02
0.55
WS-DFM ( t0=0.65 )
22.75
0.23
36.47
0.94
WS-DFM ( t0=0.5 )
19.47
0.32
34.65
1.34
Table 4: (CIFAR-10) Image generation results. The FID score and the per-image generation time (in seconds) are shown.
Figure 8: (CIFAR-10 Color) Some image samples generated by the competing methods.
Figure 9: (CIFAR-10 Color) Progress in sample generation in our WS-DFM with t0=0.5 . The left most column contains images generated by the lightweight DC-GAN model, which are progressively refined by our WS-DFM, and the rightmost column shows the finaloutput images of our WS-DFM.
Figure 10: (Text-8) Some text samples generated by the DFM ( Gat et al., 2024 ) , the LSTM draft model, and our WS-DFM models.
Figure 11: (CIFAR-10) Examples of k -nearest neighbor samples from the refinement strategy in our WS-DFM for data pairing (xt0,x1) . The top four panels are gray-scale data, and the bottom four panels colored data. For each panel, the leftmost column contains images generated by the lightweight DC-GAN models, and the next 5 columns show 5 -nearest neighbor images retrieved from the training data.
Figure 12: (CIFAR-10 Gray-scale) Sample images generated by WS-DFM t0=0.8 (TOP) and t0=0.5 (BOTTOM).
Figure 13: (CIFAR-10 Color) Sample images generated by WS-DFM t0=0.8 (TOP) and t0=0.5 (BOTTOM).
Figure 14: (Wikitext-103) Some text samples generated by the DFM ( Gat et al., 2024 ) , the LSTM draft model, and our WS-DFM models.
We present FlowLM, a flow matching language model transformed from pre-trained diffusion language models via efficient fine-tuning. By re-aligning the curved sampling trajectories of diffusion models into straight-line flows, FlowLM enables high quality few-step generation that rivals or even outperforms the quality of 2,000-step diffusion sampling with very few training epochs. Remarkably, finetuned FlowLM reaches performance saturation with only half as many training epochs as training from scratch, both approaches greatly outperforming the original diffusion model, thereby validating our method. Furthermore, we validate a more effective training objective for flow matching: predicting clean data to consistently guide the sampling process towards the true data distribution. Empirical results demonstrate that our approach is highly effective for high-quality, few-step text generation.
Flow Matching (FM) is a simulation-free method for learning a continuous, invertible flow that interpolates between two distributions, and in particular generates data from noise. Inspired by the variational nature of the diffusion process as a gradient flow, we introduce a stepwise FM model, Local Flow Matching (LFM), which sequentially learns a sequence of FM submodels, each matching a diffusion process up to the time-step size in the data-to-noise direction. In each step, the two distributions to be interpolated by the sub-flow model are closer than those in the full-flow matching model, which interpolates data to noise distributions, enabling smaller models with more efficient training. This variational perspective also allows us to prove a theoretical generation guarantee for the proposed flow model in terms of the χ2-divergence between the generated and true data distributions, leveraging the contraction property of the diffusion process. In practice, the stepwise structure of LFM is naturally amenable to model distillation, and various distillation techniques can be applied to accelerate generation. We empirically demonstrate that LFM achieves competitive generative performance compared to FM on unconditional generation of tabular and image datasets, and on conditional generation of robotic manipulation policies.
Chen Xu, Xiuyuan Cheng, Yao Xie
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology. · Department of Mathematics, Duke University
We propose Perceptual Flow Matching (PFM), a simple yet effective framework for few-step generation in flow-matching models. Rather than performing velocity regression in the conventional VAE latent space, PFM supervises flow matching in a perceptual feature space using pretrained perceptual models. This simple change substantially improves the few-step generation capability of flow-matching models, reducing the number of sampling steps from 35-50 to 4-8 while preserving generation quality. Unlike existing acceleration and distillation approaches, PFM requires neither teacher models nor auxiliary score networks and can be integrated into standard flow-matching training pipelines with minimal modifications. Extensive experiments on image generation, video generation, and image editing tasks demonstrate that PFM consistently produces high-quality results while producing fewer artifacts than existing distillation-based methods. We further show that perceptual supervision shifts the regression minimizer from mean-seeking to mode-seeking, biasing predictions toward on-manifold modes that remain accurate under coarse few-step integration. Our results reveal that standard flow-matching training can naturally yield high-quality few-step generators when supervised in an appropriate representation space. We hope this insight inspires future research into representation-aware objectives for efficient generative modeling.
Chuyang Zhao, Yifei Song, Hongfa Wang +7
1Joy Future Academy · 2Fudan University · 3Tsinghua University +1