From Euler to Dormand-Prince: ODE Solvers for Flow Matching Generative Models

Hao Xiao

cs.LGApr 4, 2026

From Euler to Dormand-Prince: ODE Solvers for Flow Matching Generative Models

Authors: Hao Xiao

Organizations: ATLAS AI Lab

Abstract

Sampling from Flow Matching generative models requires solving an ordinary differential equation (ODE) whose computational cost is dominated by neural network forward passes. We derive four classical ODE solvers -- Euler, Explicit Midpoint, Classical Runge-Kutta (RK4), and Dormand-Prince 5(4) -- from first principles via Taylor expansion, implement them from scratch in PyTorch, and systematically benchmark their efficiency on Conditional Flow Matching tasks ranging from 2D toy distributions to MNIST digits. On the quantitative side, we use sliced Wasserstein distance to construct NFE-quality Pareto frontiers,finding that RK4 at 80 function evaluations achieves sample quality comparable to Euler at 200. Beyond reproducing known convergence rates, we report two empirical observations: (1) the Jacobian eigenvalue spectrum of the learned velocity field stiffens sharply near t=1, explaining why the adaptive Dormand-Prince solver automatically concentrates its step budget at the end of the trajectory; (2) the quality gap between low-order and high-order solvers widens for undertrained and smaller models, indicating that solver choice matters most when the model is imperfect. Code and all experiment scripts are publicly available.

Figures & tables

Method	Order	LTE	GTE	NFE/step
Euler	1	$\mathcal{O}(h^{2})$	$\mathcal{O}(h)$	1
Midpoint	2	$\mathcal{O}(h^{3})$	$\mathcal{O}(h^{2})$	2
RK4	4	$\mathcal{O}(h^{5})$	$\mathcal{O}(h^{4})$	4
DOPRI5(4)	5	$\mathcal{O}(h^{6})$	$\mathcal{O}(h^{5})$	${\sim}6$

Table 1: ODE solver properties.

Figure 1: Stability regions $\{z\in\mathbb{C}:|R(z)|\leq 1\}$ . RK4’s region extends to $\mathrm{Re}(z)\approx-2.78$ , roughly three times further than Euler’s.

Figure 2: Global error vs. step size (log-log). Dashed lines show the theoretical slopes.

Figure 3: NFE–quality Pareto frontier (moons). Numbers next to markers indicate step counts. Lower-right is better.

Figure 4: Generated samples (moons) across solver configurations.

Figure 5: Left: Jacobian eigenvalues (real part) along the trajectory; shading shows $\pm 1$ std across 200 samples. Right: condition number. The stiffening near $t{=}1$ explains why more steps are needed at the end.

Figure 6: DOPRI5 step sizes vs. time (left) and their distribution (right). The solver concentrates effort near $t{=}1$ .

Figure 7: NFE–quality Pareto frontier on MNIST (64D PCA latent).

Figure 8: MNIST samples decoded from PCA latent space. Rows: Euler (50 steps), RK4 (20 steps), DOPRI5 (adaptive).

Figure 9: SWD vs. hidden dimension. The quality gap between Euler and RK4 is widest for small networks.

Figure 10: SWD vs. training epochs. RK4’s advantage is largest for undertrained models.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 11: Euler on $y^{\prime}=-15y$ : stable (left) vs. unstable (right).

Figure 12: Solver comparison on the circles dataset.

Figure 13: ODE trajectories on the moons dataset (50 steps each). Euler paths wander; RK4 paths are nearly straight—reflecting the OT structure of the learned velocity field.

Numerical Integration

See all 24 papers

ODE-free Neural Flow Matching for One-Step Generative Modeling
Apr 7, 2026Xiao ShouOne-Step Generative Modeling Flow Matching
Baylor University
Sharpen Your Flow: Sharpness-Aware Sampling for Flow Matching
May 12, 2026Aditi Gupta, Soon Hoe Lim, Annan Yu +1Flow Matching Flow-Based Generative Modeling
Lawrence Berkeley National Laboratory · International Computer Science Institute · 1Lawrence Berkeley National Laboratory, 2International Computer Science Institute +4
Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models
Jun 16, 2025Daniel Zhengyu Huang, Jiaoyang Huang, Zhengjiang LinDiffusion Model Sampling Diffusion Sampling
Beijing International Center for Mathematical Research, Center for Machine Learning Research, Peking University, Beijing, China · Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, PA, USA · Department of Mathematics, Massachusetts Institute of Technology, 77 Massachusetts Ave, 02139 Cambridge MA, USA

Apr 7, 2026J/K move · Enter open · S save

ODE-free Neural Flow Matching for One-Step Generative Modeling

Xiao Shou

Baylor University

One-Step Generative Modeling Flow Matching Flow Map Learning Image Generation

Open paper AskPDF arXiv

Diffusion and flow matching models generate samples by learning time-dependent vector fields whose integration transports noise to data, requiring tens to hundreds of network evaluations at inference. We instead learn the transport map directly. We propose Optimal Transport Neural Flow Matching (OT-NFM), an ODE-free generative framework that parameterizes the flow map with neural flows, enabling true one-step generation with a single forward pass. We show that naive flow-map training suffers from mean collapse, where inconsistent noise-data pairings drive all outputs toward the data mean. We prove that consistent coupling is necessary for non-degenerate learning and address this using optimal transport pairings with scalable minibatch and online coupling strategies. Experiments on synthetic benchmarks and image generation tasks (MNIST and CIFAR-10) demonstrate competitive sample quality while reducing inference to a single network evaluation.

Abstract

Numerical Integration

Explore similar work

ODE-free Neural Flow Matching for One-Step Generative Modeling

Sharpen Your Flow: Sharpness-Aware Sampling for Flow Matching

Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models