Density-Level Backward Error for Low-NFE Generative Flows

Ewsey Obzherin; Valentin Khrulkov

cs.LGOct 21, 2026

Density-Level Backward Error for Low-NFE Generative Flows

Authors: Ewsey Obzherin, Valentin Khrulkov

Organizations: FusionBrain Lab · MIRIAI

Abstract

Low-NFE generative ODE sampling is constrained not only by solver order, but also by how a fixed budget of network evaluations is distributed along the flow. We develop a density-level backward-error framework that identifies the component of a solver's modified-equation defect that changes the evolving marginal, while excluding solenoidal trajectory distortion that is invisible in Wasserstein tangent geometry. The resulting residual induces a solver-order-aware compute-allocation rule and, through gauge freedom, an objective for selecting numerically favorable velocity representatives of the same density path. This separates our approach from schedule search driven by trajectory or endpoint surrogates and from higher-order solver design. Exact Gaussian and mixture models establish the mechanism, including cases where curvature-based allocation fails and continuum-optimal transport is not discretization-optimal. On a pretrained CIFAR-10 EDM probability-flow ODE, density-aware allocation consistently improves fixed-update Euler sampling in the low-NFE regime. Matched-seed convergence to a dense Euler reference confirms that the method accelerates approximation of the same high-NFE flow rather than changing the asymptotic sampler.

Figures & tables

Panel of Figure 1 — Figure 1: Matched-seed CIFAR-10 samples at 8 NFE for the three Euler grids. Every row uses the same network and update rule; only timestep placement changes. The visual comparison therefore matches the fixed-solver claim tested by the first block of Table 1 .

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A.1: Offline profile used by the Euler allocation: the Galerkin estimate of the density-active residual, the empirical terminal-propagation factor, and their product. The schedule is obtained from quantiles of the square root of the propagated profile in the solver variable

\sigma

Figure A.2: Matched-seed convergence to a

1000

-step Euler reference. The proposed schedule is closest to the dense reference at every tested budget, while the three curves approach one another as NFE grows. The reference is numerical, not an exact ODE solution.

Schedule	FID- $5$ k	Generation time (s)
Euler/Ours	6.64496	5187.9
Euler/log- $\sigma$	6.64442	5112.5
Euler/EDM- $\rho$	6.64618	5289.6

Appendix

Table A.1: High-NFE Euler check on

5000

CIFAR-10 samples. Every row uses

1000

Euler evaluations; FID computation time is excluded.

Figure A.3: FID-50k versus wall-clock generation time. The Euler schedules lie at nearly identical cost for each NFE. DPM++ and EDM Heun reach lower FID by using stronger update formulas, with a small implementation-dependent overhead for DPM++.

Panel of Figure 6 — Figure A.4: Matched-seed CIFAR-10 samples at 16 NFE for the three Euler schedules. As in the main-text figure, the comparison changes only timestep placement.

Figure B.5: Exact Gaussian gauge check. Left: normalized density-active modified-equation bias, with zeros at the predicted gauges

\alpha=\pm 1

. Right: finite-step Euler covariance error for the same continuum density path; the cancelling skew gauge reduces the error at every displayed step count.

Quantity	Value
Profile grid	$31$ points in $[0,1]$ , $1536$ particles per time
Sampling budget	$2048$ particles per evaluation seed
Validation/test split	validation selects $\gamma$ ; test seeds are disjoint
MMD	RBF MMD with deterministic truncation to $512$ points
Sliced Wasserstein	$128$ random directions
Schedule family	uniform or proxy quantiles with $\gamma\in\{0.25,0.5,0.75,1\}$

Appendix

Table C.3: Primary hard-GMM evaluation protocol.

Figure C.6: Terminal MMD at fixed $\gamma=1/2$ . Every nonuniform schedule improves on uniform Euler. The close values of the three proxies reflect their shared late-time concentration on this benchmark, rather than equivalence of the underlying error signals.

Figure D.7: MMD under the unified protocol. Density-active schedules remain competitive with embedded Euler–Heun refinement and Jo–Choi velocity-variation resampling. Fair-NFE Heun illustrates the complementary benefit of higher-order integration as the budget supports more second-order intervals.

Figure D.8: Exact score–divergence, particle-only dual, robust directional-

K

-weighted, and Jo–Choi velocity-variation profiles, normalized independently.

Grid	Test MMD	Local surrogate	$d(\pi,\pi_{\rm marg})$
Uniform	$0.0400\pm 0.0094$	$1.184$	$0.0858$
Density quantile	$0.0363\pm 0.0175$	$\mathbf{1.000}$	$0.1412$
Trajectory oracle	$0.0357\pm 0.0153$	$1.040$	$0.1272$
Marginal RFF oracle	$0.0381\pm 0.0107$	$2.377$	$0$

Appendix

Table E.7: Analytic GMM grid comparison at

N=8

. MMD is mean

\pm

standard error over three evaluation seeds. The density-quantile grid exactly minimizes the local surrogate, while the terminal and trajectory oracles optimize different objectives.

Claim	Prediction	Observed check
Local marginal order	radial order $2$ ; cancelled even-dimensional skew order $3$ ; odd-dimensional obstruction order $2$	fitted tail slopes $2.000$ , $2.998$ – $3.002$ , and $2.000$ – $2.00004$ , respectively
Propagation weighting	$K(1,t)R(t)$ approaches the terminal-grid oracle	at $N=16$ : uniform $0.18544$ , residual-only $0.19615$ , $K$ -weighted $0.18514$ , oracle $0.18509$
Profile robustness	the continuous $\chi^{2}$ identity is recovered as the quantile grid is refined	correlation increases from $0.891$ at $N=8$ to $0.995$ at $N=64$ ; median relative discrepancy decreases from $0.751$ to $0.125$

Appendix

Table E.8: Compact theorem-linked checks. The propagation row reports terminal

W_{2}

N=16

; the robustness row compares the coarsest and finest grids.

Diffusion Sampling

Diffusion sampling studies how to turn a trained diffusion model into high-quality samples efficiently and reliably.

See all 264 papers

From Euler to Dormand-Prince: ODE Solvers for Flow Matching Generative Models
Apr 4, 2026Hao XiaoNumerical Integration Flow Matching
ATLAS AI Lab
Sharpen Your Flow: Sharpness-Aware Sampling for Flow Matching
May 12, 2026Aditi Gupta, Soon Hoe Lim, Annan Yu +1Flow Matching Flow-Based Generative Modeling
Lawrence Berkeley National Laboratory · International Computer Science Institute · 1Lawrence Berkeley National Laboratory, 2International Computer Science Institute +4
Learning to Solve Generative ODEs Beyond the Linear Span
Jun 7, 2026Sihyeon Kim, Seunghun Lee, Vikas Singh +1Few-Step Diffusion Sampling Flow-Based Generative Modeling
Korea University · KAIST · University of Wisconsin–Madison

Apr 4, 2026J/K move · Enter open · S save

From Euler to Dormand-Prince: ODE Solvers for Flow Matching Generative Models

Hao Xiao

ATLAS AI Lab

Numerical Integration Flow Matching Conditional Flow Matching

Open paper AskPDF arXiv

Sampling from Flow Matching generative models requires solving an ordinary differential equation (ODE) whose computational cost is dominated by neural network forward passes. We derive four classical ODE solvers -- Euler, Explicit Midpoint, Classical Runge-Kutta (RK4), and Dormand-Prince 5(4) -- from first principles via Taylor expansion, implement them from scratch in PyTorch, and systematically benchmark their efficiency on Conditional Flow Matching tasks ranging from 2D toy distributions to MNIST digits. On the quantitative side, we use sliced Wasserstein distance to construct NFE-quality Pareto frontiers,finding that RK4 at 80 function evaluations achieves sample quality comparable to Euler at 200. Beyond reproducing known convergence rates, we report two empirical observations: (1) the Jacobian eigenvalue spectrum of the learned velocity field stiffens sharply near t=1, explaining why the adaptive Dormand-Prince solver automatically concentrates its step budget at the end of the trajectory; (2) the quality gap between low-order and high-order solvers widens for undertrained and smaller models, indicating that solver choice matters most when the model is imperfect. Code and all experiment scripts are publicly available.

Abstract

Diffusion Sampling

Explore similar work

From Euler to Dormand-Prince: ODE Solvers for Flow Matching Generative Models

Sharpen Your Flow: Sharpness-Aware Sampling for Flow Matching

Learning to Solve Generative ODEs Beyond the Linear Span