Low-NFE generative ODE sampling is constrained not only by solver order, but also by how a fixed budget of network evaluations is distributed along the flow. We develop a density-level backward-error framework that identifies the component of a solver's modified-equation defect that changes the evolving marginal, while excluding solenoidal trajectory distortion that is invisible in Wasserstein tangent geometry. The resulting residual induces a solver-order-aware compute-allocation rule and, through gauge freedom, an objective for selecting numerically favorable velocity representatives of the same density path. This separates our approach from schedule search driven by trajectory or endpoint surrogates and from higher-order solver design. Exact Gaussian and mixture models establish the mechanism, including cases where curvature-based allocation fails and continuum-optimal transport is not discretization-optimal. On a pretrained CIFAR-10 EDM probability-flow ODE, density-aware allocation consistently improves fixed-update Euler sampling in the low-NFE regime. Matched-seed convergence to a dense Euler reference confirms that the method accelerates approximation of the same high-NFE flow rather than changing the asymptotic sampler.
Figures & tables
Figure 1: Matched-seed CIFAR-10 samples at 8 NFE for the three Euler grids. Every row uses the same network and update rule; only timestep placement changes. The visual comparison therefore matches the fixed-solver claim tested by the first block of Table 1 .
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A.1: Offline profile used by the Euler allocation: the Galerkin estimate of the density-active residual, the empirical terminal-propagation factor, and their product. The schedule is obtained from quantiles of the square root of the propagated profile in the solver variable σ .
Figure A.2: Matched-seed convergence to a 1000 -step Euler reference. The proposed schedule is closest to the dense reference at every tested budget, while the three curves approach one another as NFE grows. The reference is numerical, not an exact ODE solution.
Schedule
FID- 5 k
Generation time (s)
Euler/Ours
6.64496
5187.9
Euler/log- σ
6.64442
5112.5
Euler/EDM- ρ
6.64618
5289.6
Appendix
Table A.1: High-NFE Euler check on 5000 CIFAR-10 samples. Every row uses 1000 Euler evaluations; FID computation time is excluded.
Figure A.3: FID-50k versus wall-clock generation time. The Euler schedules lie at nearly identical cost for each NFE. DPM++ and EDM Heun reach lower FID by using stronger update formulas, with a small implementation-dependent overhead for DPM++.
Figure A.4: Matched-seed CIFAR-10 samples at 16 NFE for the three Euler schedules. As in the main-text figure, the comparison changes only timestep placement.
Figure B.5: Exact Gaussian gauge check. Left: normalized density-active modified-equation bias, with zeros at the predicted gauges α=±1 . Right: finite-step Euler covariance error for the same continuum density path; the cancelling skew gauge reduces the error at every displayed step count.
Quantity
Value
Profile grid
31 points in [0,1] , 1536 particles per time
Sampling budget
2048 particles per evaluation seed
Validation/test split
validation selects γ ; test seeds are disjoint
MMD
RBF MMD with deterministic truncation to 512 points
Sliced Wasserstein
128 random directions
Schedule family
uniform or proxy quantiles with γ∈{0.25,0.5,0.75,1}
Appendix
Table C.3: Primary hard-GMM evaluation protocol.
Figure C.6: Terminal MMD at fixed γ=1/2 . Every nonuniform schedule improves on uniform Euler. The close values of the three proxies reflect their shared late-time concentration on this benchmark, rather than equivalence of the underlying error signals.
Figure D.7: MMD under the unified protocol. Density-active schedules remain competitive with embedded Euler–Heun refinement and Jo–Choi velocity-variation resampling. Fair-NFE Heun illustrates the complementary benefit of higher-order integration as the budget supports more second-order intervals.
Figure D.8: Exact score–divergence, particle-only dual, robust directional- K -weighted, and Jo–Choi velocity-variation profiles, normalized independently.
Grid
Test MMD
Local surrogate
d(π,πmarg)
Uniform
0.0400±0.0094
1.184
0.0858
Density quantile
0.0363±0.0175
1.000
0.1412
Trajectory oracle
0.0357±0.0153
1.040
0.1272
Marginal RFF oracle
0.0381±0.0107
2.377
0
Appendix
Table E.7: Analytic GMM grid comparison at N=8 . MMD is mean ± standard error over three evaluation seeds. The density-quantile grid exactly minimizes the local surrogate, while the terminal and trajectory oracles optimize different objectives.
Claim
Prediction
Observed check
Local marginal order
radial order 2 ; cancelled even-dimensional skew order 3 ; odd-dimensional obstruction order 2
at N=16 : uniform 0.18544 , residual-only 0.19615 , K -weighted 0.18514 , oracle 0.18509
Profile robustness
the continuous χ2 identity is recovered as the quantile grid is refined
correlation increases from 0.891 at N=8 to 0.995 at N=64 ; median relative discrepancy decreases from 0.751 to 0.125
Appendix
Table E.8: Compact theorem-linked checks. The propagation row reports terminal W2 at N=16 ; the robustness row compares the coarsest and finest grids.
Sampling from Flow Matching generative models requires solving an ordinary differential equation (ODE) whose computational cost is dominated by neural network forward passes. We derive four classical ODE solvers -- Euler, Explicit Midpoint, Classical Runge-Kutta (RK4), and Dormand-Prince 5(4) -- from first principles via Taylor expansion, implement them from scratch in PyTorch, and systematically benchmark their efficiency on Conditional Flow Matching tasks ranging from 2D toy distributions to MNIST digits. On the quantitative side, we use sliced Wasserstein distance to construct NFE-quality Pareto frontiers,finding that RK4 at 80 function evaluations achieves sample quality comparable to Euler at 200. Beyond reproducing known convergence rates, we report two empirical observations: (1) the Jacobian eigenvalue spectrum of the learned velocity field stiffens sharply near t=1, explaining why the adaptive Dormand-Prince solver automatically concentrates its step budget at the end of the trajectory; (2) the quality gap between low-order and high-order solvers widens for undertrained and smaller models, indicating that solver choice matters most when the model is imperfect. Code and all experiment scripts are publicly available.
Flow matching models generate samples by numerically integrating a learned velocity field, with each integration step requiring a neural network evaluation. Fast generation therefore requires using a small fixed evaluation budget effectively: the key question is not only how to integrate the flow, but where the sampler should spend its steps. We propose SharpEuler, a training-free sampler that profiles a pretrained model offline by estimating where the learned velocity field changes most rapidly along calibration trajectories. This finite-difference estimate defines a solver-aware sharpness profile, which is smoothed and converted by a quantile transform into a timestep grid for any desired inference budget. At test time, sampling remains ordinary Euler integration with the same number of model evaluations as a uniform schedule. We justify SharpEuler using three principles: a numerical principle identifying trajectory acceleration as the leading source of Euler discretization error, a variational principle deriving sharpness-based power-law timestep densities, and a statistical guarantee showing that the finite-sample calibrated sampler is stable at the terminal distribution level. Our experiments show that SharpEuler improves sample quality at fixed budgets, reducing inter-mode leakage and increasing mode coverage.
Aditi Gupta, Soon Hoe Lim, Annan Yu +1
Lawrence Berkeley National Laboratory · International Computer Science Institute · 1Lawrence Berkeley National Laboratory, 2International Computer Science Institute +4
Diffusion and flow generative models sample by integrating a learned ODE, but high quality still requires many sequential model evaluations. Solver learning reduces this cost by adapting scalar coefficients, timesteps, or both, while keeping the backbone model fixed. In this work, we identify a structural bottleneck in this update family: each step remains span-limited. Since the scalar-coefficient update lies in the span of buffered velocity evaluations, it can fit only the in-span component while leaving any out-of-span residual unreachable by scalar recombination alone. We propose SpanLift, a lightweight neural solver that augments scalar-coefficient updates with a spatial residual operator. SpanLift keeps a fixed base solver as an in-span prior and learns a spatial residual operator over the state and velocity buffer. The operator is trained by endpoint teacher matching, preserves the pretrained backbone, and adds no model NFEs. Empirically, the learned correction transfers across base solvers and is predominantly out-of-span. Across pixel-space diffusion, latent flow matching, and precipitation nowcasting, SpanLift achieves state-of-the-art few-step sampling. With only 3 NFE, it improves CIFAR-10 FID from 8.16 to 5.69 and ImageNet FID from 17.37 to 11.83.
Sihyeon Kim, Seunghun Lee, Vikas Singh +1
Korea University · KAIST · University of Wisconsin–Madison