Probability flow ODEs in score-based and reflected diffusion models

Rama CONT

stat.MLOct 2, 2026

Probability flow ODEs in score-based and reflected diffusion models

Authors: Rama CONT

Organizations: Mathematical Institute, University of Oxford

Abstract

Probability-flow ordinary differential equations (PF-ODEs) are widely used as deterministic samplers for score-based diffusion models. Their usual justification is that the Fokker--Planck equation of a diffusion can be rewritten as a continuity equation driven by the score function of the forward diffusion. This identity does not, however, guarantee that the resulting velocity field generates a well-posed flow. We provide theoretical insights into the design of such deterministic samplers for generative models based on diffusions and reflected diffusions. We identify sufficient conditions for a regular Lagrangian PF-ODE flow to exist; reverse sampling and invertibility require two-sided divergence control. For learned scores, sampler stability is controlled by an unweighted velocity error, exposing a mismatch with density-weighted score matching and motivating architectural control of Jacobians, divergence, growth, and compression. Under the manifold hypothesis, positive-time regularization justifies an early-stopped PF-ODE while constants deteriorate near the data endpoint; an explicit sphere example shows that the exact deterministic flow becomes singular as the noise level vanishes even though the diffusion marginals remain well defined. These theoretical insights translate into concrete design principles for stable, invertible, and constraint-preserving diffusion samplers. We illustrate the practical relevance of these design principles using controlled numerical experiments.

Figures & tables

Panel of Figure 1 — Figure 1: Low-density diagnostics and a controlled regularization test. (a) Endpoint trajectory errors for the score perturbations with $E_{\theta}^{2}=10^{-3}$ ; (b) $90$ th percentile of unweighted velocity error over $5000$ noisy-score regressions decreases with Jacobian penalty (matched DSM error not imposed).

Requirement/ failure mode	Analytic diagnostic	Design consequence
PF-ODE existence	Sobolev/BV regularity, growth, $[\nabla\!\cdot v_{\theta}]^{-}$ control	Regularize Jacobian/divergence and network growth; do not rely on score loss alone.
Reverse/invertible flow	Both $[\nabla\!\cdot v_{\theta}]^{-}$ and $[\nabla\!\cdot v_{\theta}]^{+}$	Stronger regularization for deterministic decoding, round trips, and likelihood evaluation.
Under the manifold hypothesis	Normal score/compression constants diverge as $t\downarrow 0$	Train/sample on $[\delta,T]$ and use a terminal denoising or limiting step; tune $\delta$ jointly with schedule.
Low-density regions	$E_{v}$ depends on an unweighted norm; conversion from score loss uses $\int 1/p_{t}$	Add localization or low-density diagnostics/regularization; validate trajectories, not only DSM loss.
Reflected boundary	$v_{\theta}\cdot\nu\approx 0$ ; generally $s_{\theta}\cdot\nu\neq 0$	Enforce co-normal relation for the drift–score field; avoid generic zero-normal-score constraints.
Boundary degeneracy	Loss of positivity can permit large compression near $\partial\Omega$	Maintain positive-time regularization; monitor boundary density, normal trace, and compression.

Table 1: Theory-to-design translation for deterministic diffusion samplers.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

$\lambda_{J}$	$\|\widehat{a}_{\lambda_{J}}\|$	$E_{\theta}^{2}$	$E_{v}$
$0$	$1.8091$	$2.361\times 10^{-3}$	$0.4308$
$10^{-3}$	$0.34648$	$8.659\times 10^{-5}$	$0.08251$
$10^{-2}$	$0.14741$	$1.567\times 10^{-5}$	$0.03510$
$10^{-1}$	$0.02847$	$5.846\times 10^{-7}$	$0.006779$

Appendix

Table 2: Tail-mode regularization across

5000

independent training sets. Each column reports a separate empirical

90

th percentile. The penalty is spatially normalized as in ( 26 ), whereas

E_{v}

and

E_{\theta}^{2}

retain the integrated definitions in ( 14 ) and ( 15 ).

Diagnostic	Central defect ( $x_{0}=0$ )	Tail defect ( $x_{0}=4$ )
Score amplitude $A$	$0.09560$	$1.17745$
Population DSM error $E_{\theta}^{2}$	$0.001$	$0.001$
Unweighted velocity error $E_{v}$	$0.02276$	$0.28039$
Excess Jacobian error $J_{\rm err}$	$0.009618$	$1.45902$
Mean grid trajectory error	$0.001854$	$0.02953$
Maximum grid trajectory error	$0.03788$	$0.28037$

Appendix

Table 3: Matched population DSM error for central and tail score perturbations. Trajectory errors are evaluated at

t=\delta

; the mean is an arithmetic mean over the uniform spatial grid, not an expectation under

p_{T}

. The Jacobian error is the unnormalized space–time integral

J_{\rm err}

Panel of Figure 2 — Figure 2: Exact-score endpoint diagnostics for the flat-manifold VE model, with $T=1$ and $\delta\in[10^{-5},10^{-1}]$ . Decreasing the normal residual improves fidelity to the data subspace while increasing the sensitivity of the inverse map. Normal volume contraction is amplified by codimension $c$ ; these curves follow the exact expressions in ( 30 ).

Explore similar work

Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models
Jun 16, 2025Daniel Zhengyu Huang, Jiaoyang Huang, Zhengjiang LinDiffusion Model Sampling Diffusion Sampling
Beijing International Center for Mathematical Research, Center for Machine Learning Research, Peking University, Beijing, China · Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, PA, USA · Department of Mathematics, Massachusetts Institute of Technology, 77 Massachusetts Ave, 02139 Cambridge MA, USA
A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models
May 7, 2026Aditya Ranganath, Mukesh SinghalFlow Matching Generative Modeling
Center for Applied Scientific Computing Lawrence Livermore National Laboratory Livermore, CA 94551, USA · Department of Electrical Engineering University of California Merced, CA 95343, USA
A Plug-in Interpretation of Conditioning in Score-Based Diffusion Models
Aug 19, 2026Libo Chen, Souvik Ghosh, Teo Deveney +2Conditional Diffusion Models Diffusion Model Sampling
University of Bath, Bath, UK · International Institute of Information Technology Hyderabad, India

Jun 16, 2025J/K move · Enter open · S save

Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models

Daniel Zhengyu Huang, Jiaoyang Huang, Zhengjiang Lin

Beijing International Center for Mathematical Research, Center for Machine Learning Research, Peking University, Beijing, China · Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, PA, USA · Department of Mathematics, Massachusetts Institute of Technology, 77 Massachusetts Ave, 02139 Cambridge MA, USA

Diffusion Model Sampling Diffusion Sampling

Open paper AskPDF arXiv

Diffusion probabilistic models generate samples by learning to reverse a noise-injection process that transforms data into noise. A key development is the reformulation of the reverse sampling process as a deterministic probability flow ordinary differential equation (ODE), which allows for efficient sampling using high-order numerical solvers. Unlike traditional time integrator analysis, the accuracy of this sampling procedure depends not only on numerical integration errors but also on the approximation quality and regularity of the learned score function, as well as their interaction. In this work, we present a rigorous convergence analysis of deterministic samplers derived from probability flow ODEs for general forward processes with arbitrary variance schedules. Specifically, we develop and analyze

p

-th order (exponential) Runge-Kutta schemes, under the practical assumption that the first and second derivatives of the learned score function are bounded. We prove that the total variation distance between the generated and target distributions can be bounded as \begin{align*} O\bigl(d^{\frac{7}{4}}\varepsilon_{\text{score}}^{\frac{1}{2}} +d(dH_{\max})^p\bigr), \end{align*} where

\varepsilon^2_{\text{score}}

denotes the

L^2

error in the score function approximation,

d

is the data dimension, and

H_{\max}

represents the maximum solver step size. Numerical experiments on benchmark datasets further confirm that the derivatives of the learned score function are bounded in practice.

Abstract

Explore similar work

Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models

A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models

A Plug-in Interpretation of Conditioning in Score-Based Diffusion Models