Probability flow ODEs in score-based and reflected diffusion models

Rama CONT

stat.MLOct 2, 2026

Probability flow ODEs in score-based and reflected diffusion models

Authors: Rama CONT

Organizations: Mathematical Institute, University of Oxford

Abstract

Probability-flow ordinary differential equations (PF-ODEs) are widely used as deterministic samplers for score-based diffusion models. Their usual justification is that the Fokker--Planck equation of a diffusion can be rewritten as a continuity equation driven by the score function of the forward diffusion. This identity does not, however, guarantee that the resulting velocity field generates a well-posed flow. We provide theoretical insights into the design of such deterministic samplers for generative models based on diffusions and reflected diffusions. We identify sufficient conditions for a regular Lagrangian PF-ODE flow to exist; reverse sampling and invertibility require two-sided divergence control. For learned scores, sampler stability is controlled by an unweighted velocity error, exposing a mismatch with density-weighted score matching and motivating architectural control of Jacobians, divergence, growth, and compression. Under the manifold hypothesis, positive-time regularization justifies an early-stopped PF-ODE while constants deteriorate near the data endpoint; an explicit sphere example shows that the exact deterministic flow becomes singular as the noise level vanishes even though the diffusion marginals remain well defined. These theoretical insights translate into concrete design principles for stable, invertible, and constraint-preserving diffusion samplers. We illustrate the practical relevance of these design principles using controlled numerical experiments.

Figures & tables

Panel of Figure 1 — Figure 1: Low-density diagnostics and a controlled regularization test. (a) Endpoint trajectory errors for the score perturbations with $E_{\theta}^{2}=10^{-3}$ ; (b) $90$ th percentile of unweighted velocity error over $5000$ noisy-score regressions decreases with Jacobian penalty (matched DSM error not imposed).

Requirement/ failure mode	Analytic diagnostic	Design consequence
PF-ODE existence	Sobolev/BV regularity, growth, $[\nabla\!\cdot v_{\theta}]^{-}$ control	Regularize Jacobian/divergence and network growth; do not rely on score loss alone.
Reverse/invertible flow	Both $[\nabla\!\cdot v_{\theta}]^{-}$ and $[\nabla\!\cdot v_{\theta}]^{+}$	Stronger regularization for deterministic decoding, round trips, and likelihood evaluation.
Under the manifold hypothesis	Normal score/compression constants diverge as $t\downarrow 0$	Train/sample on $[\delta,T]$ and use a terminal denoising or limiting step; tune $\delta$ jointly with schedule.
Low-density regions	$E_{v}$ depends on an unweighted norm; conversion from score loss uses $\int 1/p_{t}$	Add localization or low-density diagnostics/regularization; validate trajectories, not only DSM loss.
Reflected boundary	$v_{\theta}\cdot\nu\approx 0$ ; generally $s_{\theta}\cdot\nu\neq 0$	Enforce co-normal relation for the drift–score field; avoid generic zero-normal-score constraints.
Boundary degeneracy	Loss of positivity can permit large compression near $\partial\Omega$	Maintain positive-time regularization; monitor boundary density, normal trace, and compression.

Table 1: Theory-to-design translation for deterministic diffusion samplers.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

$\lambda_{J}$	$\|\widehat{a}_{\lambda_{J}}\|$	$E_{\theta}^{2}$	$E_{v}$
$0$	$1.8091$	$2.361\times 10^{-3}$	$0.4308$
$10^{-3}$	$0.34648$	$8.659\times 10^{-5}$	$0.08251$
$10^{-2}$	$0.14741$	$1.567\times 10^{-5}$	$0.03510$
$10^{-1}$	$0.02847$	$5.846\times 10^{-7}$	$0.006779$

Appendix

Table 2: Tail-mode regularization across

5000

independent training sets. Each column reports a separate empirical

90

th percentile. The penalty is spatially normalized as in ( 26 ), whereas

E_{v}

and

E_{\theta}^{2}

retain the integrated definitions in ( 14 ) and ( 15 ).

Diagnostic	Central defect ( $x_{0}=0$ )	Tail defect ( $x_{0}=4$ )
Score amplitude $A$	$0.09560$	$1.17745$
Population DSM error $E_{\theta}^{2}$	$0.001$	$0.001$
Unweighted velocity error $E_{v}$	$0.02276$	$0.28039$
Excess Jacobian error $J_{\rm err}$	$0.009618$	$1.45902$
Mean grid trajectory error	$0.001854$	$0.02953$
Maximum grid trajectory error	$0.03788$	$0.28037$

Appendix

Table 3: Matched population DSM error for central and tail score perturbations. Trajectory errors are evaluated at

t=\delta

; the mean is an arithmetic mean over the uniform spatial grid, not an expectation under

p_{T}

. The Jacobian error is the unnormalized space–time integral

J_{\rm err}

Panel of Figure 2 — Figure 2: Exact-score endpoint diagnostics for the flat-manifold VE model, with $T=1$ and $\delta\in[10^{-5},10^{-1}]$ . Decreasing the normal residual improves fidelity to the data subspace while increasing the sensitivity of the inverse map. Normal volume contraction is amplified by codimension $c$ ; these curves follow the exact expressions in ( 30 ).

Explore similar work

Jun 16, 2025

Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models

Diffusion probabilistic models generate samples by learning to reverse a noise-injection process that transforms data into noise. A key development is the reformulation of the reverse sampling process as a deterministic probability flow ordinary differential equation (ODE), which allows for efficient sampling using high-order numerical solvers. Unlike traditional time integrator analysis, the accuracy of this sampling procedure depends not only on numerical integration errors but also on the approximation quality and regularity of the learned score function, as well as their interaction. In this work, we present a rigorous convergence analysis of deterministic samplers derived from probability flow ODEs for general forward processes with arbitrary variance schedules. Specifically, we develop and analyze

p

-th order (exponential) Runge-Kutta schemes, under the practical assumption that the first and second derivatives of the learned score function are bounded. We prove that the total variation distance between the generated and target distributions can be bounded as \begin{align*} O\bigl(d^{\frac{7}{4}}\varepsilon_{\text{score}}^{\frac{1}{2}} +d(dH_{\max})^p\bigr), \end{align*} where

\varepsilon^2_{\text{score}}

denotes the

L^2

error in the score function approximation,

d

is the data dimension, and

H_{\max}

represents the maximum solver step size. Numerical experiments on benchmark datasets further confirm that the derivatives of the learned score function are bounded in practice.

Daniel Zhengyu Huang, Jiaoyang Huang, Zhengjiang Lin

Beijing International Center for Mathematical Research, Center for Machine Learning Research, Peking University, Beijing, China · Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, PA, USA · Department of Mathematics, Massachusetts Institute of Technology, 77 Massachusetts Ave, 02139 Cambridge MA, USA

Diffusion Model Sampling Diffusion Sampling

Ask PDF

May 7, 2026

A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models

We survey continuous-time generative modeling methods based on transporting a simple reference distribution to a data distribution via stochastic or deterministic dynamics. We present a unified framework in which diffusion models, score-based generative models, and flow matching are instances of learning a time-dependent vector field that induces a family of marginals

(ρ_t)_{t \in [0,1]}

governed by continuity and Fokker-Planck equations. Such a unified theory is timely because these methods are converging methodologically, yet fragmented notation and competing derivations continue to obscure their shared structure and the practical tradeoffs governing sampling, stability, and computation. Within this framework, we (i) derive reverse-time sampling for diffusion and score-based models as controlled stochastic dynamics, (ii) show that the probability flow ODE yields identical marginals and connects diffusion to likelihood-based normalizing flows, and (iii) interpret flow matching as direct regression of the velocity field under a chosen interpolation, clarifying when it coincides with or differs from score-based training. We compare objectives, sampling schemes, and discretization errors under unified notation, discuss connections to Schrodinger bridges and entropic optimal transport, and summarize theoretical guarantees and open problems on approximation, stability, and scalability.

Aditya Ranganath, Mukesh Singhal

Center for Applied Scientific Computing Lawrence Livermore National Laboratory Livermore, CA 94551, USA · Department of Electrical Engineering University of California Merced, CA 95343, USA

Flow Matching Generative Modeling Flow-Based Generative Modeling Score-Based Generative Modeling Diffusion Models

Ask PDF

Aug 19, 2026

A Plug-in Interpretation of Conditioning in Score-Based Diffusion Models

We propose a conditioning mechanism for diffusion models based on multi-speed joint diffusion of the target and the condition. The mechanism learns an unconditional joint score network and enforces conditioning at inference via a plug-in correction term. The plug-in term separates the conditioning contribution from the learned unconditional dynamics, offering a transparent view of how the condition steers generation of the target distribution. Building on this, we derive explicit conditional reverse-time SDEs and approximate probability-flow ODEs, enabling principled and directly comparable conditional samplers. To reduce the induced ODE--SDE discrepancy, we introduce a log-Fokker--Planck residual regularization that improves ODE sampling quality. Experiments on conditional image generation tasks demonstrate competitive performance and support the effectiveness of the plug-in conditioning view. Additional ODE--SDE comparison experiments show that the log-Fokker--Planck residual regularization improves deterministic ODE sampling.

Libo Chen, Souvik Ghosh, Teo Deveney +2

University of Bath, Bath, UK · International Institute of Information Technology Hyderabad, India

Conditional Diffusion Models Diffusion Model Sampling Score-Based Generative Modeling

Ask PDF