Organizations: Universit´e Bourgogne Europe, CNRS, Institut de Mathematiques de Bourgogne, 21000 Dijon, France. · Friedrich - Alexander - Universität Erlangen - Nürnberg, Department of Mathematics, Chair for Dynamics, Control, Machine Learning, and Numerics (Alexander von Humboldt Professorship), 91058 Erlangen, Germany. · Universidad Aut´onoma de Madrid, Departamento de Matem´aticas, 28049 Madrid, Spain. · Chair of Computational Mathematics, Fundaci´on Deusto, 48007 Bilbao, Basque Country, Spain.
Score-based diffusion models have emerged as a powerful class of generative methods, with successful applications across diverse domains. Despite their empirical success, their mathematical foundations remain only partially understood, particularly regarding the stability and consistency of the stochastic and partial differential equations underlying their dynamics. In this work, we develop a partial differential equation (PDE) framework for score-based diffusion processes associated with the heat flow of a compactly supported data measure. After establishing weak well-posedness on positive-time intervals, we derive a sharp one-sided divergence bound for the score and use it to obtain sharp uniform Lp-stability estimates for the generative Fokker--Planck equation. This bound is the Euclidean form of the Li--Yau differential inequality. Through entropy stability methods, we prove that the generated laws converge in the 2-Wasserstein distance at rate t as t→0+ to a probability measure supported on the data support. This holds for both deterministic and stochastic generation, for positive terminal densities with finite absolute entropy and second moment. In the deterministic case, almost every trajectory converges to a point of the data support; for empirical data, we further obtain a trajectory-wise square-root rate. These results clarify how exact-score dynamics concentrate on the data support without necessarily recovering the data distribution. An explicitly solvable case identifies the limiting weights and their dependence on Gaussian initialization, while a relative-entropy estimate quantifies the effect of score approximation. Together, these findings connect the stability and concentration mechanisms with score learning and early stopping, providing a mathematical perspective on the balance between imitation fidelity and generative diversity.
Figures & tables
Figure 1. Score function (left) and log-density (right) of the heat flow originating from u0=0.7δ−5+0.2δ0+0.1δ5 , shown at t=0.01,0.1,1 . The steep negative score slopes near the atoms illustrate the 1/t scale in the Li–Yau estimate. The log-density and score are evaluated using log-sum-exp and normalized Gaussian weights, respectively.
Figure 2. Score-based generation on the lemniscate dataset: comparison between the true ((A)-(B)) and empirical scores ((C)-(D)), and the effect of the choice of the stopping time tmin∈{0.1,0.01,0.001} . The true score ((A)-(B)) corresponds to the solution of the heat equation, given by the convolution with the Gaussian heat kernel, computed by means of numerical quadrature, while the empirical score ((C)-(D)) is obtained from a finite sample of data points via the explicit expression ( 3.8 ). In all experiments, we set the backward diffusion parameter to be ϵ=0.2 . The displayed sample region encompasses 95% of the 10,000 reverse-time samples, reducing the effect of extreme stochastic trajectories.
Figure 3. Trajectories of the score-based ODE ( ϵ=0 ) with u0=21(δ(−1,0)+δ(1,0)) . Initial points with x<0 converge to (−1,0) , those with x>0 converge to (1,0) , and points with x=0 converge to the barycenter (0,0) .
Figure 4. Voronoi diagram and γ –Voronoi cores for five points in R2 . When γ=0 , we recover the classical Voronoi diagram.
Score-based diffusion models are typically formulated using continuous-time stochastic differential equations and measure-theoretic stochastic calculus. In this paper, we develop a hyperfinite formulation of score-based generative modeling within the framework of Nonstandard Analysis. Starting from an internal diffusion process on a hyperfinite grid, we derive the associated infinitesimal generator and establish its correspondence with the classical Fokker--Planck equation. We then obtain a hyperfinite backward-mean identity that yields the reverse-time drift and provides a constructive derivation of the reverse-time SDE. Building on these results, we show that minimization of an internal score-matching objective recovers the score function required by the reverse-time dynamics, thereby connecting score estimation with generative sampling directly at the hyperfinite level. Under suitable assumptions, we further derive a hyperfinite Girsanov formula and establish a relationship between likelihood optimization and Fisher-divergence objectives. Finally, we analyze the second-order consistency of the hyperfinite dynamics and show that the leading correction term depends explicitly on the fourth moment of the increment distribution, with the Gaussian value κ=3 eliminating the leading dispersion contribution. Taken together, these results provide a unified hyperfinite framework for diffusion-based generative modeling--while laying foundations for further extensions--that links discrete grid dynamics, reverse-time diffusion, score matching, and likelihood-based formulations within a common nonstandard setting.
Sunder Ram Krishnan
Kerala School of Mathematics, Kozhikode 673571, Kerala, India.
Sampling in score-based diffusion models can be performed by solving either a reverse-time stochastic differential equation (SDE) parameterized by an arbitrary stochasticity function or a probability flow ODE, corresponding to setting this stochasticity function to zero. In this work, we investigate the effect of this stochasticity on the generation process through the evolution of Kullback-Leibler (KL) divergences, obtaining general KL divergence bounds and a novel analysis of the impact of the time-profile of the score error on model performance. For exact score functions, stochasticity has a contractive effect, decreasing KL divergence along the sampling trajectory. For approximate scores, however, a trade-off arises between correcting accumulated errors and amplifying current score errors, meaning stochasticity can either improve or degrade generation performance. Theoretical considerations indicate that the gain from stochasticity depends on the time-localization of the trained model error. We test this in experiments on both toy and benchmark data sets, also comparing the KL divergence evolution with the obtained bounds. We also present a fully analytical example, where all the relevant quantities can be computed, and the optimal stochasticity function can be characterized via an optimal control analysis.
Bernardo P. Schaeffer, Ricardo M. S. Rosa, Glauco Valle
Instituto de Matem´atica, Universidade Federal do Rio de Janeiro
Probability-flow ordinary differential equations (PF-ODEs) are widely used as deterministic samplers for score-based diffusion models. Their usual justification is that the Fokker--Planck equation of a diffusion can be rewritten as a continuity equation driven by the score function of the forward diffusion. This identity does not, however, guarantee that the resulting velocity field generates a well-posed flow. We provide theoretical insights into the design of such deterministic samplers for generative models based on diffusions and reflected diffusions. We identify sufficient conditions for a regular Lagrangian PF-ODE flow to exist; reverse sampling and invertibility require two-sided divergence control. For learned scores, sampler stability is controlled by an unweighted velocity error, exposing a mismatch with density-weighted score matching and motivating architectural control of Jacobians, divergence, growth, and compression. Under the manifold hypothesis, positive-time regularization justifies an early-stopped PF-ODE while constants deteriorate near the data endpoint; an explicit sphere example shows that the exact deterministic flow becomes singular as the noise level vanishes even though the diffusion marginals remain well defined. These theoretical insights translate into concrete design principles for stable, invertible, and constraint-preserving diffusion samplers. We illustrate the practical relevance of these design principles using controlled numerical experiments.