cs.LGJul 25, 2026

From Score Learning to Discretized Sampling: An End-to-End Generalization Analysis of Diffusion Models

Authors: Jinshu Huang, Yiming Jiang, Chunlin Wu

Organizations: School of Mathematical Sciences, Nankai University, Tianjin, China

Abstract

Despite the empirical success of score-based diffusion models, a complete theoretical understanding of how finite-sample learning, network parameterization, and numerical discretization jointly dictate generative quality remains underdeveloped. Existing sampling analyses often evaluate the generative performance conditional on an oracle score or a pre-specified error threshold. In this work, we establish a unified convergence and generalization framework for score-based diffusion models parameterized by practical ResNet-type architectures. We analyze the generalization and convergence properties from the practical finite-sample, discrete-time learning problem of the score function to the ideal continuous-time, population-level objective. Based on the generalization result of the learning problem of score function, we analyze the sampling process induced by the learned score function and provide an end-to-end total variation distance estimate for the generated terminal distribution. This estimate explicitly decomposes the overall generative error into four interpretable components: the truncation error of the forward process, the reverse-time discretization error, the generalization error incorporating both finite data and forward-time discretization, and the training optimization gap. Our results quantitatively characterize how the training sample size, temporal discretization grids, and optimization accuracy jointly control the final fidelity of samples generated by diffusion models.

Explore similar work

CardsList
Jun 4, 2026stat.ML

Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors

Score-based diffusion models are typically trained by minimizing the L2L^2 score matching error, and standard theoretical analyses rely on this quantity to bound the sampling discrepancy between the learned and target distributions. We show the L2L^2 score error is not the right intrinsic measure of marginal distributional quality: a learned diffusion model can incur arbitrarily large L2L^2 score error while perfectly matching the target distribution. By decomposing score errors into a gradient and a solenoidal component (a Helmholtz-Hodge decomposition), we identify the geometric reason behind this: only the gradient component enters the marginal Fokker-Planck dynamics, while the solenoidal component is structurally invisible. We make this precise in three results. First, building on the corrected geometry, we prove an impossibility result: no monotone function of the L2L^2 score error can uniformly lower bound any divergence between the learned and target distributions. Second, we derive an upper bound on the Kullback-Leibler divergence that depends only on the observable gradient component of the error, tightening the standard Girsanov bound for generic score networks, and identifying its looseness as the cost of operating on path-space rather than marginal-space dynamics. Third, we give a tractable estimator of the gradient component via a dual Sobolev identity, which is shown to empirically correlate substantially better with sample quality than the full L2L^2 error.
Naïl B. Khelifa, Richard E. Turner, Ramji Venkataramanan
Score-Based Diffusion ModelDiffusion ModelsKullback-Leibler DivergenceWasserstein Gradient FlowsGradientFokker--Planck Equation
AskPDF
Jun 13, 2025cs.LG

The Effect of Stochasticity in Score-Based Diffusion Sampling: a KL Divergence Analysis

Sampling in score-based diffusion models can be performed by solving either a reverse-time stochastic differential equation (SDE) parameterized by an arbitrary stochasticity function or a probability flow ODE, corresponding to setting this stochasticity function to zero. In this work, we investigate the effect of this stochasticity on the generation process through the evolution of Kullback-Leibler (KL) divergences, obtaining general KL divergence bounds and a novel analysis of the impact of the time-profile of the score error on model performance. For exact score functions, stochasticity has a contractive effect, decreasing KL divergence along the sampling trajectory. For approximate scores, however, a trade-off arises between correcting accumulated errors and amplifying current score errors, meaning stochasticity can either improve or degrade generation performance. Theoretical considerations indicate that the gain from stochasticity depends on the time-localization of the trained model error. We test this in experiments on both toy and benchmark data sets, also comparing the KL divergence evolution with the obtained bounds. We also present a fully analytical example, where all the relevant quantities can be computed, and the optimal stochasticity function can be characterized via an optimal control analysis.
Bernardo P. Schaeffer, Ricardo M. S. Rosa, Glauco Valle
Score-Based Diffusion ModelStochastic Differential EquationsStochasticKullback-Leibler Divergence
AskPDF
Jan 22, 2025stat.ML

Low-dimensional adaptation of diffusion models: Convergence in total variation

This paper investigates how diffusion generative models leverage (unknown) low-dimensional structure to accelerate sampling. Focusing on two mainstream samplers -- the denoising diffusion implicit model (DDIM) and the denoising diffusion probabilistic model (DDPM), we prove that their iteration complexities under exact score functions are at most the order of k/εk/\varepsilon (up to log factor), where ε\varepsilon is the precision in total variation distance and kk is some intrinsic dimension of the target distribution. We further extend these convergence guarantees to the setting in which the score functions are learned from data rather than known exactly, showing that the convergence performance degrades gracefully under suitable score estimation assumptions. We then show that these assumptions are attainable via kernel-based score estimators with finite-sample guarantees that also adapt to the low-dimensional structure. Our results apply to a broad family of target distributions without requiring smoothness or log-concavity. Our findings provide the first rigorous evidence for the adaptivity of the DDIM-type samplers to unknown low-dimensional structure, and improve over the state-of-the-art DDPM theory regarding total variation convergence.
Jiadong Liang, Zhihan Huang, Yuxin Chen
Score-Based Diffusion ModelDiffusion ModelsDenoising Diffusion Probabilistic ModelGenerative ModelsTotal VariationLow-Dimensional Structure+1
AskPDF