Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data

Saptarshi Chakraborty; Quentin Berthet; Peter L. Bartlett

stat.MLMar 4, 2026

Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data

Authors: Saptarshi Chakraborty, Quentin Berthet, Peter L. Bartlett

Organizations: Department of Statistics, University of Michigan · Google DeepMind · Department of Statistics, University of California, Berkeley · Department of Electrical Engineering and Computer Sciences, UC Berkeley

Abstract

Despite the remarkable empirical success of score-based diffusion models, their statistical guarantees remain underdeveloped. Existing analyses often provide pessimistic convergence rates that do not reflect the intrinsic low-dimensional structure common in real data, such as that arising in natural images. In this work, we study the statistical convergence of score-based diffusion models for learning an unknown distribution $μ$ from finitely many samples. Under mild regularity conditions on the forward diffusion process and the data distribution, we derive finite-sample error bounds on the learned generative distribution, measured in the Wasserstein- $p$ distance. Unlike prior results, our guarantees hold for all $p \ge 1$ and require only a finite-moment assumption on $μ$ , without compact-support, manifold, or smooth-density conditions. Specifically, given $n$ i.i.d.\ samples from $μ$ with finite $q$ -th moment and appropriately chosen network architectures, hyperparameters, and discretization schemes, we show that the expected Wasserstein- $p$ error between the learned distribution $\hatμ$ and $μ$ scales as $\mathbb{E}\, \mathbb{W}_p(\hatμ,μ) = \widetilde{O}\!\left(n^{-1 / d^\ast_{p,q}(μ)}\right),$ where $d^\ast_{p,q}(μ)$ is the $(p,q)$ -Wasserstein dimension of $μ$ . Our results demonstrate that diffusion models naturally adapt to the intrinsic geometry of data and mitigate the curse of dimensionality, since the convergence rate depends on $d^\ast_{p,q}(μ)$ rather than the ambient dimension. Moreover, our theory conceptually bridges the analysis of diffusion models with that of GANs and the sharp minimax rates established in optimal transport. The proposed $(p,q)$ -Wasserstein dimension also extends the notion of classical Wasserstein dimension to distributions with unbounded support, which may be of independent theoretical interest.

Figures & tables

Panel of Figure 1 — Figure 1: Average generalization error (in terms of FID scores) for different values of $n$ for DDPM. The error bars denote the standard deviation out of $10$ replications.

Explore similar work

May 28, 2026

Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions

Score-based diffusion models have demonstrated remarkable empirical success in learning high-dimensional distributions, particularly those exhibiting low-dimensional and multi-modal structures. However, theoretical understanding of their statistical efficiency remains limited. Existing theories typically rely on strong regularity assumptions, such as uniformly bounded densities or globally smooth score functions, which fail to capture such intrinsic structures. In this work, we study the sample complexity of diffusion models for learning distributions supported on a union of low-dimensional subspaces. Assuming that the data distribution within each subspace is subgaussian, we show that diffusion models require at most

\widetilde{O}(\varepsilon^{-k \vee 2})

samples to achieve

\varepsilon

error in 1-Wasserstein distance, where

k

is the intrinsic dimension. This near-optimal convergence rate depends only on the intrinsic dimension and significantly improves upon prior theoretical guarantees that suffer from the curse of dimensionality. Notably, our analysis applies to a broad collection of distributions without imposing smoothness, bounded-density, or log-concavity assumptions. Overall, our results show that diffusion models can statistically adapt to intrinsic low-dimensional structure while naturally accommodating multi-modal data, offering a rigorous theoretical justification for their success in complex high-dimensional learning tasks.

Jingda Wu, Changxiao Cai

Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, USA.

Sample Complexity Statistical Learning Theory Diffusion Models

Ask PDF

Jun 18, 2026

Score Approximation for Diffusion Models on Arbitrary Low-Dimensional Structures

Score-based diffusion models have achieved remarkable empirical success, motivating extensive theoretical work to establish their foundations. However, existing complexity bounds for score approximation, a vital step in diffusion modeling, rely on rigid constraints such as Lipschitz continuous scores or lower bounded densities. This severely limits their applicability to real-world perceptual data, where singularities, sharp boundaries, and disjoint clusters routinely violate such restrictive assumptions. We bridge this gap between theory and practice, presenting the first universal score approximation theorem applicable to any compactly supported distribution in

\mathbb{R}^n

. Using a novel discretization technique that directly models the underlying distribution, we prove that the neural network complexity is governed by the support's upper Minkowski dimension

d

rather than the ambient dimension

n

. Furthermore, by leveraging the inherent smoothing of Gaussian kernels, we show that even for irregular, fractal distributions, an

ε

approximation error can be achieved with

\mathcal{O}(ε^{-{d/(M+1)}})

local Taylor modules each at size of

\mathcal{O}(n^M)

, an

ε

-scaling rate previously achieved only for distributions with

(M+1)

-Hölder smoothness. Thus, we reveal a possible mechanism by which score-based diffusion models represent non-smooth data distributions.

Xinhe Mu, Zaijiu Shang, Zhaoqi Zhou +4

Academy of Mathematics and Systems Sciences Chinese Academy of Sciences Beijing, China · Shanghai Institute for Mathematics and Interdisciplinary Sciences Shanghai, China · Huawei Technologies Co., Ltd. Beijing, China

Neural Network Approximation Theory Score-Based Generative Modeling Diffusion Models

Ask PDF

Sep 23, 2026

Robustness of Diffusion Models under Distribution Shift

Score-based diffusion models are increasingly considered in settings where the underlying data distribution may differ from the training distribution, yet existing theoretical guarantees largely focus on the no-shift setting. In this work, we study robust score estimation under Wasserstein perturbations of a reference distribution. For the Ornstein--Uhlenbeck diffusion, we show that robust estimation decomposes into two fundamental components: the statistical cost of learning the reference distribution and the intrinsic cost of distribution shift. The latter scales quadratically with the Wasserstein radius, and this dependence is minimax optimal. We construct an explicit finite-sample estimator achieving the resulting robust minimax rate without knowing the shift radius. When the reference distribution lies on an unknown low-dimensional subspace, the statistical term adapts to the intrinsic dimension while the shift cost remains unchanged. Finally, we show that the same decomposition governs positive-time reverse sampling and obtain matching minimax guarantees in KL divergence. Together, these results characterize how finite data, intrinsic dimension, and distribution shift affect the robustness of score-based diffusion models.

Wei Luo, Neil K. Chada, Shijie Zhang +1

Department of Mathematical Sciences, Tsinghua University · Department of Mathematics, City University of Hong Kong, Hong Kong SAR · Artificial Intelligence Research Institute, Shenzhen MSU-BIT University +1

Distribution Shift Robustness Diffusion Sampling Score-Based Generative Modeling Robust Estimation

Ask PDF