Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classification

Yuga Iguchi; Paul Fearnhead

stat.MLAug 19, 2026

Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classification

Authors: Yuga Iguchi, Paul Fearnhead

Organizations: School of Mathematical Sciences, Lancaster University, Lancaster, UK

Abstract

The empirical success of diffusion models in generative modelling has motivated theoretical work, including quantitative error bounds and qualitative analyses that characterise the different phases of denoising. We bring these two areas together by studying the adaptivity of diffusion models to the structured geometry of multimodal high-dimensional data that consists of multiple clusters in $\mathbb{R}^D$ , each with its own low-dimensional structure, and inter-cluster separation depending on $D$ . We employ $K$ -mixture Gaussian distributions as a canonical framework to capture this geometry and establish two theoretical results. First, we interpret denoising as a dynamical Bayesian classifier: the mixture score is a posterior-weighted average of cluster-wise scores, and we show that, with high probability, the posterior class probabilities concentrate on a single cluster once the signal-to-noise ratio reaches the scale $Θ(\log (KD)/D)$ . Second, by separately analysing the denoising process in its mixing and cluster-commitment phases, we prove that the KL error bound depends linearly on the maximum intrinsic dimension of a cluster, up to a logarithmic factor, even when $K$ grows polynomially with $D$ . This improves on ambient-dimensional bounds and extends existing low-dimensional adaptivity analyses to multimodal distributions with heterogeneous, approximately low-rank covariances.

Figures & tables

	Phase I	Phase II
Phase Characterisation	Mixing phase	Commitment to a single cluster $j$
Signal-to-noise ratio ( $\mathrm{SNR}_{T-t}$ )	$\mathcal{O}\bigl({\log(KD)}/{D}\bigr)$	$\mathcal{O}(1)$
Posterior Weights	$W_{D}^{[k]}\geq 0,\,k\in[K]$ (Onset of posterior collapse)	$W_{D}^{[j]}\approx 1,\,W_{D}^{[k]}\approx 0,\,k\neq j$ (Posterior concentration)
Effective Geometry	Inter-cluster inference + local covariances	Single local covariance (cluster $j$ )

Table 1: Summary of the dynamical regimes of the denoising diffusion for high-dimensional

K

-clustered data.

Class	$D$	$\lambda_{\max}$	$\mathrm{tr}(\widehat{\Sigma})$	$r_{\mathrm{eff},90}$
CD4 T	$500$	$3.42$	$469.88$	$313$
	$1000$	$5.41$	$943.59$	$507$
	$2000$	$9.85$	$1807.21$	$686$
	$4000$	$16.30$	$3600.29$	$817$
	$8000$	$25.72$	$7465.22$	$903$
CD14+ Mono.	$500$	$8.41$	$485.49$	$209$

Table 2: Empirical covariance geometry for PBMC and STL-10 across ambient dimensions. For each class, we report the maximum covariance eigenvalue (

\lambda_{\max}

), the trace (

\mathrm{tr}(\widehat{\Sigma})

), and the 90% effective rank (

r_{\mathrm{eff},90}

), computed from the empirical data.

Pair	$D$	$\\|\widehat{\mu}_{i}-\widehat{\mu}_{j}\\|^{2}$	$M_{\mu}$	$\\|\widehat{\Sigma}_{i}-\widehat{\Sigma}_{j}\\|_{F}^{2}$
Dog–Cat	$3072$	$9.20$	$2.09$	$4.03{\times}10^{3}$
	$12288$	$35.10$	$4.10$	$5.98{\times}10^{4}$
	$27648$	$80.46$	$6.17$	$3.19{\times}10^{5}$
Cat–Airplane	$3072$	$232.68$	$12.81$	$1.35{\times}10^{4}$
	$12288$	$928.48$	$25.59$	$2.12{\times}10^{5}$
	$27648$	$2091.26$	$38.39$	$1.09{\times}10^{6}$

Table 3: Pairwise data geometry of the STL-10 clusters across image resolutions. For each pair and ambient dimension

D

, we report the squared mean separation, the maximum mean difference projection defined in ( 16 ), and the squared Frobenius covariance separation, computed from empirical data.

Figure 1: Posterior probability dynamics in the PBMC experiment. We construct increasing-dimensional representations by retaining the top- $D$ highly variable genes, with $D\in\{500,1000,2000,4000,8000\}$ . The posterior concentration timing shifts to lower SNR as $D$ increases.

Figure 2: Posterior commitment in the STL-10 experiment. Posterior probabilities are evaluated using the reverse trajectories, grouped by their final committed class. Solid curves show the posterior probabilities of the three classes, while dashed curves show the corresponding noised KL divergences from the target class to competing classes.

Panel of Figure 3 — Figure 3: Average empirical SNR at the posterior probability concentration over five repetitions of experiments, with the error bar showing their standard deviations. The plots show the SNR at which the lower 1 % quantile of the posterior weight among trajectories assigned to a given cluster permanently exceeds 0.99. In both STL-10 and PBMC experiments, the transition SNR exhibits an approximately $\mathcal{O}(1/D)$ decay.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Pair	$D$	$\\|\widehat{\mu}_{i}-\widehat{\mu}_{j}\\|^{2}$	$M_{\mu}$	$\\|\widehat{\Sigma}_{i}-\widehat{\Sigma}_{j}\\|_{F}^{2}$
CD4 T–CD14+ Monocyte	$500$	$31.55$	$1.48$	$1.34{\times}10^{3}$
	$1000$	$91.95$	$2.67$	$4.16{\times}10^{3}$
	$2000$	$240.40$	$4.37$	$1.55{\times}10^{4}$
	$4000$	$352.70$	$4.62$	$5.35{\times}10^{4}$
	$8000$	$509.33$	$5.68$	$1.90{\times}10^{5}$
CD14+ Monocyte–CD19+ B	$500$	$42.77$	$1.24$	$2.40{\times}10^{3}$

Appendix

Table 4: Pairwise empirical geometry of the PBMC clusters under variance-ordered feature truncation. The labels are the same as in Table 3 .

Diffusion Models

Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.

See all 1,145 papers

May 28, 2026

Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions

Score-based diffusion models have demonstrated remarkable empirical success in learning high-dimensional distributions, particularly those exhibiting low-dimensional and multi-modal structures. However, theoretical understanding of their statistical efficiency remains limited. Existing theories typically rely on strong regularity assumptions, such as uniformly bounded densities or globally smooth score functions, which fail to capture such intrinsic structures. In this work, we study the sample complexity of diffusion models for learning distributions supported on a union of low-dimensional subspaces. Assuming that the data distribution within each subspace is subgaussian, we show that diffusion models require at most

\widetilde{O}(\varepsilon^{-k \vee 2})

samples to achieve

\varepsilon

error in 1-Wasserstein distance, where

k

is the intrinsic dimension. This near-optimal convergence rate depends only on the intrinsic dimension and significantly improves upon prior theoretical guarantees that suffer from the curse of dimensionality. Notably, our analysis applies to a broad collection of distributions without imposing smoothness, bounded-density, or log-concavity assumptions. Overall, our results show that diffusion models can statistically adapt to intrinsic low-dimensional structure while naturally accommodating multi-modal data, offering a rigorous theoretical justification for their success in complex high-dimensional learning tasks.

Jingda Wu, Changxiao Cai

Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, USA.

Sample Complexity Statistical Learning Theory Diffusion Models

Ask PDF

May 8, 2026

When Diffusion Model Can Ignore Dimension: An Entropy-Based Theory

Diffusion models perform remarkably well on high-dimensional data such as images, often using only a modest number of reverse-time steps. Despite this practical success, existing convergence theory does not fully explain why such samplers remain efficient in high dimensions. Many prior KL guarantees bound the discretization error in terms of the ambient dimension, while other improved results replace this dependence using intrinsic-dimensional or geometric structure assumptions. In this work, we develop an alternative information-theoretic perspective on diffusion sampler convergence. We prove that, for Gaussian mixture targets, the discretization error is controlled by the Shannon entropy of the latent mixture component rather than by the ambient dimension. Consequently, the leading step complexity scales linearly with latent entropy and depends only logarithmically on the second moment of the data. Our analysis also extends to discrete target distributions, where the relevant complexity is the entropy of the target rather than the dimension of the embedding space. These results suggest that diffusion sampling can remain efficient in high-dimensional spaces when the data distribution admits a compact latent representation, as is widely believed to be the case for natural images.

Ahmad Aghapour, Erhan Bayraktar

Department of Mathematics, University of Michigan, Ann Arbor, 48109

Diffusion Sampling

Ask PDF

Jun 18, 2026

Score Approximation for Diffusion Models on Arbitrary Low-Dimensional Structures

Score-based diffusion models have achieved remarkable empirical success, motivating extensive theoretical work to establish their foundations. However, existing complexity bounds for score approximation, a vital step in diffusion modeling, rely on rigid constraints such as Lipschitz continuous scores or lower bounded densities. This severely limits their applicability to real-world perceptual data, where singularities, sharp boundaries, and disjoint clusters routinely violate such restrictive assumptions. We bridge this gap between theory and practice, presenting the first universal score approximation theorem applicable to any compactly supported distribution in

\mathbb{R}^n

. Using a novel discretization technique that directly models the underlying distribution, we prove that the neural network complexity is governed by the support's upper Minkowski dimension

d

rather than the ambient dimension

n

. Furthermore, by leveraging the inherent smoothing of Gaussian kernels, we show that even for irregular, fractal distributions, an

ε

approximation error can be achieved with

\mathcal{O}(ε^{-{d/(M+1)}})

local Taylor modules each at size of

\mathcal{O}(n^M)

, an

ε

-scaling rate previously achieved only for distributions with

(M+1)

-Hölder smoothness. Thus, we reveal a possible mechanism by which score-based diffusion models represent non-smooth data distributions.

Xinhe Mu, Zaijiu Shang, Zhaoqi Zhou +4

Academy of Mathematics and Systems Sciences Chinese Academy of Sciences Beijing, China · Shanghai Institute for Mathematics and Interdisciplinary Sciences Shanghai, China · Huawei Technologies Co., Ltd. Beijing, China

Neural Network Approximation Theory Score-Based Generative Modeling Diffusion Models

Ask PDF

Abstract

Diffusion Models

Explore similar work

Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions

When Diffusion Model Can Ignore Dimension: An Entropy-Based Theory

Score Approximation for Diffusion Models on Arbitrary Low-Dimensional Structures