Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classification

Yuga Iguchi; Paul Fearnhead

stat.MLAug 19, 2026

Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classification

Authors: Yuga Iguchi, Paul Fearnhead

Organizations: School of Mathematical Sciences, Lancaster University, Lancaster, UK

Abstract

The empirical success of diffusion models in generative modelling has motivated theoretical work, including quantitative error bounds and qualitative analyses that characterise the different phases of denoising. We bring these two areas together by studying the adaptivity of diffusion models to the structured geometry of multimodal high-dimensional data that consists of multiple clusters in $\mathbb{R}^D$ , each with its own low-dimensional structure, and inter-cluster separation depending on $D$ . We employ $K$ -mixture Gaussian distributions as a canonical framework to capture this geometry and establish two theoretical results. First, we interpret denoising as a dynamical Bayesian classifier: the mixture score is a posterior-weighted average of cluster-wise scores, and we show that, with high probability, the posterior class probabilities concentrate on a single cluster once the signal-to-noise ratio reaches the scale $Θ(\log (KD)/D)$ . Second, by separately analysing the denoising process in its mixing and cluster-commitment phases, we prove that the KL error bound depends linearly on the maximum intrinsic dimension of a cluster, up to a logarithmic factor, even when $K$ grows polynomially with $D$ . This improves on ambient-dimensional bounds and extends existing low-dimensional adaptivity analyses to multimodal distributions with heterogeneous, approximately low-rank covariances.

Figures & tables

	Phase I	Phase II
Phase Characterisation	Mixing phase	Commitment to a single cluster $j$
Signal-to-noise ratio ( $\mathrm{SNR}_{T-t}$ )	$\mathcal{O}\bigl({\log(KD)}/{D}\bigr)$	$\mathcal{O}(1)$
Posterior Weights	$W_{D}^{[k]}\geq 0,\,k\in[K]$ (Onset of posterior collapse)	$W_{D}^{[j]}\approx 1,\,W_{D}^{[k]}\approx 0,\,k\neq j$ (Posterior concentration)
Effective Geometry	Inter-cluster inference + local covariances	Single local covariance (cluster $j$ )

Table 1: Summary of the dynamical regimes of the denoising diffusion for high-dimensional

K

-clustered data.

Class	$D$	$\lambda_{\max}$	$\mathrm{tr}(\widehat{\Sigma})$	$r_{\mathrm{eff},90}$
CD4 T	$500$	$3.42$	$469.88$	$313$
	$1000$	$5.41$	$943.59$	$507$
	$2000$	$9.85$	$1807.21$	$686$
	$4000$	$16.30$	$3600.29$	$817$
	$8000$	$25.72$	$7465.22$	$903$
CD14+ Mono.	$500$	$8.41$	$485.49$	$209$

Table 2: Empirical covariance geometry for PBMC and STL-10 across ambient dimensions. For each class, we report the maximum covariance eigenvalue (

\lambda_{\max}

), the trace (

\mathrm{tr}(\widehat{\Sigma})

), and the 90% effective rank (

r_{\mathrm{eff},90}

), computed from the empirical data.

Pair	$D$	$\\|\widehat{\mu}_{i}-\widehat{\mu}_{j}\\|^{2}$	$M_{\mu}$	$\\|\widehat{\Sigma}_{i}-\widehat{\Sigma}_{j}\\|_{F}^{2}$
Dog–Cat	$3072$	$9.20$	$2.09$	$4.03{\times}10^{3}$
	$12288$	$35.10$	$4.10$	$5.98{\times}10^{4}$
	$27648$	$80.46$	$6.17$	$3.19{\times}10^{5}$
Cat–Airplane	$3072$	$232.68$	$12.81$	$1.35{\times}10^{4}$
	$12288$	$928.48$	$25.59$	$2.12{\times}10^{5}$
	$27648$	$2091.26$	$38.39$	$1.09{\times}10^{6}$

Table 3: Pairwise data geometry of the STL-10 clusters across image resolutions. For each pair and ambient dimension

D

, we report the squared mean separation, the maximum mean difference projection defined in ( 16 ), and the squared Frobenius covariance separation, computed from empirical data.

Figure 1: Posterior probability dynamics in the PBMC experiment. We construct increasing-dimensional representations by retaining the top- $D$ highly variable genes, with $D\in\{500,1000,2000,4000,8000\}$ . The posterior concentration timing shifts to lower SNR as $D$ increases.

Figure 2: Posterior commitment in the STL-10 experiment. Posterior probabilities are evaluated using the reverse trajectories, grouped by their final committed class. Solid curves show the posterior probabilities of the three classes, while dashed curves show the corresponding noised KL divergences from the target class to competing classes.

Panel of Figure 3 — Figure 3: Average empirical SNR at the posterior probability concentration over five repetitions of experiments, with the error bar showing their standard deviations. The plots show the SNR at which the lower 1 % quantile of the posterior weight among trajectories assigned to a given cluster permanently exceeds 0.99. In both STL-10 and PBMC experiments, the transition SNR exhibits an approximately $\mathcal{O}(1/D)$ decay.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Pair	$D$	$\\|\widehat{\mu}_{i}-\widehat{\mu}_{j}\\|^{2}$	$M_{\mu}$	$\\|\widehat{\Sigma}_{i}-\widehat{\Sigma}_{j}\\|_{F}^{2}$
CD4 T–CD14+ Monocyte	$500$	$31.55$	$1.48$	$1.34{\times}10^{3}$
	$1000$	$91.95$	$2.67$	$4.16{\times}10^{3}$
	$2000$	$240.40$	$4.37$	$1.55{\times}10^{4}$
	$4000$	$352.70$	$4.62$	$5.35{\times}10^{4}$
	$8000$	$509.33$	$5.68$	$1.90{\times}10^{5}$
CD14+ Monocyte–CD19+ B	$500$	$42.77$	$1.24$	$2.40{\times}10^{3}$

Appendix

Table 4: Pairwise empirical geometry of the PBMC clusters under variance-ordered feature truncation. The labels are the same as in Table 3 .

Diffusion Models

Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.

See all 1,145 papers

Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions
May 28, 2026Jingda Wu, Changxiao CaiSample Complexity Statistical Learning Theory
Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, USA.
When Diffusion Model Can Ignore Dimension: An Entropy-Based Theory
May 8, 2026Ahmad Aghapour, Erhan BayraktarDiffusion Sampling
Department of Mathematics, University of Michigan, Ann Arbor, 48109
Score Approximation for Diffusion Models on Arbitrary Low-Dimensional Structures
Jun 18, 2026Xinhe Mu, Zaijiu Shang, Zhaoqi Zhou +4Neural Network Approximation Theory Score-Based Generative Modeling
Academy of Mathematics and Systems Sciences Chinese Academy of Sciences Beijing, China · Shanghai Institute for Mathematics and Interdisciplinary Sciences Shanghai, China · Huawei Technologies Co., Ltd. Beijing, China

May 28, 2026J/K move · Enter open · S save

Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions

Jingda Wu, Changxiao Cai

Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, USA.

Sample Complexity Statistical Learning Theory Diffusion Models

Open paper AskPDF arXiv

Score-based diffusion models have demonstrated remarkable empirical success in learning high-dimensional distributions, particularly those exhibiting low-dimensional and multi-modal structures. However, theoretical understanding of their statistical efficiency remains limited. Existing theories typically rely on strong regularity assumptions, such as uniformly bounded densities or globally smooth score functions, which fail to capture such intrinsic structures. In this work, we study the sample complexity of diffusion models for learning distributions supported on a union of low-dimensional subspaces. Assuming that the data distribution within each subspace is subgaussian, we show that diffusion models require at most

\widetilde{O}(\varepsilon^{-k \vee 2})

samples to achieve

\varepsilon

error in 1-Wasserstein distance, where

k

is the intrinsic dimension. This near-optimal convergence rate depends only on the intrinsic dimension and significantly improves upon prior theoretical guarantees that suffer from the curse of dimensionality. Notably, our analysis applies to a broad collection of distributions without imposing smoothness, bounded-density, or log-concavity assumptions. Overall, our results show that diffusion models can statistically adapt to intrinsic low-dimensional structure while naturally accommodating multi-modal data, offering a rigorous theoretical justification for their success in complex high-dimensional learning tasks.

Abstract

Diffusion Models

Explore similar work

Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions

When Diffusion Model Can Ignore Dimension: An Entropy-Based Theory

Score Approximation for Diffusion Models on Arbitrary Low-Dimensional Structures