Emergence of Distortions in High-Dimensional Guided Diffusion Models
Organizations: Department of Computing Sciences, Bocconi University, Milan, Italy · Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands · Bocconi Institute for Data Science and Analytics, Bocconi University, Milan, Italy
Abstract
Classifier-free guidance (CFG) is the de facto standard for conditional sampling in diffusion models, yet it often reduces sample diversity. Using tools from statistical physics, we analyze the emergence of generative distortions induced by CFG, namely the mismatch between the CFG sampling distribution and the true conditional distribution. We study this phenomenon in analytically tractable settings with exact score functions, characterizing its dependence on data dimensionality and the number of classes. For high-dimensional Gaussian mixtures, we use dynamic mean-field theory to show that distortions persist when the number of classes scales exponentially with the data dimension, whereas they vanish in the sub-exponential regime due to a dynamical phase transition. Moreover we found that, in the infinite-class limit, distortions remain unavoidable regardless of dimensionality because of the increasing density of classes. Finally, we show that standard CFG schedules cannot prevent variance shrinkage, and we propose a theoretically grounded guidance schedule incorporating a negative-guidance window that improves both class separability and sample diversity in real-world latent diffusion models.
Figures & tables
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Ref. | Gaussian | GMM | GMM | General & | Finite | Class | Diversity | |
|---|---|---|---|---|---|---|---|---|
| (linear score) | scaling with | Statistical Theory | Alignment | |||||
| [ 31 ] , Li et al. (2025) | x | x | x | |||||
| [ 37 ] , Pavasovic et al. (2025) | x | x | x | x | ||||
| [ 24 ] , Jin et al. (2025) | x | x | x | x | ||||
| [ 30 ] , Li & Jiao (2025) | x | |||||||
| [ 10 ] , Chidambaram et al. (2024) | x | x | x | x |
Classifier-Free Guidance
Classifier-free guidance (CFG) steers conditional generation by extrapolating between conditional and unconditional model predictions during sampling.
See all 68 papers