Breakdown of Local Denoising as Semantic Speciation

Guangkuo Liu; Mert Okyay; Yifan F. Zhang; Fangjun Hu; Rahul Nandkishore; Xun Gao

cs.LGSep 29, 2026

Breakdown of Local Denoising as Semantic Speciation

Authors: Guangkuo Liu, Mert Okyay, Yifan F. Zhang, Fangjun Hu, Rahul Nandkishore, Xun Gao

Organizations: JILA and Department of Physics University of Colorado Boulder Boulder, CO 80309, USA · CTQM and Department of Physics University of Colorado Boulder Boulder, CO 80309, USA · Department of Electrical and Computer Engineering Princeton University Princeton, NJ 08544, USA · QuEra Computing Inc. 1284 Soldiers Field Road, Boston, MA 02135, USA

Abstract

The dynamics of generative models exhibit two apparently distinct temporal windows: a speciation window, in which a sample commits to a semantic class, and a nonlocality window, in which local context windows become insufficient for generation. Motivated by evidence of their near-concurrence in a variety of frontier models, we investigate their relationship through the spatial distribution of semantic information. Under a "common cause" hypothesis, we prove that the nonlocality window must lie in the speciation window. This hypothesis postulates that semantic labels explain a fraction of the correlations between distant tokens, a condition that is natural for many real datasets. We further give conditions under which both windows shrink to a single limiting time as system size grows, defining a "phase transition", and verify this behavior analytically in Gaussian mixtures. Together, these results identify conditions under which semantic information explains the concurrence of speciation and nonlocality, connecting two complementary perspectives on the emergence of semantic structure in generative modeling.

Figures & tables

Panel of Figure 1 — Figure 1: Semantic speciation locates the window in which local denoising breaks down. Denoising trajectory is from $t=1$ (right) to $t=0$ (left). The upper panel depicts a semantic speciation from white noise to cat or dog for local region (blue) and global region (magenta); the corresponding thresholds define a speciation window (black vertical lines). The noisy images in the window depict generation times in which the semantic identity is visible in the whole image, but invisible in a local patch (blue box). In the lower row, $A$ is the patch to denoise (green), $B$ its minimal surrounding context (blue) for below-threshold denoising, and $C$ the rest of the image (magenta). The green curve depicts $I_{t}(A;C\mid B)$ whose peak characterizes a breakdown of exact denoising using local context alone. Under the common-cause hypothesis, Theorem 1 places the threshold-defined nonlocality window inside the speciation window.

Figure 2: Semantic conditioning reduces the average local–global prediction gap in SD3. A smaller conditional gap means that restricting distant image-token interactions changes the model prediction less when semantic information is supplied. This is consistent with the common-cause interpretation: the description explains part of the shared image structure that would otherwise require distant context.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Label–reconstruction similarity remains high at early diffusion times and then falls toward zero. The sharp drop occurs first for the

64\times 64

crop, then for the

128\times 128

crop, and last for the full image.

Panel of Figure 4 — Figure 4: Exact numerical evaluation of the Gaussian-mixture information measures with specific values given in E.3 . (a) Mutual information curves for the four spatial regions. The cliff diagnoses the symmetry-breaking phase transition. Inset: zoomed-in region near critical window. (b) Local and global conditional information gaps $I_{t}(S;A|B),I_{t}(S;A|BC)$ , and their difference which gives conditional interaction information $I_{t}(A;C;S|B)$ . The peak diagnoses the non-locality transition.

Generative Modeling

Generative modeling learns data distributions to create new samples, from images and text to molecules, simulations, and structured records.

See all 509 papers

Concurrence of Symmetry Breaking and Nonlocality Phase Transitions in Diffusion Models
May 6, 2026Yifan F. Zhang, Fangjun Hu, Guangkuo Liu +2Diffusion Transformer Symmetry Breaking
Department of Electrical and Computer Engineering Princeton University Princeton, NJ 08544, USA · QuEra Computing Inc. 1284 Soldiers Field Road, Boston, MA 02135, USA · JILA and Department of Physics University of Colorado Boulder Boulder, CO 80309, USA +1
A Theory of Speciation in Generative Diffusion Models on Compact Riemannian Manifolds
Aug 24, 2026Alessio Marta, Paola CausinDiffusion Models Score-Based Generative Modeling
Department of Mathematics, University of Milan, via Saldini 50, 20133 Milan, Italy
Steering Dynamical Regimes of Diffusion Models by Breaking Detailed Balance
Feb 17, 2026Haiqi Lu, Ying TangDiffusion Models Diffusion Sampling
Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China · Yingcai Honor College, University of Electronic Science and Technology of China, Chengdu 611731, China · School of Physics, University of Electronic Science and Technology of China, Chengdu 611731, China +2

May 6, 2026J/K move · Enter open · S save

Concurrence of Symmetry Breaking and Nonlocality Phase Transitions in Diffusion Models

Yifan F. Zhang, Fangjun Hu, Guangkuo Liu, Mert Okyay+1

Department of Electrical and Computer Engineering Princeton University Princeton, NJ 08544, USA · QuEra Computing Inc. 1284 Soldiers Field Road, Boston, MA 02135, USA · JILA and Department of Physics University of Colorado Boulder Boulder, CO 80309, USA+1

Diffusion Transformer Symmetry Breaking Phase Transitions Diffusion Models

Open paper AskPDF arXiv

Diffusion models undergo a phase transition in a critical time window during generation dynamics, with two complementary diagnoses of criticality. The symmetry breaking picture views the critical window as when trajectories bifurcate into different semantic minima of the energy landscape, whereas the nonlocality picture views the critical window as when local denoising fails. We study whether two notions of such phase transitions are concurrent in modern diffusion transformers. By evaluating the dynamics and outcomes of the generation trajectory, we observe a near-simultaneous occurrence of the non-locality and symmetry breaking critical times. Our work is the first to unify the two notions of phase transitions in practice: it provides a concrete diagnostic for when and why diffusion models rely on conditioning and global denoising, enabling principled evaluation of model efficiency and guiding the design of architectures and sampling schemes that avoid unnecessary computation.

Abstract

Generative Modeling

Explore similar work

Concurrence of Symmetry Breaking and Nonlocality Phase Transitions in Diffusion Models

A Theory of Speciation in Generative Diffusion Models on Compact Riemannian Manifolds

Steering Dynamical Regimes of Diffusion Models by Breaking Detailed Balance