Steering Dynamical Regimes of Diffusion Models by Breaking Detailed Balance
Authors: Haiqi Lu, Ying Tang
Organizations: Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China · Yingcai Honor College, University of Electronic Science and Technology of China, Chengdu 611731, China · School of Physics, University of Electronic Science and Technology of China, Chengdu 611731, China · Key Laboratory of Quantum Physics and Photonic Quantum Information, Ministry of Education, University of Electronic Science and Technology of China, Chengdu 611731, China · Non-classical Information Science Basic Discipline Research Center of Sichuan Province, University of Electronic Science and Technology of China, Chengdu 611731, China
We show that deliberately breaking detailed balance in generative diffusion processes can accelerate the reverse process without changing the stationary distribution. Considering the Ornstein--Uhlenbeck process, we decompose the dynamics into a symmetric component and a non-reversible anti-symmetric component that generates rotational probability currents. We then construct an exponentially optimal non-reversible perturbation that improves the long-time relaxation rate while preserving the stationary target. We analyze how such non-reversible control reshapes the macroscopic dynamical regimes of the phase transitions recently identified in generative diffusion models. We derive a general criterion for the speciation time and show that suitable non-reversible perturbations can accelerate speciation. In contrast, the collapse transition is governed by a trace-controlled phase-space contraction mechanism that is fixed by the symmetric component, and the corresponding collapse time remains unchanged under anti-symmetric perturbations. Numerical experiments on Gaussian mixture models support these findings.
Figures & tables
Figure 1: Acceleration of speciation by non-reversible drift in Gaussian mixture. Left: Dynamics under Lelièvre’s exponentially optimal drift. The labels Q1 , Q2 , and Q3 correspond to different choices of the auxiliary spectrum used in the construction algorithm. Q1 uses a standard linear spectrum with λ∈[1,d] , Q2 uses a shifted high-frequency spectrum with λ∈[d+1,2d] , and Q3 uses a geometric spectrum. Right: Dynamics under a simple Q strategy defined as a dense anti-symmetric matrix where all upper-triangular elements are set to a constant magnitude q , specifically Qij=q for i<j . The theoretical speciation times tS in legends are computed by solving λmin(M(tS))=0 . Notably, the simple strategy achieves faster speciation than the “exponentially optimal” designs. This arises because Lelièvre’s optimality applies to the asymptotic decay rate when t→∞ , whereas the speciation event occurs at short times where transient non-normal effects dominate the dynamics.
Figure 2: Validation of the theoretical scaling for speciation dynamics. The probability ϕ(t) is plotted against the rescaled time t/tS using the same experimental configuration and drift matrices defined in Figure 1. Left: Results for Lelièvre’s exponentially optimal drift. Right: Results for the simple Q strategy. The theoretical tS matches perfectly with the numerical results obtained from the Gaussian mixture simulations.
Figure 3: Invariance of the collapse time to non-reversible perturbations. The normalized excess entropy density is plotted against time for different values of Q . The collapse time tC is defined as the time at which the curves lift off from zero. Left: For non-diagonal U , while the shapes of the transition curves differ, their onset point remains the same regardless of the value of Q , validating that the collapse time is robust to non-reversible perturbations. Right:For diagonal U , the entropy curves for different Q values are indistinguishable, showing that both the collapse time and subsequent dynamics are independent of the non-reversible component.
Diffusion models undergo a phase transition in a critical time window during generation dynamics, with two complementary diagnoses of criticality. The symmetry breaking picture views the critical window as when trajectories bifurcate into different semantic minima of the energy landscape, whereas the nonlocality picture views the critical window as when local denoising fails. We study whether two notions of such phase transitions are concurrent in modern diffusion transformers. By evaluating the dynamics and outcomes of the generation trajectory, we observe a near-simultaneous occurrence of the non-locality and symmetry breaking critical times. Our work is the first to unify the two notions of phase transitions in practice: it provides a concrete diagnostic for when and why diffusion models rely on conditioning and global denoising, enabling principled evaluation of model efficiency and guiding the design of architectures and sampling schemes that avoid unnecessary computation.
Yifan F. Zhang, Fangjun Hu, Guangkuo Liu +2
Department of Electrical and Computer Engineering Princeton University Princeton, NJ 08544, USA · QuEra Computing Inc. 1284 Soldiers Field Road, Boston, MA 02135, USA · JILA and Department of Physics University of Colorado Boulder Boulder, CO 80309, USA +1
We discuss three simple connections between generative diffusion and optimal transport. First, the joint law of the data and of their Ornstein-Uhlenbeck noising is, at every time, the solution of an entropic optimal-transport problem, at a temperature set by the noise level. Backward diffusion can consequently be read as an annealing which gradually decreases the entropic regularization. Second, we study the speciation transition--how the modes of the data distribution emerge during generation--and compare how it happens in diffusion and flow matching with respect to optimal transport. Using Gaussian mixtures as high-dimensional example, we show that the dynamical instability through which diffusion and flow matching separate the modes becomes, for optimal transport, an exponentially thin boundary layer at the beginning of generation, which is rounded by entropic regularization. Third, considering a generic forward dynamics with a general confining potential, we show that the backward process driven by the score also satisfies a drift-based variational principle whose minimal cost is the entropy difference between initial and final distributions.
Giulio Biroli, Marc Mézard
Laboratoire de Physique de l’Ecole Normale Supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris-Diderot, Sorbonne Paris Cité, Paris, France · Department of Computing Sciences, Bocconi University, Milano, Italy
Speciation in generative diffusion models denotes the emergence of distinct stable branches during denoising, through which initially undifferentiated trajectories progressively commit to different data classes. In this work we develop an intrinsic theory of speciation for diffusion models supported on compact Riemannian manifolds: the aim is to go beyond existing theoretical descriptions, which usually identify speciation with a symmetric pitchfork bifurcation and assume to work in a large-dimensional space. We characterize speciation by bifurcations of the critical points of the evolving probability density. A spectral heat-kernel representation makes explicit the role of the manifold geometry, while Poincaré-Hopf and Morse theory impose global constraints on the number and type of score equilibria and reveal topologically-imposed geometrical modes. For mixtures of heat kernels, we prove that generic speciation events have a one-dimensional critical kernel and admit an A2 fold normal form; pitchforks and simultaneous multidirectional transitions arise from nongeneric symmetric configurations. We derive geometry-dependent estimates of speciation times for bimodal mixtures and Riemannian regular simplices. We further establish structural stability of nondegenerate folds under score perturbations and show that the first-order time shift is determined solely by the component of the score error along the critical direction. The theory is illustrated on the sphere using mixtures of von Mises-Fisher distributions, where pitchfork and saddle-node bifurcations, topological modes, and hierarchical multiple speciations are observed. Finally, a chart-based intrinsic score-learning scheme based on neural networks contrasts the theoretically predicted transitions on prototypal and more complex datasets.
Alessio Marta, Paola Causin
Department of Mathematics, University of Milan, via Saldini 50, 20133 Milan, Italy