stat.MLNov 10, 2026

An Eyring-Kramers Formula for Singular Wells

Authors: Yingli Wang, Lingjiong Zhu

Organizations: School of Mathematical Sciences, Fudan University, Shanghai, People’s Republic of China · Department of Mathematics, Florida State University, Tallahassee, Florida, United States of America

Abstract

Understanding the persistence of learned behaviors in AI safety motivates an investigation of the dynamical counterpart to the equilibrium geometry studied in singular learning theory. We analyze this persistence through transitions between singular wells. We establish an Eyring-Kramers formula for a low temperature Langevin diffusion between two compact connected zero components of a real-analytic potential. The wells may be singular and nonisolated, and the unique minimizing gate is a nondegenerate index-one saddle, with gates understood as inclusion-minimal sets meeting every minimizing path. A boundary-gate construction shows that uniqueness of this inclusion-minimal gate is equivalent to every minimizing path passing through the saddle and one such path meeting the communication level only there. Compact semianalytic topology provides a separating function away from a fixed saddle neighborhood; analytic Morse coordinates give matching capacity bounds by first sending the temperature to zero and then shrinking that neighborhood. An anchored energy estimate and a small-ball potential-theoretic identity give uniformity for deterministic starting points. The leading constant is expressed in terms of the sublevel-volume coefficient, a Gamma factor, and the saddle Hessian.

Explore similar work

Jul 8, 2026

Avoiding unsafe sets when training with Langevin Dynamics

Training a model with noisy gradient descent can be idealized as overdamped Langevin dynamics, and a natural safety question is to bound the probability νt(AH)=P(Qt∈AH)ν_t(\mathcal{A}_H) = \mathbb{P}(Q_t \in \mathcal{A}_H) that the trajectory lies in a designated failure region AH\mathcal{A}_H. We study this for a smooth, strongly convex loss in dd dimensions, with AH\mathcal{A}_H separated from the minimizer by an energy gap. At the end of training, the equilibrium mass π(AH)π(\mathcal{A}_H) is exponentially small in dd, with a complementary energy-barrier rate when the noise is small. Along the trajectory, a shape-free bound νt(AH)≤π(AH)(1+χ02/π(AH) e−mt)ν_t(\mathcal{A}_H) \le π(\mathcal{A}_H)(1 + \sqrt{χ_0^2/π(\mathcal{A}_H)}\,e^{-mt}) shows the in-set probability relaxes to (twice) the static value after a burn-in of order dd, using only the global spectral gap mm. A worked Ornstein-Uhlenbeck example shows this burn-in is necessary: an angular slice of the equilibrium shell can transiently swell by a factor exponential in dd, though its equilibrium mass is tiny. To rule this out we introduce a local relaxation rate, defined through the spectral measure of the region's centered indicator rather than a Dirichlet-form Rayleigh quotient. For geometrically isolated regions this rate exceeds the global one, shrinking the burn-in, and with a maximum-principle ceiling it caps the trajectory probability uniformly in time. Strong convexity sets how fast training relaxes, but the shape of the unsafe set decides whether the trajectory bulges through it on the way to equilibrium.
Apr 5, 2026

Non-Equilibrium Stochastic Dynamics as a Unified Framework for Insight and Repetitive Learning: A Kramers Escape Approach to Continual Learning

Continual learning in artificial neural networks is fundamentally limited by the stability--plasticity dilemma: systems that retain prior knowledge tend to resist acquiring new knowledge, and vice versa. Existing approaches, most notably elastic weight consolidation~(EWC), address this empirically without a physical account of why plasticity eventually collapses as tasks accumulate. Separately, the distinction between sudden insight and gradual skill acquisition through repetitive practice has lacked a unified theoretical description. Here, we show that both problems admit a common resolution within non-equilibrium statistical physics. We model the state of a learning system as a particle evolving under Langevin dynamics on a double-well energy landscape, with the noise amplitude governed by a time-dependent effective temperature T(t)T(t). The probability density obeys a Fokker--Planck equation, and transitions between metastable states are governed by the Kramers escape rate k=(ω0ωb/2π) e−ΔE/Tk = (ω_0ω_b/2π)\,e^{-ΔE/T}. We make two contributions. First, we identify the EWC penalty term as an energy barrier whose height grows linearly with the number of accumulated tasks, yielding an exponential collapse of the transition rate predicted analytically and confirmed numerically. Second, we show that insight and repetitive learning correspond to two qualitatively distinct temperature protocols within the same Fokker--Planck equation: insight events produce transient spikes in T(t)T(t) that drive rapid barrier crossing, whereas repetitive practice operates at a modestly elevated but fixed temperature, achieving transitions through sustained stochastic diffusion. These results establish a physically grounded framework for understanding plasticity and its failure in continual learning systems, and suggest principled design criteria for adaptive noise schedules in artificial intelligence.
Oct 5, 2026

Sharp dimensional analysis of midpoint methods for Langevin sampling

We study deterministic and randomized midpoint discretizations of Langevin dynamics for a target π∝e−Vπ\propto e^{-V}, where 0≺αI⪯∇2V⪯βI0 \prec αI\preceq\nabla^2V\preceqβI and κ=β/ακ=β/α. To achieve α W2⩽ε\sqrtα\,W_2\leqslant\varepsilon, we show that deterministic Heun uses at most O~(κ4/3d1/3ε−2/3)\widetilde O(κ^{4/3}d^{1/3}\varepsilon^{-2/3}) gradient queries, and underdamped exponential midpoint uses O~(κ5/4d1/4ε−1/2)\widetilde O(κ^{5/4}d^{1/4}\varepsilon^{-1/2}). The proofs exploit cancellation at stationarity and smoothing using techniques from Malliavin calculus, outperforming previous upper bounds based on standard couplings. At bounded condition number, a lower bound matches the dd and ε\varepsilon powers of both deterministic methods. To contrast, for the randomized midpoint methods and Poisson midpoint with at least two grid points (both overdamped and underdamped variants), a simple Gaussian calculation yields a lower bound d1/3ε−1/3d^{1/3}\varepsilon^{-1/3} to get an ε\varepsilon-close sample despite starting at a benign initialization. This shows surprisingly that in high dimensions, deterministic discretizations can outperform their random counterparts.