Edge of Stability

The Edge of Stability studies what happens when gradient-based training uses step sizes near the point where updates along high-curvature directions stop being stable. Neural networks can progressively sharpen until their largest Hessian eigenvalue approaches roughly twice the inverse learning rate, then show persistent oscillations or self-stabilization. Researchers analyze these dynamics with Hessian and directional-curvature measurements, dynamical-systems and bifurcation theory, and models of stochastic, momentum, adaptive, and non-Euclidean optimizers. A central open question is how stability thresholds and their effects on convergence, generalization, and learned representations depend on the optimizer, batch size, data, and geometry.