Overview

The Edge of Stability studies what happens when gradient-based training uses step sizes near the point where updates along high-curvature directions stop being stable. Neural networks can progressively sharpen until their largest Hessian eigenvalue approaches roughly twice the inverse learning rate, then show persistent oscillations or self-stabilization. Researchers analyze these dynamics with Hessian and directional-curvature measurements, dynamical-systems and bifurcation theory, and models of stochastic, momentum, adaptive, and non-Euclidean optimizers. A central open question is how stability thresholds and their effects on convergence, generalization, and learned representations depend on the optimizer, batch size, data, and geometry.

Latest papers

CardsList

42 papers7 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

  1. High-dimensional limit theorems for adaptive variants of SGD I: Ballistic limits

    Nov 14, 2026Aukosh Jagannath, Varnan SarangianSGD with MomentumAdaptive Gradient Methods

  2. Symmetry Breaking at the Edge of Stability via Central Flow Analysis

    Nov 4, 2026Naima Elosegui Borras, Georgios ArvanitidisNeural Network Training DynamicsGradient Descent Dynamics

  3. Stochastic learning flow: moment equations for stochastic gradient descent at finite step size

    Oct 30, 2026Christian Schmid, James M MurrayNeural Network Training DynamicsStochastic Gradient Descent

  4. Does LLM Pre-Training Typically Occur at the Edge of Stability?

    Oct 27, 2026Yuhang Cai, Haodong Wen, Haofeng Huang +3Edge of StabilityNeural Network Training Dynamics

  5. Muon Operates Beyond the Edge of Stability

    Oct 16, 2026Niko Covic, Baraq Lipshitz, Alexander Padula +1Muon OptimizerEdge of Stability

  6. A Minimal ReLU Classifier on Non-Linearly Separable Data: Convergence and Implicit Bias at and beyond the Edge of Stability

    Oct 16, 2026Quan Nguyen, Robin Yadav, Param Mody +1Neural Network OptimizationReLU Neural Networks

  7. A Solvable Model of Adaptive Learning Rate Rescaling: Acceleration, Stability & Scaling

    Oct 5, 2026Itay Lavie, Clarissa Lauditi, Cengiz PehlevanEdge of StabilityNeural Scaling Laws

  8. Mind the Drift: Diagonal Linear Networks Under Large Learning Rates

    Oct 5, 2026Aniket Sanyal, Tom Jacobs, Rebekka BurkholzEdge of StabilityDeep Learning Optimization

  9. Tight Transition Time Bounds for Separable Logistic Regression at the Edge of Stability

    Oct 1, 2026Haodong Wen, Kaiyue Wen, Jiaye TengEdge of StabilityLogistic Regression

  10. A Journey to the Edge of Stability

    Sep 26, 2026Jaerin Lee, Kyoung Mu LeeDeep Learning OptimizationNeural Network Training Dynamics

  11. Information Geometric Self-Organization at the Edge of Stability in High-Capacity Kernel Associative Memories

    Sep 15, 2026Akira TamamoriInformation GeometryEdge of Stability

  12. The Multiple Timescales of Gradient Descent on the Edge of Stability: A Perturbative Derivation of the Central Flow

    Sep 1, 2026Raphaël BerthierDynamical SystemsEdge of Stability

  13. How Edge of Stability Hinders SCAFFOLD in Federated Optimization

    Aug 26, 2026Anant Khandelwal, Michael Crawshaw, Mingrui LiuFederated LearningNon-IID Federated Learning

  14. Adam at the Edge of Stability: Adaptive Feedback, Provable Oscillation, and Gradient Reversal

    Aug 21, 2026Yiman Fong, Heng YangEdge of StabilityAdaptive Gradient Methods

  15. The Road Taken: The Role of Optimizers at the Edge of Stability

    Aug 19, 2026Jaerin Lee, Kyoung Mu LeeEdge of StabilityDeep Learning Optimization

  16. A Defense of the Quadratic Model

    Jul 23, 2026Alexandru Meterez, Pranav Ajit Nair, Depen Morwani +3Language Model PretrainingEdge of Stability

  17. Convergence of Large-Stepsize SGD for the Cross-entropy Loss via Stochastic Lyapunov Stability

    Jun 29, 2026Konstantinos Emmanouilidis, Lachlan MacDonald, Salma Tarmoun +1Edge of StabilityStochastic Optimization Convergence

  18. Taming Curvature: Architecture Warm-Up for Stable Transformer Training

    Jun 15, 2026Sameera Ramasinghe, Ajanthan Thalaiyasingam, Hadi Mohaghegh Dolatabadi +6Edge of StabilityTransformer

  19. Flatland: The Adventures of Gradient Descent with Large Step Sizes

    Jun 4, 2026Leonardo Galli, Curtis Fox, Wiebke Bartolomaeus +2Gradient Descent DynamicsEdge of Stability

  20. Mean--Fluctuation Dynamics at the Edge of Stability

    Jun 3, 2026Antonin Chodron de CourcelEdge of StabilityGradient Descent Dynamics

  21. Gradient Descent with Large Step Size Restores Symmetry in Deep Linear Networks with Multi-Pathway

    May 29, 2026Hee-Sung Kim, Sungyoon LeeEdge of StabilitySymmetry Breaking