Loss Landscape Sharpness

Latest papers

19 papers1 paper in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

A Journey to the Edge of Stability
Sep 26, 2026Jaerin Lee, Kyoung Mu LeeDeep Learning Optimization Neural Network Training Dynamics
Computer Vision Lab, ASRI Seoul National University Seoul 08826, Korea
Sparsity-Adaptive Sharpness-Aware Minimization
Sep 13, 2026Shiryu Ueno, Yoshikazu Hayashi, Kunihito KatoSparse Neural Networks Neural Network Pruning
Faculty of Engineering Gifu University Gifu, Japan
EMASAM: a Computationally Efficient Sharpness-Aware Minimization via EMA-Guided Perturbations
Aug 15, 2026Tanapat Ratchatorn, Masayuki TanakaLoss Landscape Sharpness Sharpness-Aware Minimization
Institute of Science Tokyo, Meguro, Tokyo, Japan
Mini-batch Noise Lowers Sharpness via Dominant-Subspace Fluctuations
Jul 25, 2026Junho So, Dongwook ShinNeural Network Training Dynamics Gradient Descent Dynamics
Department of Mathematics, Ajou University, South Korea
From Perturbation Correction to Geometry-Aware Sampling: Sharpness-Guided Equilibrium Sampling for Balanced Flat Minima in Long-Tailed Learning
Jul 24, 2026Jiaxin Deng, Junbiao PangSharpness-Aware Minimization Long-Tail Learning
School of Information Science and Technology, Beijing University of Technology, Beijing, China
How the Hessian-Spectrum of Neural Networks Depends on Data
Jul 15, 2026Jasraj Singh, Enea Monzio Compagnoni, Antonio OrvietoLoss Landscape Sharpness Neural Network Optimization
ELLIS Institute, Tübingen, Germany · Universität Basel, Basel, Switzerland · MPI-IS, Tübingen, Germany
Directional Curvature from Armijo Backtracking: A Low-Cost Sharpness Probe and a Calibration-Free Learning-Rate Safeguard for Adam
Jul 4, 2026Ashmitha R, Jörg FrochteGradient Descent Adaptive Gradient Methods
Department of Artificial Intelligence and Data Science, Sri Ramakrishna Engineering College, Anna University, Coimbatore 641022, Tamil Nadu, India · Interdisciplinary Institute for Applied AI and Data Science Ruhr (AKIS), Department of Electrical Engineering and Computer Science, Bochum University of Applied Sciences, Am Hochschulcampus 1, 44801 Bochum, Germany
How Far Can Sharpness and Complexity Jointly Explain Generalization?
Jun 27, 2026Ziyu Cheng, Xitong Zhang, Longxiu Huang +1Neural Network Generalization Statistical Learning Theory
Michigan State University
Flatland: The Adventures of Gradient Descent with Large Step Sizes
Jun 4, 2026Leonardo Galli, Curtis Fox, Wiebke Bartolomaeus +2Edge of Stability Gradient Descent
LMU Munich · Munich Center for Machine Learning · University of British Columbia +1
Why Muon Outperforms Adam: A Curvature Perspective
Jun 3, 2026Shuche Wang, Fengzhuo Zhang, Jiaxiang Li +2Loss Landscape Sharpness Neural Network Optimization
National University of Singapore · Yale University · University of Minnesota
Navigating Potholes with Geometry-Aware Sharpness Minimization
May 15, 2026Simon Dufort-Labbé, Mehrab Hamidi, Razvan Pascanu +3Sharpness-Aware Minimization Flat Minima
Mila, Université de Montréal · Samsung – SAIL Montreal
Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learning
May 14, 2026Oubo Ma, Ruixiao Lin, Yang Dai +4Reinforcement Learning Loss Landscape Sharpness
1Zhejiang University · 2National University of Defense Technol · 3Xi’an Jiaotong University
Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimization
May 11, 2026Jinping Wang, Qinhan Liu, Zhiwu Xie +1Neural Network Generalization Sharpness-Aware Minimization
Wenzhou-Kean University · International Frontier Interdisciplinary Research Institute, Wenzhou-Kean University
From Flat Facts to Sharp Hallucinations: Detecting Stubborn Errors via Gradient Sensitivity
May 1, 2026Yee Zhing Liew, Andrew Huey Ping Tan, Anwar P. P. Abdul MajeedLoss Landscape Sharpness Hallucination Detection
School of Intelligent Manufacturing Ecosystem, Xi’an Jiaotong-Liverpool University, People’s Republic of China · Department of Computer Science, University of Liverpool, United Kingdom · Faculty of Engineering and Technology, Sunway University, Malaysia +1
Too Sharp, Too Sure: When Calibration Follows Curvature
Apr 22, 2026Alessandro Morosini, Matea Gjika, Tomaso Poggio +1Loss Landscape Sharpness Margin-Based Learning
Massachusetts Institute of Technology, Cambridge, MA, USA.
Product-Stability: Provable Convergence for Gradient Descent on the Edge of Stability
Apr 3, 2026Eric GanDeep Learning Optimization Edge of Stability
Independent Researcher.
Why the Maximum Second Derivative of Activations Matters for Adversarial Robustness
Mar 25, 2026Yunrui Yu, Hang Su, Jun ZhuLoss Landscape Sharpness Adversarial Robustness
1Tsinghua University, Beijing, China.
Revisiting Sharpness-Aware Minimization: A More Faithful and Effective Implementation
Mar 9, 2026Jianlong Chen, Zhiming ZhouSharpness-Aware Minimization Deep Learning Optimization
Key Laboratory of Interdisciplinary Research of Computation and Economics Shanghai University of Finance and Economics
Do Flat Minima Improve Sparse Novel View Synthesis?
Nov 22, 2025Youngsik Yun, Dongjun Gu, Youngjung UhLoss Landscape Sharpness Novel View Synthesis
Yonsei University, Seoul, Korea

Sep 26, 2026J/K move · Enter open · S save

A Journey to the Edge of Stability

Jaerin Lee, Kyoung Mu Lee

Computer Vision Lab, ASRI Seoul National University Seoul 08826, Korea

Deep Learning Optimization Neural Network Training Dynamics Edge of Stability Gradient Descent Dynamics Loss Landscape Sharpness

Open paper AskPDF arXiv

It has recently been found that deep learning often occurs at the "edge of stability (EoS)," where the maximum Hessian eigenvalue of the model is stabilized at a value reciprocal to the learning rate. However, what happens before we reach that regime? We fix a deep learning problem and vary first order optimization methods with dense learning rate sweeps. We then track the characterizing quantities of a learning trajectory: the loss, the sharpness, and the alignment between consecutive gradients. To our surprise, if we scale the learning rate by the dc gain of the optimizer, these traces from the sweeps from different optimizers almost perfectly overlap across a large range of learning rates. The dc-normalized optimizers have another role that only becomes apparent in high learning rates: they select when the sharpness value detaches from this universal curve and enters the edge of stability. Upon this discovery, we specify three distinct regimes with respect to the dc-adjusted learning rate: the low-LR regime where the trajectory is nearly insensitive to the optimizer, the high-LR regime, where the optimizer governs the sharpness according to the EoS reciprocal rule, and the in-between mid-LR regime where so-called progressive sharpening originates independently of the optimizer. This distinguishes the role of the optimizer, the learning rate, and the model in shaping the learning progress.