Sharpness-Aware Minimization

Also known as SAM

Sharpness-Aware Minimization studies how to train models that are less sensitive to small changes in their parameters, often by taking a gradient step at a nearby high-loss point. Its variants adapt the perturbation radius or direction, use curvature and blockwise information, or reduce the extra computation; researchers also apply these ideas to federated learning, fine-tuning, and long-tailed data. Theory analyzes convergence and stability, while evaluations commonly measure accuracy, corruption robustness, calibration, and performance across domains. A central tension is that flatter minima do not always guarantee better generalization, and the extra optimization cost and sensitivity to perturbation choices remain practical concerns.

Latest papers

39 papers4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Sharpness-aware Orthogonalized Momentum Algorithms
Oct 23, 2026Geetika, Shivani Singh Chauhan, Gauranshi Gupta +3Deep Learning Optimization Sharpness-Aware Minimization
Department of Computer Science and Engineering Indraprastha Institute of Information Technology, Delhi, India. · Department of Mathematics Indraprastha Institute of Information Technology, Delhi, India.
Curvature-Scaled Sharpness-Aware Minimization: Convergence Beyond Smoothness
Oct 21, 2026Dimitris Oikonomou, Nicolas LoizouSharpness-Aware Minimization Optimization Convergence Analysis
Mathematical Institute for Data Science (MINDS), Johns Hopkins University · Department of Computer Science, Johns Hopkins University · Department of Applied Mathematics and Statistics, Johns Hopkins University
One Size Does Not Fit All: Curvature-Aware Adaptive Perturbation for SAM
Oct 14, 2026Alessia Rossi, Felipe Campelo, Gabriel L. OliveiraCurvature-Aware Optimization Sharpness-Aware Minimization
School of Engineering Mathematics and Technology, University of Bristol, Bristol, Lovelace Building, Ada, Bristol, BS8 1TW, UK
Do Flatter Minima Drive Better Generalization? An Algorithmic Separation in Grokking
Oct 8, 2026Mohnish HarwaniNeural Network Generalization Flat Minima
Purdue University
Sparsity-Adaptive Sharpness-Aware Minimization
Sep 13, 2026Shiryu Ueno, Yoshikazu Hayashi, Kunihito KatoSparse Neural Networks Neural Network Pruning
Faculty of Engineering Gifu University Gifu, Japan
Explaining f-Divergence-Based Regularization via Local Curvature and Sharpness-Aware Minimization
Sep 8, 2026Nour Jamoussi, Marios KountourisNeural Network Generalization Sharpness-Aware Minimization
Communication Systems Department EURECOM France · Department of Computer Science and Artificial Intelligence, University of Granada, Spain
EMASAM: a Computationally Efficient Sharpness-Aware Minimization via EMA-Guided Perturbations
Aug 15, 2026Tanapat Ratchatorn, Masayuki TanakaLoss Landscape Sharpness Sharpness-Aware Minimization
Institute of Science Tokyo, Meguro, Tokyo, Japan
A Compositional Theory of Curvature in Probabilistic Circuits
Aug 13, 2026Hrithik Suresh, Sahil Sidheekh, Shelar Parth Vijay +3Sharpness-Aware Minimization Probabilistic Circuits
Mehta Family School of Data Science and Artificial Intelligence, Department of Data Science, Indian Institute of Technology Palakkad, Kerala, India · Erik Jonsson School of Engineering & Computer Science, The University of Texas at Dallas, Richardson, TX, USA · AT&T CDO
On the Implicit Flatness Bias of Sharpness-Aware Minimization: A Linear Stability Analysis with Quantitative Hyperparameter Bounds
Aug 4, 2026Jiaxin Deng, Junbiao PangSharpness-Aware Minimization Flat Minima
Beijing University of Technology
Sharpness-Aware Minimization and Muon: Robustness under the Spectral Norm
Jul 28, 2026Wenzhi Zhong, Edward Milsom, Michael MurraySharpness-Aware Minimization Muon Optimizer
Department of Mathematical Sciences University of Bath · Department of Computer Science University of Bath
From Perturbation Correction to Geometry-Aware Sampling: Sharpness-Guided Equilibrium Sampling for Balanced Flat Minima in Long-Tailed Learning
Jul 24, 2026Jiaxin Deng, Junbiao PangSharpness-Aware Minimization Long-Tail Learning
School of Information Science and Technology, Beijing University of Technology, Beijing, China
Gradient-Energy Guided Block-Wise Perturbations for Sharpness-Aware Minimization
Jul 17, 2026Zhen Huang, Jiaxin Deng, Junbiao PangSharpness-Aware Minimization Neural Network Optimization
Faculty of Information Technology, Beijing University of Technology, Beijing, China
Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning
Jul 7, 2026Yao Fu, Chunxia Zhang, Junmin Liu +3Neural Network Generalization Sharpness-Aware Minimization
School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, 710049, China · SGIT AI Lab, State Grid Corporation of China, Xi’an, 710054, China · School of Software, Xi’an Jiaotong University, Xi’an, 710049, China +2
SAMPLe: SAM-based Optimizer for Prompt Learning in VLMs
Jul 7, 2026Hossein Rajoli, Fatemeh Lotfi, Niloufar Alipour Talemi +3Sharpness-Aware Minimization VLM Adaptation
Clemson University, Clemson SC 29634, USA · Siemens Energy (AI Lab), Orlando FL 32826, USA · University of Arizona, Tucson AZ 85721, USA
FedFFT: Taming Client Drift in Federated SAM via Spectral Perturbation Filtering
Jul 5, 2026Liyang Yuan, Yibo Yang, Dandan GuoSharpness-Aware Minimization Non-IID Federated Learning
School of Artificial Intelligence, Jilin University, Changchun, China · King Abdullah University of Science and Technology, Saudi Arabia
Flatness Preserves Instruction Following in Vision-Language-Action Models
Jun 22, 2026Haochen Zhang, Yonatan BiskSharpness-Aware Minimization Vision-Language-Action Models
Carnegie Mellon University
SHAPO: Sharpness-Aware Policy Optimization for Safe Exploration
Jun 8, 2026Kaustubh Mani, Yann Pequignot, Vincent Mai +1Sharpness-Aware Minimization RL Exploration
Universit´e de Montr´eal · Mila - Qu´ebec AI Institute · Universit´e Laval +2
Adaptive Sharpness-Aware Minimization with a Polyak-type Step size: A Theory-Grounded Scheduler
Jun 1, 2026Dimitris Oikonomou, Nicolas LoizouSharpness-Aware Minimization Learning Rate Scheduling
Mathematical Institute for Data Science (MINDS), Johns Hopkins University, Baltimore, MD, USA · Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA · Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA
DASM: Domain-Aware Sharpness Minimization for Multi-Domain Voice Stream Steganalysis
May 19, 2026Pengcheng Zhou, Pianran Guo, Shuhua Chen +3Sharpness-Aware Minimization Domain Generalization
Department of Electrical and Computer Engineering, National University of Singapore, Singapore 117583, Singapore · School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing 100876, China
Navigating Potholes with Geometry-Aware Sharpness Minimization
May 15, 2026Simon Dufort-Labbé, Mehrab Hamidi, Razvan Pascanu +3Sharpness-Aware Minimization Flat Minima
Mila, Université de Montréal · Samsung – SAIL Montreal
Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimization
May 11, 2026Jinping Wang, Qinhan Liu, Zhiwu Xie +1Neural Network Generalization Sharpness-Aware Minimization
Wenzhou-Kean University · International Frontier Interdisciplinary Research Institute, Wenzhou-Kean University
FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning
May 9, 2026Bingnan Xiao, Yuan Gao, Bingcong Li +3Sharpness-Aware Minimization Non-IID Federated Learning
Fudan University · ETH Zurich · Edith Cowan University +1
Flatness and Gradient Alignment Are Both Necessary: Spectral-Aware Gradient-Aligned Exploration for Multi-Distribution Learning
May 8, 2026Aristotelis Ballas, Christos DiouSharpness-Aware Minimization Multi-Task Learning
Department of Informatics and Telematics Harokopio University of Athens Omirou 9, Athens, Greece
Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting
May 4, 2026Ishaan Watts, Catherine Li, Sachin Goyal +2Language Model Pretraining Sharpness-Aware Minimization
Carnegie Mellon University
Sharpness-Aware Poisoning: Enhancing Transferability of Injective Attacks on Recommender Systems
Apr 24, 2026Junsong Xie, Yonghui Yang, Pengyang Shao +1Sharpness-Aware Minimization Adversarial Attacks
Key Laboratory of Knowledge Engineering with Big Data, Hefei University of Technology, Hefei 230009, China · Intelligent Interconnected Systems Laboratory of Anhui Province, Hefei University of Technology, Hefei 230009, China · Laboratory of NExT++, National University of Singapore, Singapore
Sharpness-Aware Surrogate Training for On-Sensor Spiking Neural Networks
Apr 6, 2026Maximilian NicholsonSpiking Neural Networks Event-Based Vision
University of Bath, United Kingdom
Beyond Semantic Priors: Mitigating Optimization Collapse for Generalizable Visual Forensics
Mar 25, 2026Jipeng Liu, Haichao Shi, Siyu Xing +2Deepfake Detection Digital Image Forensics
Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100193, China · School of Cyber Science and Technology, Beihang University, Beijing 100191, China
Sharpness-Aware Minimization in Logit Space Efficiently Enhances Direct Preference Optimization
Mar 18, 2026Haocheng Luo, Zehang Deng, Thanh-Toan Do +3LLM Alignment Direct Preference Optimization
Monash University, Australia · Swinburne University of Technology, Australia
Sharpness Aware Surrogate Training for Spiking Neural Networks
Mar 14, 2026Maximilian NicholsonSpiking Neural Networks Sharpness-Aware Minimization
University of Bath, United Kingdom
ZO-SAM: Zero-Order Sharpness-Aware Minimization for Efficient Sparse Training
Mar 13, 2026Jie Ji, Gen Li, Kaiyuan Deng +2Sharpness-Aware Minimization Sparse Neural Networks
Clemson University · University of Arizona

Oct 23, 2026J/K move · Enter open · S save

Sharpness-aware Orthogonalized Momentum Algorithms

Geetika, Shivani Singh Chauhan, Gauranshi Gupta, Anant Jain+2

Department of Computer Science and Engineering Indraprastha Institute of Information Technology, Delhi, India. · Department of Mathematics Indraprastha Institute of Information Technology, Delhi, India.

Deep Learning Optimization Sharpness-Aware Minimization Stochastic Gradient Descent

Open paper AskPDF arXiv

Adversarial sharpness penalty per iteration of a base optimizer, known as sharpness-aware minimization (SAM), is an accepted technique to improve generalization in deep learning. Recently, Muon has emerged as a dominant optimization algorithm for training large language models (LLMs). It achieves lower sample complexity for heavy-tailed gradient updates - a phenomenon common in LLM training. However, sharpness penalties with Muon updates are yet largely unexplored in the literature. Moreover, the behavior of SAM under heavy-tailed noise is unknown both theoretically and empirically. A potential reason for this gap is the additional cost of extra gradient computation per iteration for penalty perturbations in SAM, which becomes prohibitive for LLMs. This paper introduces Sharpness-aware Orthogonalized Momentum Algorithm (SOMA) to combine a computationally efficient and theoretically grounded perturbation scheme with the Muon algorithm. The key idea is to use stale (purely) stochastic gradient noise for adversarial perturbations. Theoretically, we show that SOMA's sample complexity is invariant to the perturbation direction. We provide explorations and insights into SOMA's robustness to heavy-tailed gradient noise. We extensively explore SOMA's efficacy in (a) mitigating quality degradation due to post-training quantization, and (b) catastrophic forgetting during fine-tuning of LLMs.