Flat Minima

Follow this topic Ask about this topic

Latest papers 27

2 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

All topics

Do Flatter Minima Drive Better Generalization? An Algorithmic Separation in Grokking
Oct 8, 2026Mohnish HarwaniNeural Network Generalization Flat Minima
Purdue University
When a Flatness Proxy Is Not a Function: Robustness Certificates and Training Interventions
Sep 29, 2026Vicente Opazo, Jose Calatayud-Mateu, Cristobal Rojas +1Flat Minima Certified Robustness
Centro Nacional de Inteligencia Artificial (CENIA) · Barcelona Supercomputing Center (BSC) · Pontificia Universidad Cat´olica de Chile
A Flatness-Generalization Relation in the Teacher-Student Tree-Committee Machine
Sep 25, 2026Brandon Livio Annesi, Davide Straziota, Enrico Maria MalatestaFlat Minima Neural Network Generalization
Department of Quantitative Life Sciences, ICTP Trieste, Italy
Across the Loss Landscape with Progressive Growth
Aug 25, 2026Paul Caillon, Christophe Cerisara, Alexandre AllauzenDeep Learning Optimization Neural Network Training Dynamics
MILES Team, LAMSADE, Université Paris Dauphine-PSL · LORIA CNRS, Campus Scientifique, rue du Jardin Botanique, 54500 Vandoeuvre-les-Nancy · ESPCI-PSL
How Much Regularization Survives Averaging? Update Masking in Federated Learning
Aug 24, 2026Wenhao Yan, Fu Kuroda, Yucheng Jin +1Federated Learning Implicit Regularization
Faculty of Science & Technology Sophia University Tokyo, Japan · Technical R&D Department Shendian Energy Co., Ltd. Lishui, China
On the Implicit Flatness Bias of Sharpness-Aware Minimization: A Linear Stability Analysis with Quantitative Hyperparameter Bounds
Aug 4, 2026Jiaxin Deng, Junbiao PangSharpness-Aware Minimization Flat Minima
Beijing University of Technology
From Perturbation Correction to Geometry-Aware Sampling: Sharpness-Guided Equilibrium Sampling for Balanced Flat Minima in Long-Tailed Learning
Jul 24, 2026Jiaxin Deng, Junbiao PangSharpness-Aware Minimization Long-Tail Learning
School of Information Science and Technology, Beijing University of Technology, Beijing, China
Scaling Limits of Constant-Stepsize SGD at Flat Minima
Jul 17, 2026Jingyi Zhang, Cheng Mao, Debankur MukherjeeStochastic Optimization Convergence Flat Minima
Georgia Institute of Technology
Dynamics of Gradient Descent with Large Step Size Near a Manifold of Flat Minima
Jul 9, 2026Lachlan Ewen MacDonald, René VidalGradient Descent Flat Minima
Innovation in Data Engineering and Science (IDEAS) · University of Pennsylvania Pennsylvania, PA 19104
Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning
Jul 7, 2026Yao Fu, Chunxia Zhang, Junmin Liu +3Neural Network Generalization Sharpness-Aware Minimization
School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, 710049, China · SGIT AI Lab, State Grid Corporation of China, Xi’an, 710054, China · School of Software, Xi’an Jiaotong University, Xi’an, 710049, China +2
Improving Sparse-View 3DGS Generalization via Flat Minima Optimization
Jul 1, 2026Kangmin Seo, Sangeek Hyun, MinKyu Lee +13D Gaussian Splatting Novel View Synthesis
Sungkyunkwan University, South Korea
Closed-Form Steepest Descent Direction toward Flat Minima: Reducing Upper Bounds on the Loss Hessian Eigenspectrum in Neural Networks
Jun 27, 2026Yuto Omae, Kazuki Sakai, Yohei Kakimoto +3Gradient Descent Neural Network Optimization
Nihon University (Japan) · National Institute of Technology, Nagaoka College (Japan) · Tokyo City University (Japan)
Flatness Preserves Instruction Following in Vision-Language-Action Models
Jun 22, 2026Haochen Zhang, Yonatan BiskSharpness-Aware Minimization Vision-Language-Action Models
Carnegie Mellon University
Fisher-Geometric Sharpness and the Implicit Bias of SGD toward Flat Minima
Jun 18, 2026Md Sakir Ahmed, Kumaresh Sarmah, Hemen DuttaInformation Geometry Implicit Bias
Department of Electronics and Communication Technology Gauhati University Guwahati, Assam, India · Department of Mathematics Gauhati University Guwahati, Assam, India
5% > 100%: Flatness Preference is All You Need for Multimodal Parameter-Efficient Fine-Tuning
Jun 9, 2026Yifan Zhu, Can Lin, Hangjie Yuan +4Deep Learning Optimization Flat Minima
School of Computer Sciences, Beijing University of Posts and Telecommunications, Beijing 100876, China · College of Computer Science and Technology, Zhejiang University, Zhejiang 310027, China · ETH Zürich, Switzerland
Flatness and Generalization: Learning Multi-Index Models with Homogeneous Neural Networks
Jun 3, 2026Harsh Vardhan, Hossein Taheri, Arya MazumdarNeural Network Generalization Flat Minima
Towards the Connection between Activation Sparsity and Flat Minima
May 25, 2026Ze Peng, Jian Zhang, Lei Qi +2Efficient Neural Network Inference Activation Sparsity
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China · Institute of Brain-Machine Interface, Nanjing University, Nanjing 210023, China · School of Computer Science and Engineering, Southeast University, Nanjing 211189, China
Momentum Streams for Optimizer-Inspired Transformers
May 23, 2026Jingchu Gai, Nai-Chieh Huang, Jiayun WuDeep Learning Optimization Transformer
Carnegie Mellon University
Same Target, Different Basins: Hard vs. Soft Labels for Annotator Distributions
May 20, 2026Mirerfan Gheibi, Gashin GhazizadehAnnotator Disagreement Label Distribution Learning
Independent Researcher.
From Mean-Field Limits to Semiclassical Concentration: Global Convergence of the Canonical Evolutionary Strategy
May 19, 2026Matías Neto, Nicolás Garay, Luis Martí +1Dynamical Systems Evolutionary Optimization
Inria Chile Research Center, Av Apoquindo 2827, Las Condes, RM, Chile · Departamento de Ingeniería Matemática, Universidad de Chile, Santiago, Chile
Navigating Potholes with Geometry-Aware Sharpness Minimization
May 15, 2026Simon Dufort-Labbé, Mehrab Hamidi, Razvan Pascanu +3Sharpness-Aware Minimization Flat Minima
Mila, Université de Montréal · Samsung – SAIL Montreal
Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimization
May 11, 2026Jinping Wang, Qinhan Liu, Zhiwu Xie +1Neural Network Generalization Sharpness-Aware Minimization
Wenzhou-Kean University · International Frontier Interdisciplinary Research Institute, Wenzhou-Kean University
FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning
May 9, 2026Bingnan Xiao, Yuan Gao, Bingcong Li +3Sharpness-Aware Minimization Non-IID Federated Learning
Fudan University · ETH Zurich · Edith Cowan University +1
Decentralized SGD with Controlled Disagreement Finds Flatter Minima
Feb 2, 2026Zesen Wang, Mikael JohanssonDecentralized Learning Flat Minima
Division of Decision and Control Systems KTH Royal Institute of Technology
Do Flat Minima Improve Sparse Novel View Synthesis?
Nov 22, 2025Youngsik Yun, Dongjun Gu, Youngjung UhNovel View Synthesis Flat Minima
Yonsei University, Seoul, Korea
On the Escaping Efficiency of Distributed Adversarial Training Algorithms
Sep 14, 2025Ying Cao, Kun Yuan, Ali H. SayedAdversarial Training Adversarial Robustness
Institute of Electrical and Micro Engineering in EPFL, Lausanne · Center for Machine Learning Research (CMLR) in Peking University
Convergence of Sharpness-Aware Minimization Algorithms using Increasing Batch Size and Decaying Learning Rate
Sep 16, 2024Hinata Harada, Hideaki IidukaSharpness-Aware Minimization Nonconvex Optimization
Department of Computer Science Meiji University

Oct 8, 2026, cs.LGJ/K move · Enter open · S save

Do Flatter Minima Drive Better Generalization? An Algorithmic Separation in Grokking

Mohnish Harwani

Purdue University

Neural Network Generalization Flat Minima Grokking Sharpness-Aware Minimization

Open paper AskPDF arXiv

Flat loss landscapes have long been linked to better generalization in neural networks. However, its role as a causal mechanism for generalization is less established. Grokking provides an unique testbed to understand this distinction: models are prone to fit observed data using non-generalizing structure and remain in that regime for prolonged periods, transitioning to generalization only under particular training conditions. In this work, we study whether flat loss landscapes can act as a driving mechanism in this transition. While recent work has argued for flatness as a necessary geometric condition for this transition, we find that biasing training toward flatter solutions using sharpness-aware minimization (SAM) is insufficient to reliably induce this transition, despite producing flatter solutions. However, when SAM is paired with mechanisms that drive generalization such as weight decay, an interesting property emerges: SAM can accelerate the transition to generalizing solutions by up to 4x at the epoch-level. We theoretically untangle this relationship between SAM and weight decay using a minimal interpolating two-layer ReLU model with both memorizing and generalizing solutions. We show that even in this simple setup, flatness alone cannot distinguish a memorizing solution from a generalizing one, while weight decay favors generalizing solutions. However, under a local stability analysis, there exists a window where a memorizing interpolant is locally stable under gradient descent but unstable under SAM in the low-norm regime, which can explain SAM's ability to accelerate this transition. Overall, our results provide a more interpretable account of the role of flatness in driving generalization, especially in settings where models are vulnerable to minimizing loss through learning non-generalizing structure.