Flat Minima

Ask about this topic

Overview

Flat-minima research studies how the shape of a model’s loss landscape relates to what it learns, including its ability to generalize, resist perturbations, and adapt across tasks. Methods such as sharpness-aware minimization (SAM), Hessian-based regularization, and geometry-aware perturbations try to steer training toward flatter solutions; theory also examines how SGD noise, learning rates, and batch size shape the minima it reaches. Work spans neural-network theory and practical settings such as federated learning, fine-tuning, and vision tasks. A central open question is whether flatness reliably predicts generalization: parameter symmetries and task differences can make common measures misleading, and flatter solutions do not always perform better.

Latest papers

CardsList

34 papers2 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. Muon Operates Beyond the Edge of Stability

    Oct 16, 2026Niko Covic, Baraq Lipshitz, Alexander Padula +1Muon OptimizerEdge of Stability

    Department of Informatics, ETH Zurich, Switzerland.

  2. Do Flatter Minima Drive Better Generalization? An Algorithmic Separation in Grokking

    Oct 8, 2026Mohnish HarwaniNeural Network GeneralizationFlat Minima

    Purdue University

  3. When a Flatness Proxy Is Not a Function: Robustness Certificates and Training Interventions

    Sep 29, 2026Vicente Opazo, Jose Calatayud-Mateu, Cristobal Rojas +1Flat MinimaCertified Robustness

    Centro Nacional de Inteligencia Artificial (CENIA) · Barcelona Supercomputing Center (BSC) · Pontificia Universidad Cat´olica de Chile

  4. A Flatness-Generalization Relation in the Teacher-Student Tree-Committee Machine

    Sep 25, 2026Brandon Livio Annesi, Davide Straziota, Enrico Maria MalatestaFlat MinimaNeural Network Generalization

    Department of Quantitative Life Sciences, ICTP Trieste, Italy

  5. Across the Loss Landscape with Progressive Growth

    Aug 25, 2026Paul Caillon, Christophe Cerisara, Alexandre AllauzenDeep Learning OptimizationLoss Landscape

    MILES Team, LAMSADE, Université Paris Dauphine-PSL · LORIA CNRS, Campus Scientifique, rue du Jardin Botanique, 54500 Vandoeuvre-les-Nancy · ESPCI-PSL

  6. How Much Regularization Survives Averaging? Update Masking in Federated Learning

    Aug 24, 2026Wenhao Yan, Fu Kuroda, Yucheng Jin +1Federated LearningImplicit Regularization

    Faculty of Science & Technology Sophia University Tokyo, Japan · Technical R&D Department Shendian Energy Co., Ltd. Lishui, China

  7. On the Implicit Flatness Bias of Sharpness-Aware Minimization: A Linear Stability Analysis with Quantitative Hyperparameter Bounds

    Aug 4, 2026Jiaxin Deng, Junbiao PangSharpness-Aware MinimizationFlat Minima

    Beijing University of Technology

  8. From Perturbation Correction to Geometry-Aware Sampling: Sharpness-Guided Equilibrium Sampling for Balanced Flat Minima in Long-Tailed Learning

    Jul 24, 2026Jiaxin Deng, Junbiao PangSharpness-Aware MinimizationLong-Tail Learning

    School of Information Science and Technology, Beijing University of Technology, Beijing, China

  9. Scaling Limits of Constant-Stepsize SGD at Flat Minima

    Jul 17, 2026Jingyi Zhang, Cheng Mao, Debankur MukherjeeStochastic Optimization ConvergenceFlat Minima

    Georgia Institute of Technology

  10. Dynamics of Gradient Descent with Large Step Size Near a Manifold of Flat Minima

    Jul 9, 2026Lachlan Ewen MacDonald, René VidalGradient DescentFlat Minima

    Innovation in Data Engineering and Science (IDEAS) · University of Pennsylvania Pennsylvania, PA 19104

  11. Leveraging Extragradient for Effective Sharpness-Aware Minimization in Deep Learning

    Jul 7, 2026Yao Fu, Chunxia Zhang, Junmin Liu +3Neural Network GeneralizationSharpness-Aware Minimization

    School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, 710049, China · SGIT AI Lab, State Grid Corporation of China, Xi’an, 710054, China · School of Software, Xi’an Jiaotong University, Xi’an, 710049, China +2

  12. Improving Sparse-View 3DGS Generalization via Flat Minima Optimization

    Jul 1, 2026Kangmin Seo, Sangeek Hyun, MinKyu Lee +13D Gaussian SplattingNovel View Synthesis

    Sungkyunkwan University, South Korea

  13. Closed-Form Steepest Descent Direction toward Flat Minima: Reducing Upper Bounds on the Loss Hessian Eigenspectrum in Neural Networks

    Jun 27, 2026Yuto Omae, Kazuki Sakai, Yohei Kakimoto +3Gradient DescentNeural Network Optimization

    Nihon University (Japan) · National Institute of Technology, Nagaoka College (Japan) · Tokyo City University (Japan)

  14. Flatness Preserves Instruction Following in Vision-Language-Action Models

    Jun 22, 2026Haochen Zhang, Yonatan BiskSharpness-Aware MinimizationVision-Language-Action Models

    Carnegie Mellon University

  15. Fisher-Geometric Sharpness and the Implicit Bias of SGD toward Flat Minima

    Jun 18, 2026Md Sakir Ahmed, Kumaresh Sarmah, Hemen DuttaInformation GeometryImplicit Bias

    Department of Electronics and Communication Technology Gauhati University Guwahati, Assam, India · Department of Mathematics Gauhati University Guwahati, Assam, India

  16. 5% > 100%: Flatness Preference is All You Need for Multimodal Parameter-Efficient Fine-Tuning

    Jun 9, 2026Yifan Zhu, Can Lin, Hangjie Yuan +4Deep Learning OptimizationFlat Minima

    School of Computer Sciences, Beijing University of Posts and Telecommunications, Beijing 100876, China · College of Computer Science and Technology, Zhejiang University, Zhejiang 310027, China · ETH Zürich, Switzerland

  17. Flatness and Generalization: Learning Multi-Index Models with Homogeneous Neural Networks

    Jun 3, 2026Harsh Vardhan, Hossein Taheri, Arya MazumdarNeural Network GeneralizationFlat Minima

  18. Towards the Connection between Activation Sparsity and Flat Minima

    May 25, 2026Ze Peng, Jian Zhang, Lei Qi +2Efficient Neural Network InferenceActivation Sparsity

    State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China · Institute of Brain-Machine Interface, Nanjing University, Nanjing 210023, China · School of Computer Science and Engineering, Southeast University, Nanjing 211189, China

  19. Momentum Streams for Optimizer-Inspired Transformers

    May 23, 2026Jingchu Gai, Nai-Chieh Huang, Jiayun WuDeep Learning OptimizationTransformer

    Carnegie Mellon University

  20. Same Target, Different Basins: Hard vs. Soft Labels for Annotator Distributions

    May 20, 2026Mirerfan Gheibi, Gashin GhazizadehAnnotator DisagreementLabel Distribution Learning

    Independent Researcher.

  21. From Mean-Field Limits to Semiclassical Concentration: Global Convergence of the Canonical Evolutionary Strategy

    May 19, 2026Matías Neto, Nicolás Garay, Luis Martí +1Dynamical SystemsEvolutionary Optimization

    Inria Chile Research Center, Av Apoquindo 2827, Las Condes, RM, Chile · Departamento de Ingeniería Matemática, Universidad de Chile, Santiago, Chile

  22. Navigating Potholes with Geometry-Aware Sharpness Minimization

    May 15, 2026Simon Dufort-Labbé, Mehrab Hamidi, Razvan Pascanu +3Sharpness-Aware MinimizationFlat Minima

    Mila, Université de Montréal · Samsung – SAIL Montreal

  23. Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimization

    May 11, 2026Jinping Wang, Qinhan Liu, Zhiwu Xie +1Neural Network GeneralizationSharpness-Aware Minimization

    Wenzhou-Kean University · International Frontier Interdisciplinary Research Institute, Wenzhou-Kean University

  24. FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning

    May 9, 2026Bingnan Xiao, Yuan Gao, Bingcong Li +3Sharpness-Aware MinimizationNon-IID Federated Learning

    Fudan University · ETH Zurich · Edith Cowan University +1

  25. The Geometry of Robustness: Optimizing Loss Landscape Curvature and Feature Manifold Alignment for Robust Finetuning of Vision-Language Models

    Mar 28, 2026Shivang Chopra, Shaunak Halbe, Chengyue Huang +2PEFT of VLMsVLM Robustness

    Georgia Institute of Technology

  26. Are Flat Minima an Illusion?

    Mar 24, 2026Michael Timothy BennettNeural Network GeneralizationFlat Minima

    School of Computing, The Australian National University, Canberra, ACT 2601, Australia

  27. FedNSAM:Consistency of Local and Global Flatness for Federated Learning

    Feb 27, 2026Junkang Liu, Fanhua Shang, Yuxuan Tian +2Federated LearningSharpness-Aware Minimization

    College of Intelligence and Computing, Tianjin University, Tianjin, China · College of Management and Economics, Tianjin University, Tianjin, China · Medical College, Tianjin University; Peng Cheng Laboratory, Tianjin, China +1

  28. Decentralized SGD with Controlled Disagreement Finds Flatter Minima

    Feb 2, 2026Zesen Wang, Mikael JohanssonDecentralized LearningFlat Minima

    Division of Decision and Control Systems KTH Royal Institute of Technology

  29. Do Flat Minima Improve Sparse Novel View Synthesis?

    Nov 22, 2025Youngsik Yun, Dongjun Gu, Youngjung UhLoss Landscape SharpnessNovel View Synthesis

    Yonsei University, Seoul, Korea

  30. A Function-Centric Perspective on Flat and Sharp Minima

    Oct 14, 2025Israel Mason-Williams, Gabryel Mason-Williams, Helen YannakoudakisFlat MinimaLoss Landscape Geometry

    Department of Informatics, UKRI Safe and Trusted AI, London, United Kingdom · Queen Mary University of London, London, United Kingdom · Department of Informatics, King’s College London, London, United Kingdom