Flat Minima

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, with none the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 55

All topics

Removing spurious minima for planar features by skip connections
Oct 1, 2026Jakob Paul Zimmermann, Moritz Grillo, Andrei Balakin +1Flat Minima Skip
Freie Universität Berlin · Technische Universität Berlin
Q-MINO: A Minimal-Norm Method for Quantization-Aware Training
Sep 30, 2026Don LiQuantization-Aware Training Flat Minima
Department of Mathematics & Statistics Portland State University
Local-Minimum Escaper: Programmatic Subgoal Generation for Robust Navigation in Unknown Environments
Sep 30, 2026Yin Gu, Xinming Zhang, Shanze Wang +2Robot Navigation Object Goal Navigation
Eastern Institute of Technology, Ningbo, China · Tsinghua University, China · Department of Aeronautical and Aviation Engineering, Polytechnic University, Hong Kong
Precise Convergence Speed of Clipped SGD
Sep 24, 2026David A. R. RobinGradient Descent Convergence
LAMSADE, Université Paris-Dauphine, PSL Research University
Scalable Minimum-Volume Simplex Estimation with Non-asymptotic Analysis
Sep 22, 2026Jun LI, Yanlong Guo, Zhaozhao ZengSimplex Optimal Sample Complexity
School of Computer Science, China University of Geosciences
Learning-Based 3D Reconstruction of Power Networks from Aerial Point Clouds
Sep 20, 2026Rishabh Jain, Anuja Saini, Vishal JainPoint Clouds Power Systems
AiDash
Teacher Geometry Shapes Learnability in Teacher-Student Networks
Sep 10, 2026Kai J. Sandbrink, Flavio Martinelli, Alexander van Meegen +2Two-Layer Neural Networks Teacher
Department of Experimental Psychology, University of Oxford · Department of Physics, RWTH Aachen
No Unique Minimizer, No Problem: On the Consistency of Robust Neural Classifiers
Aug 9, 2026Subhabrata Majumdar, Anand Deo, Partha Pratim Saha +1Empirical Risk Minimization Cross-Entropy Losses
Indian Institute of Management Bangalore · Independent Researcher · Indian Statistical Institute, Kolkata
Unifying quantum measurement constructions via a relative-entropy minimum change principle
Aug 4, 2026Nana Liu, Mark M. WildeFlat Minima Principles
Institute of Natural Sciences, School of Mathematical Sciences, Ministry of Education Key Laboratory in Scientific and Engineering Computing, and Global College, Shanghai Jiao Tong University, Shanghai 200240, China · School of Electrical and Computer Engineering, Cornell University, Ithaca, New York 14850, USA
Tight Worst-Case Bounds for the Smallest Eigenvalue of ReLU NTK Gram Matrices
Aug 4, 2026Zhao SongRectified Linear Unit Spectral Decomposition
From Perturbation Correction to Geometry-Aware Sampling: Sharpness-Guided Equilibrium Sampling for Balanced Flat Minima in Long-Tailed Learning
Jul 24, 2026Jiaxin Deng, Junbiao PangSharpness-Aware Minimization Balanced Learning
School of Information Science and Technology, Beijing University of Technology, Beijing, China
Deep Reinforcement-Learning-Guided Model Predictive Control for Preventing Overtakes in Autonomous Racing
Jul 23, 2026Yufei Xi, Yijie Liao, Tulga ErsalAutonomous Racing Model Predictive Control
Department of Mechanical Engineering, College of Engineering, University of Michigan, Ann Arbor, MI, USA. · Department of Electrical Engineering and Computer Science, College of Engineering, University of Michigan, Ann Arbor, MI, USA. · School of Computing, National University of Singapore, Singapore.
TRIM: Reducing AI-Generated CodeSlop via Agent Trajectory Minimization
Jul 20, 2026Alex Mathai, Shobini Iyer, Aleksandr Nogikh +4Code Generation Code Optimization
Dept. of Computer Science Columbia University, New York City, USA · Google Inc München, Germany · Google DeepMind Mountain View, CA, USA +1
Effects of width-dependent model hyperparameters and $\ell_2$ -regularization on the loss landscape of two-layer ReLU networks
Jul 18, 2026Haruka Eshima, Makoto YamadaRectified Linear Unit Networks Anisotropic Loss Landscapes
Okinawa Institute of Science and Technology, Okinawa, Japan
How to Build Marcus's Algebraic Mind: From Minsky's Emotion-Machine Viewpoint
Jul 18, 2026Hiroyuki Chuma, Kanji Otsuka, Yoichi SatoCognitive Science Reasoning Traces
Institute of Innovation Research, Hitotsubashi University (Professor Emeritus) · Meisei University (Professor Emeritus) · Shuhari System
Scaling Limits of Constant-Stepsize SGD at Flat Minima
Jul 17, 2026Jingyi Zhang, Cheng Mao, Debankur MukherjeeStochastic Gradient Descent Step Accuracy
Georgia Institute of Technology
Dynamics of Gradient Descent with Large Step Size Near a Manifold of Flat Minima
Jul 9, 2026Lachlan Ewen MacDonald, René VidalGradient Descent Hessian
Innovation in Data Engineering and Science (IDEAS) · University of Pennsylvania Pennsylvania, PA 19104
Smooth $\%$ MinMax: A Differentiable Relaxation for Codon Harmonization
Jul 4, 2026Yoonho Jeong, Hyunwoo Choi, Ryan Fernandez Medina Hariri +3Feature Alignment Flat Minima
Department of Chemistry, KAIST, Daejeon 34141, Korea. · School of Chemistry and Chemical Engineering, Highfield Campus, University of Southampton, Southampton SO17 1BJ, United Kingdom.
Improving Sparse-View 3DGS Generalization via Flat Minima Optimization
Jul 1, 2026Kangmin Seo, Sangeek Hyun, MinKyu Lee +1Sparse-View Novel View Synthesis
Sungkyunkwan University, South Korea
Minimizing Quantized Semantic Age of Information (QSAoI) in Foundation Model-Based Semantic Communications
Jun 30, 2026Huanyu Zhang, Yulin Hu, Xiaopeng Yuan +2Semantic Communications Source-Channel Coding
INDA Chair, RWTH Aachen University, Germany · School of Electronic Information, Wuhan University, China · Department of Digital Communication Systems, Ruhr University Bochum, Germany
Local-Minima-Preserving Continuous Relaxation of Ising Problems
Jun 29, 2026Debraj Banerjee, Santanu Mahapatra, Kunal N. ChaudhurySpatial Photonic Ising Machines Convex Relaxation
Department of Electrical Engineering, Indian Institute of Science, Bangalore, India · Department of Electronic Systems Engineering, Indian Institute of Science, Bangalore, India
Closed-Form Steepest Descent Direction toward Flat Minima: Reducing Upper Bounds on the Loss Hessian Eigenspectrum in Neural Networks
Jun 27, 2026Yuto Omae, Kazuki Sakai, Yohei Kakimoto +3Hessian Gradient Descent
Nihon University (Japan) · National Institute of Technology, Nagaoka College (Japan) · Tokyo City University (Japan)
Noise-Debiased Thermodynamic Variance for Local Learning Coefficient Probes
Jun 21, 2026Yingjia CaiAnisotropic Loss Landscapes Singular Learning Theory
Geometry-Aware Online Scheduling for LLM Serving: From Theoretical Bound to System Practice
Jun 21, 2026Li Kong, Qi Qi, Yinyu Ye +1Large Language Model Serving Schedulers
Gaoling School of Artificial Intelligence Renmin University of China Beijing, China · Department of Management Science and Engineering Stanford University Stanford, CA 94305-4026 · Department of Industrial Engineering and Decision Analytics HKUST Hongkong, China
Fisher-Geometric Sharpness and the Implicit Bias of SGD toward Flat Minima
Jun 18, 2026Md Sakir Ahmed, Kumaresh Sarmah, Hemen DuttaStochastic Gradient Descent Sharpness-Aware Minimization
Department of Electronics and Communication Technology Gauhati University Guwahati, Assam, India · Department of Mathematics Gauhati University Guwahati, Assam, India
Functional Gradient Descent with Adaptive Representations
Jun 15, 2026Daniel Csillag, Rodrigo Schuller, Pedro Dall'Antonia +3Gradient Descent Convex Loss
FGV EMAp · IMPA · Stanford University
Squeeze-Release: Iterative Pruning with Exact Structural Minimization
Jun 12, 2026Roman Denkin, Ida Akerholm, Prashant Singh +1Unstructured Pruning Batch Normalization
Department of Information Technology, Uppsala University, Uppsala, Sweden · Science for Life Laboratory, Uppsala University, Uppsala, Sweden
Capability Minimization as a Safety Primitive: Risk-Aware Causal Gating for Least-Privilege LLM Agents
Jun 11, 2026Laxmipriya Ganesh Iyer, Rahul Suresh BabuConformal Risk Control Large Language Model Agents
Independent Researcher
Avoiding Exponential Blow-Up in Distributive Lattice Submodular Minimization
Jun 8, 2026Ishant ShanuSubmodular Functions Lattice
Local Search on Vertex Coloring for Bipartite Graphs
Jun 8, 2026Johanna GasseLarge Neighborhood Search Acyclic Graphs
Fachgebiet Algorithm Engineering der Digital-Engineering-Fakultät der Universität Potsdam
Learning-Augmented Online Minimization with Dual Predictions
Jun 3, 2026Christian Coester, Alexa Tudose, Alexander TuroczyLearning-Augmented Algorithms Flat Minima
University of Oxford, United Kingdom
A Geometric Characterization of the Stationary Plateau for Two-Layer Neural Networks
Jun 3, 2026Tian Ding, Dawei Li, Ruoyu SunTwo-Layer Neural Networks Anisotropic Loss Landscapes
Shenzhen International Center of Industrial and Applied Mathematics, Shenzhen Research Institute of Big Data, China; Shenzhen Loop Area Institute, China; AutoKernel, China. · Shenzhen Loop Area Institute, China · AutoKernel, China +3
Doing well with less! On Sampling Techniques for Empirical Pairwise Loss Estimation/Minimization
Jun 1, 2026Louise Davy, Stephan Clémençon, Charlotte LaclauOptimal Sample Complexity Loss Function
IDS, LTCI Télécom Paris Palaiseau, France
Entropy Minimization without Model Collapse: Mitigating Prediction Bias in Medical Imaging
Jun 1, 2026Tim Nielen, Sameer Ambekar, Johannes Kiechle +2Stable Test-Time Adaptation Test-Time Adaptation
School of Computation, Information and Technology, Technical University of Munich, Germany · Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich, Germany · Munich Center for Machine Learning (MCML) +3
Inconsistency-Aware Minimization: Improving Generalization with Unlabeled Data
May 29, 2026Hee-Sung Kim, Hyeonseong Kim, Sungyoon LeeUnlabeled Data Flat Minima
Department of Computer Science, Hanyang University, Seoul, Korea · Agency for Defense Development, Daejeon, Korea
Towards the Connection between Activation Sparsity and Flat Minima
May 25, 2026Ze Peng, Jian Zhang, Lei Qi +2Sparsity Modern Deep Networks
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China · Institute of Brain-Machine Interface, Nanjing University, Nanjing 210023, China · School of Computer Science and Engineering, Southeast University, Nanjing 211189, China
PMF-CL: Pareto-Minimal-Forgetting Continual Learner for Conflicting Tasks
May 18, 2026Srijith Nair, Atilla Eryilmaz, Jia LiuReplay-Based Continual Learning Catastrophic Forgetting
Department of Electrical and Computer Engineering The Ohio State University Columbus, OH 43201
Proximal basin hopping: global optimization with guarantees
May 18, 2026Guillaume Lauga, Cesare Molinari, Samuel VaiterProximal Operators Flat Minima
LJAD · LJAD Université Côte d’Azur France · MaLGA +3
Navigating Potholes with Geometry-Aware Sharpness Minimization
May 15, 2026Simon Dufort-Labbé, Mehrab Hamidi, Razvan Pascanu +3Sharpness-Aware Minimization Flat Minima
Mila, Université de Montréal · Samsung – SAIL Montreal
Don't Stop Me Yet: Sampling Loss Minima via Dissipative Riemannian Mechanics
May 14, 2026Albert Kjøller Jacobsen, Leo Uhre Jakobsen, Johanna Marie Gegenfurtner +1Reparameterization Flat Minima
Section for Cognitive Systems, Technical University of Denmark · Center for Quantum Information Physics, New York University
Stochastic global optimization of continuous functions via random walks on Grassmannians
May 13, 2026Kartik Gupta, Stephen D. Miller, Pradeep Ravikumar +1Stochastic Optimization Flat Minima
Revisiting Policy Gradients for Restricted Policy Classes: Escaping Myopic Local Optima with $k$ -step Policy Gradients
May 11, 2026Alex DeWeese, Guannan QuPolicy Gradient Optimal Policies
Department of Electrical and Computer Engineering Carnegie Mellon University Pittsburgh, PA 15213-3890, USA
On the global convergence of gradient descent for wide shallow models with bounded nonlinearities
May 11, 2026Romain Petit, Clarice Poon, Gabriel PeyréGradient Descent Flat Minima
CNRS and ENS, PSL Universit´e · Mathematics Institute, University of Warwick
Select-then-differentiate: Solving Bilevel Optimization with Manifold Lower-level Solution Sets
May 9, 2026Saeed Masiha, Zebang Shen, Negar Kiyavash +1Bilevel Optimization Flat Minima
EPFL School of Management of Technology, Station 5, 1015 Lausanne, Switzerland · ETH Department of Computer Science, Universitätstrasse 6, 8092 Zürich, Switzerland
An Interpretable and Scalable Framework for Evaluating Large Language Models
May 7, 2026Xinhao Qu, Qiang Heng, Hao Zeng +1Large Language Model Evaluation Item Response Theory
Department of Statistics, University of California, Riverside · School of Statistics and Data Science, Southeast University · Department of Statistics and Data Science, Southern University of Science and Technology
Super-Level-Set Regression: Conditional Quantiles via Volume Minimization
May 7, 2026Sacha Braun, Michael I. Jordan, Francis BachQuantile Regression Conditional Distribution
Sierra team, Inria Paris, France · Departments of EECS and Statistics, UC Berkeley, USA
Backdoor Mitigation in Object Detection via Adversarial Fine-Tuning
May 7, 2026Kealan Dunnett, Reza Arablouei, Dimity Miller +2Clean Label Backdoor Attack Object Detection
Queensland University of Technology, Brisbane Australia · CSIRO, Pullenvale QLD, Australia
Predicting missing values: A good idea?
May 5, 2026Stef van BuurenImputation Missing Data
TNO Netherlands Organization for Applied Scientific Research, Leiden · Dept. of Methodology & Statistics, University of Utrecht
When Flat Minima Fail: Characterizing INT4 Quantization Collapse After FP32 Convergence
Apr 16, 2026Marcus ArmstrongPost-Training Quantization Quantizer
Department of Computer Science University of Houston Houston, TX 77004
Decentralized SGD with Controlled Disagreement Finds Flatter Minima
Feb 2, 2026Zesen Wang, Mikael JohanssonDecentralized Learning Stochastic Gradient Descent
Division of Decision and Control Systems KTH Royal Institute of Technology
Comparison of D-Wave Quantum Annealing and Gibbs Monte Carlo for Sampling from a Probability Distribution of a Restricted Boltzmann Machine
Aug 13, 2025Abdelmoula El-Yazizi, Yaroslav KoshkaQuantum Annealing Monte Carlo
Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762, USA
A Morse-Bott Framework for Blind Inverse Problems: Local Recovery Guarantees and the Failure of the MAP
Aug 4, 2025Minh-Hai Nguyen, Edouard Pauwels, Pierre WeissBayesian Inverse Problems Gaussian Deblurring
IRIT, Toulouse University, Toulouse, France · Toulouse School of Economics, Universit´e Toulouse Capitole et IUF, Toulouse, France
Temporal horizons in forecasting: a performance-learnability trade-off
Jun 4, 2025Pau Vilimelis Aceituno, Jack William Miller, Noah Marti +2Autoregressive Model Dynamical Systems
Institute of Neuroinformatics, ETH Zürich and University of Zürich, Winterhurerstrasse 190, Zürich 8057, Switzerland · School of Computing, National Australian University, 108 North Rd, Acton ACT 2601, Australia · Unit of Land Change Science, Swiss Federal Research Institute for Forest, Snow and Landscape Zürcherstrasse 111, Birmensdorf 8903, Switzerland
Favorability of Loss Landscape with Weight Decay Requires Both Large Overparametrization and Initialization
May 28, 2025Etienne Boursier, Matthew Bowditch, Matthias Englert +1Anisotropic Loss Landscapes Overparameterization
INRIA LMO, Université Paris-Saclay Orsay, France · Mathematics Institute University of Warwick Coventry, UK · Department of Computer Science University of Warwick Coventry, UK
Minimisation of Quasar-Convex Functions Using Random Zeroth-Order Oracles
May 4, 2025Amir Ali Farzin, Yuen-Man Pun, Philipp Braun +1Nonconvex Zeroth-Order
School of Engineering Australian National University · Department of Electrical and Electronic Engineering University of Melbourne · Department of Electrical and Electronic Engineering22 University of Melbourne

Oct 1, 2026, cs.LGJ/K move · Enter open · S save

Removing spurious minima for planar features by skip connections

Jakob Paul Zimmermann, Moritz Grillo, Andrei Balakin, Georg Loho

Freie Universität Berlin · Technische Universität Berlin

Flat Minima Skip Model Weights Neurons Planar Removal

Open paper AskPDF arXiv

Understanding loss landscapes is central to explaining neural-network training, yet their structure remains only partially understood even in simple models. We study the Gaussian population loss of shallow, bias-free ReLU networks in the teacher--student setting. This provides a simple model for studying essential aspects such as feature learning and overparameterization. For teacher networks with positive output weights and planar features, we show that including a learned linear skip removes all spurious local minima with non-negative student output weights once the student network is at least as wide as the teacher network. In contrast, without the skip, we construct a fixed teacher network with positive output weights and only three hidden neurons in input dimension two whose spurious local minima persist at every student width at least three. Thus, a learned linear skip can remove spurious minima that persist under arbitrary overparameterization. Furthermore, we show that a positive output weight student network always learns the subspace spanned by the teacher features: student features at local minima with non-negative student output weights lie in the span of the teacher features. For ReLU networks in two dimensions, even heavily overparameterized student networks have effective width controlled by the teacher width: every critical point with positive student output weights has at most twice as many distinct student feature directions as teacher neurons. Finally, we transfer the benignity result to empirical minima over parameter balls of any prescribed radius, with the required sampling accuracy depending on that radius.

Momentum

Related topics

Latest papers 55