Loss Landscape Geometry

Ask about this topic

Overview

Loss landscape geometry studies the structure of objective functions, including minima, saddle points, curvature, symmetries, and the connectedness of low-loss regions. Researchers analyze these features with tools such as Hessian spectra, sharpness measures, landscape visualization, and theoretical models, then test how they affect gradient methods, stochastic optimization, and search algorithms. The work spans neural-network training and settings such as matrix factorization, forecasting, quantum optimization, and inverse problems. A central tension is that flatness or benign minima do not by themselves guarantee good generalization or easy optimization: outcomes also depend on parameterization, data, initialization, and optimizer dynamics.

Latest papers

CardsList

34 papers5 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

  1. A General Topological Symmetry Index for Critical Orbits In Neural Network Loss Landscapes

    Oct 20, 2026Jingzhou LiuLoss Landscape GeometryNeural Network Optimization

  2. Global Optimality of Pose Graph Optimization

    Oct 10, 2026Shuan Ding, Ruishuang Chen, Mengfan Cao +1Pose Graph OptimizationRobotic Optimization

    Guangdong Provincial Key Laboratory of FullyActuated System Control Theory and Technology, and the School of Automation and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen 518055, China · Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China

  3. Optimization Geometry of QAOA and Variational Quantum Algorithms

    Oct 4, 2026Vojtěch Novák, Ivan Zelinka, Silvie Illésová +2Quantum Approximate Optimization AlgorithmVariational Quantum Algorithms

    Department of Computer Science, Faculty of Electrical Engineering and Computer Science, VSB – Technical University of Ostrava, Ostrava, Czech Republic · IT4Innovations National Supercomputing Center, VSB – Technical University of Ostrava, 708 00 Ostrava, Czech Republic · Department of Informatics and Statistics, Marine Research Institute, Klaipeda University, Klaipeda, Lithuania +3

  4. Misalignment of Low-Loss Regions Causes Grokking

    Sep 30, 2026Yongding Tian, Zaid Al-Ars, Maksim Kitsak +1Loss Landscape GeometryGrokking

    Computer Engineering Lab Delft University of Technology Delft, NL · HDL TypeTech Delft, NL · Network and Architecture Service Delft University of Technology Delft, NL +1

  5. How Does Local Landscape Geometry Evolve in Language Model Pre-Training?

    Sep 30, 2026Zhanpeng Zhou, Yuhan Sun, Bingrui Li +4Language Model PretrainingLearning Rate Scheduling

    Shanghai Jiao Tong University · Tsinghua University · Peking University

  6. Towards Understanding Momentum Acceleration in River-Valley Loss Landscape

    Sep 25, 2026Miao Lu, Zeyu Bian, Kaiyue Wen +4Momentum MethodsGradient Descent Dynamics

    Stanford University · University of California, San Diego · University of Chicago +2

  7. When is global evolutionary search useful for variational quantum algorithms? A landscape-first study

    Sep 13, 2026Vojtěch Novák, Ivan ZelinkaVariational Quantum AlgorithmsEvolutionary Optimization

    Department of Computer Science, Faculty of Electrical Engineering and Computer Science, VSB - Technical University of Ostrava, Ostrava, Czech Republic · IT4Innovations National Supercomputing Center, VSB - Technical University of Ostrava, 708 00 Ostrava, Czech Republic

  8. Benign Loss Landscapes Can Coexist with Worst-Case Hardness

    Sep 11, 2026Zach Furman, Stephan Wäldchen, Yangda Bei +1Tensor NetworksStatistical Learning Theory

    University of Melbourne · Iliad

  9. Teacher Geometry Shapes Learnability in Teacher-Student Networks

    Sep 9, 2026Kai J. Sandbrink, Flavio Martinelli, Alexander van Meegen +2Neural Network OptimizationNeural Network Training Dynamics

    Department of Experimental Psychology, University of Oxford · Department of Physics, RWTH Aachen

  10. On the Abundance of Critical Points of the t-SNE Energy

    Sep 3, 2026Nakul Haridas, Ryan MurrayDimensionality ReductionLoss Landscape Geometry

    Department of Mathematics North Carolina State University Raleigh, North Carolina, USA

  11. Dataset Complexity Shapes Finite-Distance Loss Geometry in Neural Networks

    Aug 23, 2026Jaeyong Bae, Hawoong JeongLoss LandscapeLoss Landscape Geometry

    Department of Physics, Korea Advanced Institute of Science and Technology, Daejeon, Korea · Center of Complex Systems, Korea Advanced Institute of Science and Technology, Daejeon, Korea

  12. Loss Landscape Features That Make Adam Stall: Definitions, Estimators, and the Preconditioned Hessian View

    Aug 23, 2026Rodion PodorozhnyLoss LandscapeAdaptive Gradient Methods

    Texas State University

  13. Kähler landscapes for complex neural network descents and guarantees including a search and destroy of the Calabi-Yau manifold

    Aug 20, 2026Andrew GracykRiemannian OptimizationGradient Descent

    Department of Mathematics, Purdue University, West Lafayette, IN 47907, USA

  14. Effects of width-dependent model hyperparameters and ℓ2\ell_2-regularization on the loss landscape of two-layer ReLU networks

    Jul 18, 2026Haruka Eshima, Makoto YamadaReLU Neural NetworksWeight Decay

    Okinawa Institute of Science and Technology, Okinawa, Japan

  15. Fisher-Geometric Sharpness and the Implicit Bias of SGD toward Flat Minima

    Jun 18, 2026Md Sakir Ahmed, Kumaresh Sarmah, Hemen DuttaInformation GeometryImplicit Bias

    Department of Electronics and Communication Technology Gauhati University Guwahati, Assam, India · Department of Mathematics Gauhati University Guwahati, Assam, India

  16. Loss Landscape Diagnosis for Gradient-Based Gray-Scott System Inversion: Disentangling the Roles of PINN Components

    Jun 9, 2026Yan YangLoss LandscapeDynamical Systems

    Independent Researcher.

  17. Why Muon Outperforms Adam: A Curvature Perspective

    Jun 3, 2026Shuche Wang, Fengzhuo Zhang, Jiaxiang Li +2Loss Landscape SharpnessNeural Network Optimization

    National University of Singapore · Yale University · University of Minnesota

  18. A Geometric Characterization of the Stationary Plateau for Two-Layer Neural Networks

    Jun 3, 2026Tian Ding, Dawei Li, Ruoyu SunTwo-Layer Neural NetworksNeural Network Optimization

    Shenzhen International Center of Industrial and Applied Mathematics, Shenzhen Research Institute of Big Data, China; Shenzhen Loop Area Institute, China; AutoKernel, China. · Shenzhen Loop Area Institute, China · AutoKernel, China +3

  19. Exploiting weight-space symmetries for approximating curvature

    May 30, 2026Artem Artemev, Rui Xia, Benjamin M. Boyd +4Neural Network OptimizationSecond-Order Optimization

    MediaTek, Cambridge, UK · University of Cambridge, Cambridge, UK · Vector Institute, Toronto, Canada

  20. Worker Disagreement Reveals Sharp Directions in Local SGD

    May 26, 2026Tolga Dimlioglu, Kristi Topollai, Anna ChoromanskaNeural Network Training DynamicsLoss Landscape Geometry

    New York University, New York, USA

  21. Model Merging on Loss Landscape: A Geometry Perspective

    May 26, 2026Juanwu Lu, Anand Bhaskar, Brian Axelrod +2Task Vector MergingModel Merging

    Purdue University · Waymo LLC

  22. Navigating Potholes with Geometry-Aware Sharpness Minimization

    May 15, 2026Simon Dufort-Labbé, Mehrab Hamidi, Razvan Pascanu +3Sharpness-Aware MinimizationFlat Minima

    Mila, Université de Montréal · Samsung – SAIL Montreal

  23. The Role of Symmetry in Optimizing Overparameterized Networks

    Apr 28, 2026Kusha Sareen, Mohammad Pedramfar, Sékou-Oumar Kaba +2Neural Network OptimizationWeight-Space Analysis

    McGill University · Mila - Quebec Artificial Intelligence Institute

  24. Curvature-Aligned Probing for Local Loss-Landscape Stabilization

    Apr 16, 2026Nikita Kiselev, Andrey GrabovoyNeural Network OptimizationNeural Network Training Dynamics

    Department of Intelligent Systems Moscow Institute of Physics and Technology

  25. Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networks

    Apr 10, 2026Jie Huang, Bruno Loureiro, Stefano Sarao MannelliOverparameterizationLoss Landscape

    Physics, Chalmers University of Technology and University of Gothenburg · Ecole Normale Superieure, PSL & CNRS · Data Science and AI, Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg +1

  26. On the Loss Landscape Geometry of Regularized Deep Matrix Factorization: Uniqueness and Sharpness

    Mar 28, 2026Anil Kamber, Rahul ParhiLoss LandscapeLoss Landscape Geometry

    UC San Diego

  27. Reframing Long-Tailed Learning via Loss Landscape Geometry

    Mar 22, 2026Shenghan Chen, Yiming Liu, Yanzhen Wang +2Long-Tail LearningLoss Landscape Geometry

    Shandong University · Zhejiang Sci-Tech University

  28. Adapting Critic Match Loss Landscape Visualization to Off-policy Reinforcement Learning

    Mar 15, 2026Jingyi Liu, Jian Guo, Eberhard GillLoss Landscape GeometryRL Control

    Delft University of Technology, Delft, 2629 HS, the Netherlands

  29. Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrieval

    Feb 19, 2026Antoine Maillard, Tony Bonnaire, Giulio BiroliLoss LandscapeLoss Landscape Geometry

    INRIA Paris & DI ENS, PSL University, Paris, France. · Université Paris-Saclay, CNRS, Institut d’Astrophysique Spatiale, 91405 Orsay, France · Laboratoire de Physique de l’École normale supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris Cité, F-75005 Paris, France

  30. Early-Warning Signals of Grokking via Loss-Landscape Geometry

    Feb 19, 2026Yongzhong XuGrokkingNeural Network Generalization