Overview

Loss landscape geometry studies the structure of objective functions, including minima, saddle points, curvature, symmetries, and the connectedness of low-loss regions. Researchers analyze these features with tools such as Hessian spectra, sharpness measures, landscape visualization, and theoretical models, then test how they affect gradient methods, stochastic optimization, and search algorithms. The work spans neural-network training and settings such as matrix factorization, forecasting, quantum optimization, and inverse problems. A central tension is that flatness or benign minima do not by themselves guarantee good generalization or easy optimization: outcomes also depend on parameterization, data, initialization, and optimizer dynamics.

Latest papers

CardsList

38 papers4 papers in the last four weeks, down 20% on the four weeks before. 0.0% of all new papers.

  1. Conic Structure of the Cross-Entropy Loss at the Final Layer and Its Absence in Rectifier Backbones

    Nov 16, 2026Omer Nezih GerekLoss Landscape Geometry

  2. Two mathematical models of knowledge distillation

    Nov 12, 2026Audrey Xie, Ludwig Schmidt, John DuchiKnowledge DistillationLoss Landscape Geometry

  3. Physically Interpretable Basin Selection in Discrete-Variable Origami Design

    Nov 12, 2026Lucy Liu, Leon M. Kamp, Katia Bertoldi +1Loss Landscape GeometryDifferentiable Optimization

  4. Formalizing the Intrinsic Dimension of Objective Landscapes by a Generalized Notion of Co-dimension

    Oct 29, 2026Ernesto Araya, Gitta Kutyniok, Weilin Li +2Intrinsic DimensionalityLoss Landscape Geometry

  5. Global Optimality of Pose Graph Optimization

    Oct 10, 2026Shuan Ding, Ruishuang Chen, Mengfan Cao +1Pose Graph OptimizationRobotic Optimization

  6. Optimization Geometry of QAOA and Variational Quantum Algorithms

    Oct 4, 2026Vojtěch Novák, Ivan Zelinka, Silvie Illésová +2Quantum Approximate Optimization AlgorithmVariational Quantum Algorithms

  7. Misalignment of Low-Loss Regions Causes Grokking

    Sep 30, 2026Yongding Tian, Zaid Al-Ars, Maksim Kitsak +1Loss Landscape GeometryGrokking

  8. How Does Local Landscape Geometry Evolve in Language Model Pre-Training?

    Sep 30, 2026Zhanpeng Zhou, Yuhan Sun, Bingrui Li +4Language Model PretrainingLearning Rate Scheduling

  9. Towards Understanding Momentum Acceleration in River-Valley Loss Landscape

    Sep 25, 2026Miao Lu, Zeyu Bian, Kaiyue Wen +4Momentum MethodsGradient Descent Dynamics

  10. When is global evolutionary search useful for variational quantum algorithms? A landscape-first study

    Sep 13, 2026Vojtěch Novák, Ivan ZelinkaVariational Quantum AlgorithmsEvolutionary Optimization

  11. Benign Loss Landscapes Can Coexist with Worst-Case Hardness

    Sep 11, 2026Zach Furman, Stephan Wäldchen, Yangda Bei +1Tensor NetworksStatistical Learning Theory

  12. Teacher Geometry Shapes Learnability in Teacher-Student Networks

    Sep 9, 2026Kai J. Sandbrink, Flavio Martinelli, Alexander van Meegen +2Neural Network OptimizationNeural Network Training Dynamics

  13. On the Abundance of Critical Points of the t-SNE Energy

    Sep 3, 2026Nakul Haridas, Ryan MurrayDimensionality ReductionLoss Landscape Geometry

  14. Dataset Complexity Shapes Finite-Distance Loss Geometry in Neural Networks

    Aug 23, 2026Jaeyong Bae, Hawoong JeongLoss LandscapeLoss Landscape Geometry

  15. Fisher-Geometric Sharpness and the Implicit Bias of SGD toward Flat Minima

    Jun 18, 2026Md Sakir Ahmed, Kumaresh Sarmah, Hemen DuttaInformation GeometryImplicit Bias

  16. Why Muon Outperforms Adam: A Curvature Perspective

    Jun 3, 2026Shuche Wang, Fengzhuo Zhang, Jiaxiang Li +2Loss Landscape SharpnessNeural Network Optimization

  17. A Geometric Characterization of the Stationary Plateau for Two-Layer Neural Networks

    Jun 3, 2026Tian Ding, Dawei Li, Ruoyu SunTwo-Layer Neural NetworksNeural Network Optimization

  18. Exploiting weight-space symmetries for approximating curvature

    May 30, 2026Artem Artemev, Rui Xia, Benjamin M. Boyd +4Neural Network OptimizationSecond-Order Optimization

  19. Worker Disagreement Reveals Sharp Directions in Local SGD

    May 26, 2026Tolga Dimlioglu, Kristi Topollai, Anna ChoromanskaNeural Network Training DynamicsLoss Landscape Geometry

  20. Model Merging on Loss Landscape: A Geometry Perspective

    May 26, 2026Juanwu Lu, Anand Bhaskar, Brian Axelrod +2Task Vector MergingModel Merging

  21. Navigating Potholes with Geometry-Aware Sharpness Minimization

    May 15, 2026Simon Dufort-Labbé, Mehrab Hamidi, Razvan Pascanu +3Sharpness-Aware MinimizationFlat Minima

  22. The Role of Symmetry in Optimizing Overparameterized Networks

    Apr 28, 2026Kusha Sareen, Mohammad Pedramfar, Sékou-Oumar Kaba +2Neural Network OptimizationWeight-Space Analysis

  23. Curvature-Aligned Probing for Local Loss-Landscape Stabilization

    Apr 16, 2026Nikita Kiselev, Andrey GrabovoyNeural Network OptimizationNeural Network Training Dynamics

  24. Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networks

    Apr 10, 2026Jie Huang, Bruno Loureiro, Stefano Sarao MannelliOverparameterizationLoss Landscape