Optimization Convergence Analysis

Ask about this topic

Overview

The field studies whether optimization methods converge, what they converge to, and how their error or gradient changes with iterations. Researchers use tools such as Lyapunov functions, smoothness and error-bound conditions, dynamical-systems analysis, and oracle-complexity bounds to examine gradient and adaptive methods, stochastic and distributed optimization, and fixed-point or game-solving algorithms. Guarantees distinguish settings such as convex versus nonconvex objectives, different noise models, and best-iterate versus last-iterate performance. A central tension is that useful guarantees often require restrictive assumptions or small step sizes, while practical behavior can include oscillations, cycles, and gaps between worst-case bounds and observed performance.

Latest papers

CardsList

75 papers8 papers in the last four weeks, up 60% on the four weeks before. 0.0% of all new papers.

  1. Global O(k−2)\mathcal{O}\left(k^{-2}\right) Convergence of Multilevel Gradient-Regularized Newton Methods

    Oct 26, 2026Nick Tsipinakis, Matthias VoigtOptimization Convergence AnalysisSecond-Order Optimization

    Faculty of Mathematics and Computer Science, UniDistance Suisse, Brig, Switzerland

  2. Curvature-Scaled Sharpness-Aware Minimization: Convergence Beyond Smoothness

    Oct 21, 2026Dimitris Oikonomou, Nicolas LoizouSharpness-Aware MinimizationOptimization Convergence Analysis

    Mathematical Institute for Data Science (MINDS), Johns Hopkins University · Department of Computer Science, Johns Hopkins University · Department of Applied Mathematics and Statistics, Johns Hopkins University

  3. Controlled-Minimax: Stabilizing Adversarial Optimization via Control Theory

    Oct 9, 2026Tianjiao Luo, Jianfei Chen, Jun ZhuMin-Max OptimizationOptimization Convergence Analysis

    Department of Computer Science and Technology, Tsinghua University, Beijing, China.

  4. Stochastic Gradient Descent Ascent is Suboptimal for Nonconvex-PŁ Min-Max Games

    Oct 6, 2026Junsoo HaOptimization Convergence AnalysisMin-Max Optimization

  5. Last-Iterate Convergence Rate of Normalized Gradient Descent under Hölder Smoothness

    Oct 5, 2026Yuki Takezawa, Eduard GorbunovLearning Rate SchedulingLast-Iterate Convergence

    Toyota Motor Corporation · MBZUAI

  6. Convergence Analysis of STORM Under Different Geometries

    Oct 1, 2026Wei Jiang, Yibo Wang, Wenhao Yang +3Stochastic OptimizationLast-Iterate Convergence

    School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China

  7. The Quantum Sphere: A Physically Realizable Optimization Benchmark with Provable Linear Convergence in White- and Black-Box Settings

    Sep 30, 2026Ofer M. ShirOptimization Convergence AnalysisBlack-Box Optimization

    Tel-Hai University, Israel

  8. The Row Normalization Puzzle in Muon

    Sep 30, 2026Jiayu Zhang, Tianyi LinStochastic OptimizationOptimization Convergence Analysis

    Department of Industrial Engineering and Operations Research Columbia University

  9. Strict-Saddle Landscapes and Multi-Rank Geometry in Low-Tubal-Rank Tensor Sensing

    Sep 29, 2026Eugene Agyei-Kodie, Longxiu Huang, Shuang Li +1Nonconvex OptimizationOptimization Convergence Analysis

    Department of Mathematics, Michigan State University, 619 Red Cedar, East Lansing, 48824, MI, USA. · Department of Computational Mathematics, Science and Engineering, Michigan State University, 428 S Shaw Ln, East Lansing, 48824, MI, USA. · Department of Electrical and Computer Engineering, Iowa State University, 2520 Osborn Dr, Ames, 50011, IA, USA.

  10. Convergence guarantees for Muon: New parameter regimes and generalizations

    Sep 24, 2026Arthur C. B. de Oliveira, Dhruv D. Jatkar, Guilherme S. Vicinansa +1Muon OptimizerOptimization Convergence Analysis

    Department of Electrical and Computer Engineering Northeastern University Boston, MA 02115 · Department of Telecommunications and Control Escola Politecnica, University of São Paulo São Paulo, SP 05508-010

  11. Complexities of Weak Proximal Oracle Methods for Composite Convex Optimization

    Sep 21, 2026Dan GarberOracle ComplexityOptimization Convergence Analysis

    Faculty of Data and Decision Sciences Technion - Israel Institute of Technology

  12. The fixed-point bundle method over product-of-simplex domains arising from game equilibria

    Sep 14, 2026Hongbo SunOptimization Convergence AnalysisNash Equilibrium Computation

    Hongbo Sun∗

  13. Silver Rate Is (Almost) Optimal for Gradient Descent

    Sep 8, 2026Yuhan Ye, Kaizhao LiuLearning Rate SchedulingOptimization Convergence Analysis

    MIT

  14. Projected Riemannian Gradient Descent for the Bures-Wasserstein Barycenter: Dimension-Independent Linear Convergence at Unit Step Size

    Sep 3, 2026A. AfhamRiemannian OptimizationOptimization Convergence Analysis

    Centre for Quantum Technologies, National University of Singapore, Singapore

  15. Tight Majorizations and Convergence Rates of Nuclear Norm Minimization IRLS

    Aug 24, 2026Christian Kümmerle, Tomas Masak, Dominik StögerNuclear Norm MinimizationOptimization Convergence Analysis

    School of Data, Mathematical, and Statistical Sciences, Department of Computer Science & Institute of Artificial Intelligence, University of Central Florida, Oviedo, FL 32816, USA · Institute for Statistics and Mathematics, Wirtschaftsuniversität Wien, 1020 Vienna, Austria · Department of Mathematics, KU Eichstätt-Ingolstadt, 85049 Ingolstadt, Germany

  16. A Local-Linearly Convergent Algorithm for Nonconvex Equality-Constrained Optimization

    Aug 12, 2026Frank E. Curtis, Lingjun Guo, Daniel P. RobinsonOptimization Convergence AnalysisNonconvex Optimization

    Department of Industrial and Systems Engineering, Lehigh University

  17. Tight Nonasymptotic Local Convergence of Sinkhorn-Knopp

    Aug 12, 2026Wenzhi Gao, Zhaonan Qu, Yinyu Ye +1Optimization Convergence Analysis

    ICME, Stanford University · Google · Department of Management Science and Engineering, Stanford University

  18. Stable Density Ridges: Consistency and Convergence of Subspace Constrained Mean Shift

    Aug 5, 2026Wanli QiaoOptimization Convergence Analysis

  19. Non-KKT Accumulation in Entropic Mirror Descent

    Aug 3, 2026Kuangyu Ding, Kim-Chuan TohOptimization Convergence AnalysisMirror Descent

    School of Industrial Engineering, Purdue University, West Lafayette, IN 47907, USA. · Department of Mathematics and Institute of Operations Research and Analytics, National University of Singapore, Singapore 119076.

  20. Global Convergence of DGM and PINN Algorithms for Solving Nonlinear PDEs

    Jul 27, 2026Justin Sirignano, Konstantinos Spiliopoulos, Samuel CohenNeural PDE SolversOptimization Convergence Analysis

    Mathematical Institute, University of Oxford · Department of Mathematics and Statistics, Boston University

  21. When Rates Are Geometric: Rate-Certificate Transfer for Contact Splittings in Optimization

    Jul 26, 2026George A KevrekidisOptimization Convergence AnalysisHamiltonian Dynamics

    Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD 21218, USA

  22. Convergence analysis of a family of Zermelo-type iterations for the Bradley--Terry model

    Jul 24, 2026Ruijian Han, Ding Lu, Yiming XuOptimization Convergence AnalysisBradley–Terry Model

    Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong SAR · Department of Mathematics, University of Kentucky, Lexington, KY, 40506, USA

  23. Barzilai-Borwein Fails Superlinear Convergence on an Open Set of Quadratics for Every Dimension n≥4n\geq 4

    Jul 23, 2026Dawei Li, Xiaotian Jiang, Mingyi HongOptimization Convergence Analysis

    University of Minnesota

  24. What's in a Smoothness Constant? Tighter Rates for Local SGD with Bounded Second-order Heterogeneity

    Jul 16, 2026Kumar Kshitij Patel, Rustem Islamov, Sebastian U Stich +3Optimization Convergence AnalysisStochastic Optimization Convergence

    Department of Mathematics and Computer Science University of Basel Basel, Switzerland 4001 · Simons Institute for the Theory of Computing, Berkeley · Department of Data Science New Jersey Institute of Technology Newark, NJ, USA 07102

  25. Sharper Analysis of Single-Loop Methods for Bilevel Optimization

    Jul 11, 2026Yubo Zhou, Jun Shu, Luo Luo +4Implicit DifferentiationOptimization Convergence Analysis

    School of Mathematics and Statistics, Xi’an Jiaotong University · School of Data Science, Fudan University · 3SGIT AI Lab, State Grid Corporation of China +1

  26. Understanding Schedule-Free Methods in Nonconvex Optimization: Rate Guarantees and Escaping Saddles

    Jul 10, 2026Jiseok Chae, Donghwan KimLearning Rate SchedulingOptimization Convergence Analysis

    Natural Science Research Institute KAIST Daejeon, Republic of Korea · Department of Mathematical Sciences KAIST Daejeon, Republic of Korea

  27. On the Convergence of Self-Improving Online LLM Alignment

    Jun 30, 2026Xudong Wu, Pangpang Liu, Vaneet Aggarwal +1LLM AlignmentOptimization Convergence Analysis

    The University of Hong Kong, Hong Kong SAR · Yale University, New Haven, CT, USA · Purdue University, West Lafayette, IN, USA

  28. How AI settled the complexity of the oldest SGD algorithm

    Jun 28, 2026Michał Dereziński, Xiaoyu DongOptimization Convergence AnalysisStochastic Gradient Descent

    University of Michigan · National University of Singapore

  29. Analysis of Parameter Settings for the Bat Algorithm Using Variance Evolution

    Jun 26, 2026Xin-She Yang, Mehmet KaramanogluExploration-Exploitation TradeoffEvolutionary Optimization

    Faculty of Science and Technology, Middlesex University, The Burroughs, London NW4 4BT, United Kingdom.

  30. Second-Order KKT Guarantees for Bregman ADMM in Nonconvex and Non-Lipschitz Optimization

    Jun 26, 2026Shuang Li, Zhihui Zhu, Qiuwei LiOptimization Convergence AnalysisDistributed Optimization

    Department of Electrical and Computer Engineering, Iowa State University, Ames, Iowa 50014, USA · Department of Computer Science and Engineering, Ohio State University, Columbus, Ohio 43210, USA · DAMO Academy, Alibaba Group US, Bellevue, Washington 98004, USA