Optimization Convergence Analysis

Overview

The field studies whether optimization methods converge, what they converge to, and how their error or gradient changes with iterations. Researchers use tools such as Lyapunov functions, smoothness and error-bound conditions, dynamical-systems analysis, and oracle-complexity bounds to examine gradient and adaptive methods, stochastic and distributed optimization, and fixed-point or game-solving algorithms. Guarantees distinguish settings such as convex versus nonconvex objectives, different noise models, and best-iterate versus last-iterate performance. A central tension is that useful guarantees often require restrictive assumptions or small step sizes, while practical behavior can include oscillations, cycles, and gaps between worst-case bounds and observed performance.

Latest papers

CardsList

78 papers4 papers in the last four weeks, down 56% on the four weeks before. 0.0% of all new papers.

  1. On the Convergence of Continuous-Time AdamW

    Nov 16, 2026Bangjie Wang, Ning NingOptimization Convergence AnalysisAdaptive Gradient Methods

  2. The Polyak-Łojasiewicz Paper: A 10-Year Retrospective

    Nov 13, 2026Hamed Karimi, Julie Nutini, Mark SchmidtOptimization Convergence Analysis

  3. Global O(k−2)\mathcal{O}\left(k^{-2}\right) Convergence of Multilevel Gradient-Regularized Newton Methods

    Oct 26, 2026Nick Tsipinakis, Matthias VoigtOptimization Convergence AnalysisSecond-Order Optimization

  4. Curvature-Scaled Sharpness-Aware Minimization: Convergence Beyond Smoothness

    Oct 21, 2026Dimitris Oikonomou, Nicolas LoizouSharpness-Aware MinimizationOptimization Convergence Analysis

  5. Controlled-Minimax: Stabilizing Adversarial Optimization via Control Theory

    Oct 9, 2026Tianjiao Luo, Jianfei Chen, Jun ZhuMin-Max OptimizationOptimization Convergence Analysis

  6. Convergence Analysis of STORM Under Different Geometries

    Oct 1, 2026Wei Jiang, Yibo Wang, Wenhao Yang +3Stochastic OptimizationLast-Iterate Convergence

  7. The Row Normalization Puzzle in Muon

    Sep 30, 2026Jiayu Zhang, Tianyi LinStochastic OptimizationOptimization Convergence Analysis

  8. Strict-Saddle Landscapes and Multi-Rank Geometry in Low-Tubal-Rank Tensor Sensing

    Sep 29, 2026Eugene Agyei-Kodie, Longxiu Huang, Shuang Li +1Nonconvex OptimizationOptimization Convergence Analysis

  9. Convergence guarantees for Muon: New parameter regimes and generalizations

    Sep 24, 2026Arthur C. B. de Oliveira, Dhruv D. Jatkar, Guilherme S. Vicinansa +1Muon OptimizerOptimization Convergence Analysis

  10. Complexities of Weak Proximal Oracle Methods for Composite Convex Optimization

    Sep 21, 2026Dan GarberOracle ComplexityOptimization Convergence Analysis

  11. Tight Majorizations and Convergence Rates of Nuclear Norm Minimization IRLS

    Aug 24, 2026Christian Kümmerle, Tomas Masak, Dominik StögerNuclear Norm MinimizationOptimization Convergence Analysis

  12. A Local-Linearly Convergent Algorithm for Nonconvex Equality-Constrained Optimization

    Aug 12, 2026Frank E. Curtis, Lingjun Guo, Daniel P. RobinsonOptimization Convergence AnalysisNonconvex Optimization

  13. Tight Nonasymptotic Local Convergence of Sinkhorn-Knopp

    Aug 12, 2026Wenzhi Gao, Zhaonan Qu, Yinyu Ye +1Optimization Convergence Analysis

  14. Non-KKT Accumulation in Entropic Mirror Descent

    Aug 3, 2026Kuangyu Ding, Kim-Chuan TohOptimization Convergence AnalysisMirror Descent

  15. Global Convergence of DGM and PINN Algorithms for Solving Nonlinear PDEs

    Jul 27, 2026Justin Sirignano, Konstantinos Spiliopoulos, Samuel CohenNeural PDE SolversOptimization Convergence Analysis

  16. When Rates Are Geometric: Rate-Certificate Transfer for Contact Splittings in Optimization

    Jul 26, 2026George A KevrekidisOptimization Convergence AnalysisHamiltonian Dynamics

  17. Convergence analysis of a family of Zermelo-type iterations for the Bradley--Terry model

    Jul 24, 2026Ruijian Han, Ding Lu, Yiming XuOptimization Convergence AnalysisBradley–Terry Model

  18. What's in a Smoothness Constant? Tighter Rates for Local SGD with Bounded Second-order Heterogeneity

    Jul 16, 2026Kumar Kshitij Patel, Rustem Islamov, Sebastian U Stich +3Optimization Convergence AnalysisStochastic Optimization Convergence

  19. Sharper Analysis of Single-Loop Methods for Bilevel Optimization

    Jul 11, 2026Yubo Zhou, Jun Shu, Luo Luo +4Implicit DifferentiationOptimization Convergence Analysis

  20. Understanding Schedule-Free Methods in Nonconvex Optimization: Rate Guarantees and Escaping Saddles

    Jul 10, 2026Jiseok Chae, Donghwan KimLearning Rate SchedulingOptimization Convergence Analysis

  21. On the Convergence of Self-Improving Online LLM Alignment

    Jun 30, 2026Xudong Wu, Pangpang Liu, Vaneet Aggarwal +1LLM AlignmentOptimization Convergence Analysis