Stochastic Optimization Convergence

Ask about this topic

Momentum

21 papers in the last four weeks, against 2 the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 141

All topics
CardsList

  1. Stochastic Gradient Descent Ascent is Suboptimal for Nonconvex-PL Min-Max Games

    Oct 6, 2026Junsoo HaOptimization Convergence AnalysisMin-Max Optimization

  2. A Single-Loop, Constant-Batch First-Order Penalty Method for Stochastic Bilevel Optimization

    Oct 5, 2026Xingyu Chen, Ming Yang, Quanqi Hu +1Bilevel OptimizationStochastic Optimization Convergence

    Department of Computer Science and Engineering Texas A&M University, College Station, USA

  3. Gaussian Limits for SGD Without Stationary Moments

    Oct 4, 2026Xiaoli Li, Wei Biao WuStochastic Optimization ConvergenceStochastic Gradient Descent

    University of Chicago

  4. Smoothed Gradient Method for Nonconvex Federated Stochastic Bilevel Optimization

    Oct 4, 2026Xinwen Zhang, Peiran Yu, Zhaosong Lu +1Gradient DescentNonconvex Stochastic Optimization

    Temple University · University of Texas at Arlington · University of Minnesota

  5. Convergence Analysis of STORM Under Different Geometries

    Oct 1, 2026Wei Jiang, Yibo Wang, Wenhao Yang +3Stochastic OptimizationLast-Iterate Convergence

    School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China

  6. Optimal Momentum Methods for Stochastic Multilevel Compositional Optimization

    Oct 1, 2026Wei Jiang, Rui Yan, Sifan Yang +3Stochastic OptimizationMomentum Methods

    School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China · School of Software Technology, Zhejiang University, Ningbo, China

  7. Exact information accounting for SGD methods

    Sep 30, 2026Akshay BalsubramaniNeural Network GeneralizationStochastic Optimization Convergence

  8. Convergence of Practical Muon with Finite Newton-Schulz Iterations and Nesterov Momentum

    Sep 30, 2026Hanyng Peng, Hui Wang, Yue YuNewton-Schulz IterationGradient Descent

    Pengcheng Laboratory, Shenzhen, China

  9. Parameter-Free Zeroth-Order Optimization with Ellipsoidal Sampling

    Sep 29, 2026Alexander Sholokhov, Alexander RogozinZeroth-Order OptimizationCovariance Matrix Adaptation Evolution Strategy

    Moscow Institute of Physics and Technology, Russia

  10. Adam under Generalized Smoothness with Second-Moment-Type Stochastic Gradients

    Sep 29, 2026Ruinan Jin, Difei Cheng, Ling Chen +3Convergence AnalysisStochastic Optimization Convergence

    Mohamed bin Zayed University of Artificial Intelligence · Aerospace Information Technology University · The Ohio State University +2

  11. Second-Moment Stochastic Approximation Methods

    Sep 29, 2026Tao Jiang, Lin XiaoStochastic ApproximationMuon Optimizer

    Fundamental AI Research (FAIR), Meta Superintelligence Labs

  12. Ordinary Nonconvex SGD under Distance-Dependent Moments: Finite-Horizon Stationarity and Nagaev Bounds

    Sep 24, 2026Wei Biao WuNonconvex Stochastic OptimizationStochastic Optimization Convergence

  13. A Contraction Framework for Stochastic Operators with Bootstrapping: Application to TD Learning

    Sep 24, 2026Ids van der Werf, Sergio Rozada, Antonio G. MarquesStochastic ApproximationTemporal-Difference Learning

    Delft University of Technology, Dep. Microelectronics, Delft, The Netherlands

  14. Precise Convergence Speed of Clipped SGD

    Sep 24, 2026David A. R. RobinGradient ClippingNonconvex Stochastic Optimization

    LAMSADE, Université Paris-Dauphine, PSL Research University

  15. Gradient Descent with Stochastic Subspaces via Persistence of Memory

    Sep 16, 2026Subhroshekhar Ghosh, Clement Z. Q. Ng, Pierre-Louis Poirion +1Gradient DescentStochastic Optimization Convergence

    Department of Mathematics, National University of Singapore, Singapore · Center for Advanced Intelligence Project, RIKEN, Tokyo, Japan · Department of Mathematical Informatics, The University of Tokyo, Tokyo, Japan

  16. Bridging the Gap Between Homogeneous and Heterogeneous Asynchronous Optimization Is Surprisingly Difficult

    Sep 15, 2026Alexander TyurinDistributed OptimizationStochastic Optimization Convergence

    AXXX, Moscow, Russia · Applied AI Institute, Moscow, Russia

  17. Same Flow, Different Paths: Variance Reduction in Flow Matching

    Sep 15, 2026Alexander TyurinFlow MatchingStochastic Optimization Convergence

    AXXX, Moscow, Russia · Applied AI Institute, Moscow, Russia

  18. How I learned to stop worrying and love StopGrads: Stationarity, Convergence, and a case study on Flow Map Learning

    Sep 14, 2026Max W. Shen, Mark Goldstein, Zichu Wang +2Flow Map LearningStochastic Optimization Convergence

    Flatiron Institute · NYU

  19. High-Probability Convergence of SGD via Batched Updates

    Sep 14, 2026Feng Zhu, Robert W. Heath Jr., Aritra MitraCommunication-Efficient Distributed TrainingStochastic Optimization Convergence

  20. Quantile-based Loss Filtering for Outlier-Robust Stochastic Gradient Descent

    Sep 14, 2026Jamie Haddock, Anna Ma, Elizaveta RebrovaStochastic OptimizationStochastic Optimization Convergence

    Department of Mathematics, Harvey Mudd College, Claremont, CA 91711-3116, USA. · Department of Mathematics, University of California, Irvine, CA 92617, USA. · Mathematical Institute, University of Oxford, Oxford, OX1 2JD, UK.

  21. Almost Sure Convergence Analysis of Stochastic Gradient Methods with Clipping and Additive Noise

    Sep 10, 2026Amartya Mukherjee, Jun LiuStochastic OptimizationGradient Clipping

    Department of Applied Mathematics, University of Waterloo, Waterloo, Ontario, Canada N2L 3G1

  22. The Exact Time-Uniform Rate Frontier for Stochastic Gradient Descent on Smooth Convex Objectives

    Sep 8, 2026Ruijie Li, Kang Chen, Tianyu WangLearning Rate SchedulingLast-Iterate Convergence

    School of Data Science, Fudan University · Shanghai Center for Mathematical Sciences, Fudan University

  23. How to Make the Gradient Mapping Small for Constrained Stochastic Min-Max Problems and Beyond

    Sep 8, 2026Ahmet AlacaogluStochastic OptimizationMin-Max Optimization

    Department of Mathematics, University of British Columbia

  24. Convergence rates for the RMSprop optimizer with full control of the hyperparameters

    Aug 31, 2026Steffen Dereich, Arnulf JentzenStochastic ApproximationStochastic Optimization Convergence

  25. A Tight Lower Bound for Smooth Nonconvex Stochastic Optimization with Bounded Gradient Noise

    Aug 10, 2026Jikai JinStochastic OptimizationNonconvex Stochastic Optimization

  26. Optimized Certainty Equivalent Risk Minimization Using Samples: Algorithms, Convergence Rates, and Applications

    Aug 7, 2026Sumedh Gupte, Prashanth L. A., Sanjay P. BhatStochastic Optimization ConvergencePortfolio Optimization

    TCS Research, IIIT-H Research Park, Hyderabad, 500032, India · Department of Computer Science and Engineering, Indian Institute of Technology Madras, Chennai, 600036, India

  27. On Same-Sample and Independent-Sample Stochastic Extragradient for Monotone Variational Inequalities

    Aug 6, 2026TaeHo Yoon, Nicolas LoizouStochastic OptimizationLast-Iterate Convergence

    Department of Applied Mathematics and Statistics & Mathematical Institute for Data Science, Johns Hopkins University.

  28. A proximal subgradient method for nonconvex stochastic optimization under the Kurdyka-Łojasiewicz condition

    Aug 5, 2026Felipe Atenas, Alejandro Jofré, Pedro Pérez-Aros +1Stochastic OptimizationNonconvex Stochastic Optimization

    Centro de Modelamiento Matem´atico (CNRS IRL2807), Universidad de Chile, Santiago, Chile · Departamento de Ingenier´ıa Matem´atica and Centro de Modelamiento Matem´atico (CNRS IRL2807), Universidad de Chile, Santiago, Chile · Department of Mathematics, University of Alicante, Alicante, Spain

  29. On MUON optimization: From non-convergence to an error analysis with Polar Express and the Newton-Schulz polynomial from implementations

    Aug 5, 2026Thang Do, Steffen Dereich, Arnulf JentzenStochastic OptimizationNewton-Schulz Iteration

    School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), China · Department of Probability and Statistics, Institute of Mathematics, Vietnam Academy of Science and Technology, Vietnam · Institute for Mathematical Stochastics, Faculty of Mathematics and Computer Science, University of Münster, Germany +2

  30. Particle-based Generalised Stochastic Optimisation

    Aug 3, 2026Jiechen Jackie Zhang, O. Deniz AkyildizStochastic OptimizationLangevin Dynamics

    Department of Mathematics, Imperial College London