Stochastic Optimization Convergence

Ask about this topic

Momentum

21 papers in the last four weeks, against 2 the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 141

All topics
CardsList

  1. Uniform a priori bounds and error analysis for the Adam stochastic gradient descent optimization method

    Mar 19, 2026Steffen Dereich, Thang Do, Arnulf JentzenStochastic OptimizationStochastic Optimization Convergence

    Institute for Mathematical Stochastics, Faculty of Mathematics and Computer Science, University of Münster, Germany · School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), China · Department of Probability and Statistic, Institute of Mathematics, Vietnam Academy of Science and Technology, Vietnam +2

  2. Unbiased and Biased Variance-Reduced Forward-Reflected-Backward Splitting Methods for Stochastic Composite Inclusions

    Mar 16, 2026Quoc Tran-Dinh, Nghia Nguyen-TrungStochastic OptimizationStochastic Optimization Convergence

    Department of Statistics and Operations Research The University of North Carolina at Chapel Hill 318 Hanes Hall, UNC-Chapel Hill, NC 27599-3260.

  3. Convergence Rate of a Functional Learning Method for Contextual Stochastic Optimization

    Mar 13, 2026Noel Smith, Andrzej RuszczynskiStochastic OptimizationStochastic Approximation

    Department of Management Science and Information Systems, Rutgers University

  4. Adaptive Momentum and Nonlinear Damping for Neural Network Training

    Jan 30, 2026Aikaterini Karoni, Rajit Rajpal, Benedict Leimkuhler +1Momentum MethodsStochastic Optimization Convergence

    School of Mathematics University of Bristol · School of Mathematics University of Edinburgh · CERMICS, CNRS, École des Ponts, Institut Polytechnique de Paris; Inria Paris, France

  5. Provable Benefit of SignGD: A Minimal Model Under Heavy-Tailed Class Imbalance

    Nov 30, 2025Robin Yadav, Shuo Xie, Tianhao Wang +1Sign-Based OptimizationClass-Imbalanced Learning

    Toyota Technological Institute at Chicago · University of British Columbia · University of California, San Diego

  6. Non-Euclidean SGD for Structured Optimization: Unified Analysis and Improved Rates

    Nov 14, 2025Dmitry Kovalev, Ekaterina BorodichNonconvex Stochastic OptimizationStochastic Optimization Convergence

    Yandex Research

  7. Adam symmetry theorem: characterization of the convergence of the stochastic Adam optimizer

    Nov 10, 2025Steffen Dereich, Thang Do, Arnulf Jentzen +1Stochastic OptimizationStochastic Optimization Convergence

    Institute for Mathematical Stochastics, Faculty of Mathematics and Computer Science, University of Münster, Germany · School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), China · School of Data Science and School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), China +1

  8. Convergence Analysis of the ProbAbilistic Gradient Estimator Algorithm for Weakly Convex Finite-Sum Optimization

    Aug 31, 2025Laurent Condat, Peter RichtárikStochastic OptimizationNonconvex Stochastic Optimization

    King Abdullah University of Science and Technology (KAUST) Thuwal, Kingdom of Saudi Arabia

  9. Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation

    Aug 29, 2025Bangti Jin, Longjun WuNeural PDE SolversPDE Solving

    Department of Mathematics, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong, P.R. China

  10. Random Walk Learning and the Pac-Man Attack

    Aug 1, 2025Xingran Chen, Parimal Parag, Rohit Bhagat +2CybersecurityDecentralized Learning

    Department of Electrical and Computer Engineering, Rutgers University, Piscataway Township, NJ 08854, USA · Department of Electrical Communication Engineering, Indian Institute of Science, Bangalore, Karnataka 560012, India

  11. Better Convergence Guarantees for Sign-Based Momentum Methods

    Jul 16, 2025Wei Jiang, Dingzhi Yu, Sifan Yang +3Sign-Based OptimizationMomentum Methods

    National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China

  12. On the O(dK1/4)O(\frac{\sqrt{d}}{K^{1/4}}) Convergence Rate of AdamW Measured by ℓ1\ell_1 Norm

    May 17, 2025Huan Li, Yiming Dong, Zhouchen LinOptimization Convergence AnalysisStochastic Optimization Convergence

    Institute of Robotics and Automatic Information Systems, College of Artificial Intelligence, Nankai University, Tianjin, China. · National Key Lab of General AI, School of Intelligence Science and Technology, Peking University, Beijing, China.

  13. Rethinking the Global Convergence of Softmax Policy Gradient with Linear Function Approximation: The Case of Multi-Armed Bandits

    May 6, 2025Max Qiushi Lin, Jincheng Mei, Matin Aghaei +6Multi-Armed BanditsPolicy Gradient

    Simon Fraser University · Google DeepMind · Google Research +1

  14. A Provably Convergent Plug-and-Play Framework for Stochastic Bilevel Optimization

    May 2, 2025Tianshu Chu, Dachuan Xu, Wei Yao +2Bilevel OptimizationStochastic Optimization Convergence

    Institute of Operations Research and Information Engineering, Beijing University of Technology · National Center for Applied Mathematics Shenzhen · National Center for Applied Mathematics Shenzhen, and Department of Mathematics, Southern University of Science and Technology +2

  15. Preconditioned Inexact Stochastic ADMM for Deep Model

    Feb 15, 2025Shenglong Zhou, Ouya Wang, Ziyan Luo +2Stochastic OptimizationDeep Learning Optimization

    School of Mathematics and Statistics, Beijing Jiaotong University, China · ITP Lab, Department of EEE, Imperial College London, UK · National Mobile Communications Research Laboratory, Southeast University, China

  16. Convergence Rate Analysis of LION

    Nov 12, 2024Yiming Dong, Huan Li, Zhouchen LinSign-Based OptimizationDeep Learning Optimization

    State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University, Beijing, 100871, China · Institute of Robotics and Automatic Information Systems, College of Artificial Intelligence, Nankai University, Tianjin, 300071, China · Pazhou Laboratory (Huangpu), Guangzhou, 510555, China

  17. Convergence of Sharpness-Aware Minimization Algorithms using Increasing Batch Size and Decaying Learning Rate

    Sep 16, 2024Hinata Harada, Hideaki IidukaSharpness-Aware MinimizationNonconvex Optimization

    Department of Computer Science Meiji University

  18. Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses

    Jun 20, 2024Steffen Dereich, Arnulf Jentzen, Adrian RiekertStochastic Optimization ConvergenceAdaptive Gradient Methods

    Institute for Mathematical Stochastics, University of Münster, Germany · School of Data Science and Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), China · Applied Mathematics: Institute for Analysis and Numerics, University of Münster, Germany

  19. Convergence Analysis of Sequential Federated Learning on Heterogeneous Data

    Nov 6, 2023Yipeng Li, Xinchen LyuNon-IID Federated LearningStochastic Optimization Convergence

    National Engineering Research Center for Mobile Network Technologies Beijing University of Posts and Telecommunications Beijing, 100876, China

  20. Can SGD Select Good Fishermen? Local Convergence under Self-Selection Biases

    Date pendingAlkis Kalavasis, Anay Mehrotra, Felix ZhouLinear RegressionStochastic Optimization Convergence

    Yale University

  21. Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization

    Date pendingSharan Sahu, Abir Sarkar, Cameron J. Hogan +1Stochastic OptimizationStochastic Optimization Convergence

    Department of Statistics and Data Science Cornell University