Nonconvex Stochastic Optimization

Ask about this topic

Momentum

10 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 67

All topics
CardsList

  1. Composite Online-to-Nonconvex Conversion with Optimal Oracle Complexity

    Oct 8, 2026Mingyi Li, Taira Tsuchiya, Kenji YamanishiNonconvex Stochastic OptimizationNonconvex Optimization

    The University of Tokyo · The University of Tokyo, The University of Osaka, and RIKEN

  2. Decentralized SGD under Heavy-Tailed Noise: Optimal Convergence Rates and the Role of Gradient Clipping

    Oct 7, 2026Aleksandar Armacki, Haoyuan Cai, Ali H. SayedDecentralized OptimizationStochastic Gradient Descent

    Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

  3. Dimension-Free Decentralized Nonsmooth Nonconvex Stochastic Optimization

    Oct 5, 2026Yuanyu Wan, Lan Xue, Haomin Bai +2Nonconvex Stochastic OptimizationDecentralized Optimization

    School of Software Technology, Zhejiang University · State Key Laboratory of Blockchain and Security, Zhejiang University · School of Artificial Intelligence, Nanjing University +1

  4. Smoothed Gradient Method for Nonconvex Federated Stochastic Bilevel Optimization

    Oct 4, 2026Xinwen Zhang, Peiran Yu, Zhaosong Lu +1Gradient DescentNonconvex Stochastic Optimization

    Temple University · University of Texas at Arlington · University of Minnesota

  5. Optimal Stochastic Bilevel Optimization with First-Order Oracles

    Oct 1, 2026Linxuan Pan, Junchi YangOracle ComplexityNonconvex Stochastic Optimization

    The Chinese University of Hong Kong, Shenzhen

  6. Convergence Analysis of STORM Under Different Geometries

    Oct 1, 2026Wei Jiang, Yibo Wang, Wenhao Yang +3Stochastic OptimizationLast-Iterate Convergence

    School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China

  7. Optimal Momentum Methods for Stochastic Multilevel Compositional Optimization

    Oct 1, 2026Wei Jiang, Rui Yan, Sifan Yang +3Stochastic OptimizationMomentum Methods

    School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China · School of Software Technology, Zhejiang University, Ningbo, China

  8. Convergence of Practical Muon with Finite Newton-Schulz Iterations and Nesterov Momentum

    Sep 30, 2026Hanyng Peng, Hui Wang, Yue YuNewton-Schulz IterationGradient Descent

    Pengcheng Laboratory, Shenzhen, China

  9. Ordinary Nonconvex SGD under Distance-Dependent Moments: Finite-Horizon Stationarity and Nagaev Bounds

    Sep 24, 2026Wei Biao WuNonconvex Stochastic OptimizationStochastic Optimization Convergence

  10. Precise Convergence Speed of Clipped SGD

    Sep 24, 2026David A. R. RobinGradient ClippingNonconvex Stochastic Optimization

    LAMSADE, Université Paris-Dauphine, PSL Research University

  11. Revisiting Distributed Sign-Based Variance Reduction

    Sep 16, 2026Wei Jiang, Zechao Li, Lijun ZhangSign-Based OptimizationCommunication-Efficient Distributed Training

    School of Computer Science and Engineering, Nanjing University of Science and Technology, China · National Key Laboratory for Novel Software Technology,Nanjing University, China · School of Artificial Intelligence, Nanjing University, China

  12. Projection-Free Multi-level Algorithms for Stochastic Constrained Compositional Optimization

    Sep 14, 2026Wei Jiang, Sifan Yang, Wenhao Yang +4Stochastic OptimizationNonconvex Stochastic Optimization

    School of Computer Science and Engineering, Nanjing University of Science and Technology, China · School of Artificial Intelligence, Nanjing University, China · School of Software Technology, Zhejiang University, China

  13. Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and H"{o}lder Smoothness

    Sep 14, 2026Misbah Uz Zaman, Anirbit MukherjeeGradient ClippingNonconvex Stochastic Optimization

    Department of Mathematics and Statistics Indian Institute of Science Education and Research Kolkata · Department of Computer Science The University of Manchester

  14. Adaptive Bregman Proximal Stochastic Gradient with a Stabilized Barzilai--Borwein Step Size

    Aug 12, 2026Chenhan Jin, Shengze Xu, Binghui Xie +4Nonconvex Stochastic OptimizationAdaptive Gradient Methods

    1The Chinese University of Hong Kong · University of Utah · 3Beijing Normal-Hong Kong Baptist University +1

  15. A Tight Lower Bound for Smooth Nonconvex Stochastic Optimization with Bounded Gradient Noise

    Aug 10, 2026Jikai JinStochastic OptimizationNonconvex Stochastic Optimization

  16. A proximal subgradient method for nonconvex stochastic optimization under the Kurdyka-Łojasiewicz condition

    Aug 5, 2026Felipe Atenas, Alejandro Jofré, Pedro Pérez-Aros +1Stochastic OptimizationNonconvex Stochastic Optimization

    Centro de Modelamiento Matem´atico (CNRS IRL2807), Universidad de Chile, Santiago, Chile · Departamento de Ingenier´ıa Matem´atica and Centro de Modelamiento Matem´atico (CNRS IRL2807), Universidad de Chile, Santiago, Chile · Department of Mathematics, University of Alicante, Alicante, Spain

  17. Stochastic Saddle Avoidance Beyond Unit Excitation and Smoothness: A Pathwise Lyapunov-Perron Framework

    Aug 4, 2026Junwen Qiu, Bohao Ma, Andre Milzarek +1Stochastic OptimizationNonconvex Stochastic Optimization

    Industrial Systems Engineering and Management, National University of Singapore, Singapore. · School of Data Science, The Chinese University of Hong Kong, Shenzhen, China.

  18. The Convergence Behavior of Adam under Heavy-Tailed Noise

    Jul 29, 2026Yijiang PangNonconvex Stochastic OptimizationHeavy-Tailed Noise

    Michigan State University, East Lansing, USA

  19. Variance-Reduced Conditional Gradient Methods under Markovian Sampling for Nonconvex Composite Optimization

    Jul 28, 2026Zhaojun PengStochastic OptimizationNonconvex Stochastic Optimization

    School of Computer Science Nanjing University of Information Science and Technology

  20. Quantum Speedups for Stochastic Optimization with Heavy-Tailed Noise

    Jul 28, 2026Bin Luo, Chengchang Liu, Jonathan Allcock +2Nonconvex Stochastic OptimizationHeavy-Tailed Noise

    The Chinese University of Hong Kong · Westlake University · Tencent Quantum Laboratory

  21. On the Convergence of Stochastic Low-Rank Adaptation

    Jul 24, 2026Ru Wang, Chengchang Liu, John C. S. LuiNonconvex Stochastic OptimizationLow-Rank Adaptation

    The Chinese University of Hong Kong · Westlake University

  22. RELTA-SGLD: Relative-Growth Localized Taming for Nonconvex Stochastic-Gradient Langevin Learning

    Jul 21, 2026Yiwei Zhou, Ziheng ChenStochastic Gradient Langevin DynamicsLangevin Dynamics

    School of Mathematics and Statistics, Yunnan University, Kunming, Yunnan 650500, China.

  23. Optimizing the Preconditioner: A Black-box Online-to-Nonconvex Conversion with Static Regret Minimization Oracles

    Jul 20, 2026Haichen Hu, David Simchi-LeviNonconvex Stochastic OptimizationAdaptive Gradient Methods

    Center for Computational Science and Engineering, MIT · Department of Civil and Environmental Engineering, MIT · Institute for Data, Systems, and Society, MIT

  24. K-ABENA: K-Adaptive Backpropagation with Error-based N-exclusion Algorithm : (Compensated Loss-Based Sample Exclusion with Unbiased Gradient Estimation)

    Jul 7, 2026Jean-Francois BonbhelBackpropagationNeural Network Optimization

    NeuroSoft IA, Qu´ebec City, Canada · YekoElite University, Brazzaville, Republic of Congo

  25. Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models

    Jul 6, 2026Stanislas Strasman, Sobihan Surendran, Sylvain Le CorffNonconvex Stochastic OptimizationScore-Based Generative Modeling

    SU, LPSM

  26. Federated learning with heavy-tailed gradient noise and communication noise: a variance-reduction based algorithm

    Jun 21, 2026Shengchao Zhao, Yongchao LiuStochastic OptimizationNonconvex Stochastic Optimization

    School of Mathematics, China University of Mining and Technology, Xuzhou, China · School of Mathematical Sciences, Dalian University of Technology, Dalian, China

  27. SILAGE: Memory-Efficient, Full-Gradient-Free Nonconvex Optimization for Nested Finite Sums

    Jun 14, 2026Igor Sokolov, Laurent Condat, Peter RichtárikMemory-Efficient OptimizationNonconvex Stochastic Optimization

    *Center of Excellence for Generative AI, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia. · KAUST

  28. Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Success

    Jun 12, 2026Florian Hübler, Thomas Pethick, Suvrit SraStochastic OptimizationNonconvex Stochastic Optimization

    Department of Computer Science, ETH Zurich, Switzerland · Department of Mathematics, Technical University of Munich, Germany · Munich Center for Machine Learning (MCML)

  29. OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality

    Jun 7, 2026Ganzhao YuanMomentum MethodsNonconvex Stochastic Optimization

    Faculty of Computer Science and Artificial Intelligence Shenzhen University of Advanced Technology (SUAT), China

  30. When Do Fewer Coordinates Suffice in DP-SGD?

    Jun 3, 2026Huiqi Zhang, Fang XieDifferentially Private Stochastic Gradient DescentNonconvex Stochastic Optimization

    Guangdong Provincial Key Laboratory of IRADS, Beijing Normal-Hong Kong Baptist University Zhuhai, China