Stochastic Optimization Convergence

Latest papers 141

All topics
CardsList
  1. Adam Converges in Nonsmooth Nonconvex Optimization

    Jun 21, 2026Zijian LiuHeavy-Tailed NoiseNonconvex Optimization

  2. Accelerated and Stable Convergence with Anchored Optimistic Method

    Jun 19, 2026Motahareh Sohrabi, Jianxin You, Simon Lacoste-Julien +2Last-Iterate ConvergenceGradient Descent

  3. Central limit theorem for the averaged Adam optimizer

    Jun 19, 2026Steffen Dereich, Arnulf JentzenStochastic ApproximationStochastic Optimization Convergence

  4. Compute Efficiency and Serial Runtime Tradeoffs for Stochastic Momentum Methods

    Jun 17, 2026Depen Morwani, Alexandru Meterez, Pranav Nair +1Stochastic OptimizationMomentum Methods

  5. MGUP: A Momentum-Gradient Alignment Update Policy for Stochastic Optimization

    Jun 16, 2026Da Chang, Ganzhao YuanNeural Network OptimizationStochastic Optimization Convergence

  6. SILAGE: Memory-Efficient, Full-Gradient-Free Nonconvex Optimization for Nested Finite Sums

    Jun 14, 2026Igor Sokolov, Laurent Condat, Peter RichtárikMemory-Efficient OptimizationNonconvex Stochastic Optimization

  7. Schattor: Schatten-family methods for deep learning optimization

    Jun 14, 2026Bohao Ma, Junyu Zhang, Chuan HeDeep Learning OptimizationStochastic Optimization Convergence

  8. Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Success

    Jun 12, 2026Florian Hübler, Thomas Pethick, Suvrit SraStochastic OptimizationNonconvex Stochastic Optimization

  9. Clipping Makes Distributed and Federated Asynchronous SGD Robust to Stragglers

    Jun 11, 2026Samuel Erickson, Mikael JohanssonGradient ClippingStochastic Optimization Convergence

  10. OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality

    Jun 7, 2026Ganzhao YuanMomentum MethodsNonconvex Stochastic Optimization

  11. Near-Optimal Decentralized Stochastic Convex Optimization over Networks

    Jun 3, 2026Nitai Kluger, Amit Attia, Tomer KorenStochastic Optimization ConvergenceAccelerated Gradient Methods

  12. Deterministic Envelopes for Tamed SGLD: Decoupling Stochastic Gradient Noise and Localizing Taming

    Jun 3, 2026Yiwei Zhou, Ziheng ChenStochastic Differential EquationsStochastic Gradient Langevin Dynamics

  13. Bregman meets Lévy: Stochastic mirror descent with heavy-tailed noise in continuous and discrete time

    Jun 2, 2026Pierre-Louis Cauvin, Panayotis MertikopoulosStochastic OptimizationStochastic Differential Equations

  14. Adaptive Sharpness-Aware Minimization with a Polyak-type Step size: A Theory-Grounded Scheduler

    Jun 1, 2026Dimitris Oikonomou, Nicolas LoizouSharpness-Aware MinimizationLearning Rate Scheduling

  15. Stochastic convergence of parallel asynchronous adaptive first-order methods

    Jun 1, 2026Serge Gratton, Philippe L. TointStochastic OptimizationNonconvex Stochastic Optimization

  16. Convergence of Two-Timescale Markovian Stochastic Approximations with Applications in Reinforcement Learning

    May 29, 2026Vagul Mahadevan, Claire Chen, Shuze Daniel Liu +1Stochastic ApproximationTemporal-Difference Learning

  17. A Unified Framework for Gradient Aggregation in Multi-Objective Optimization

    May 28, 2026Zeou Hu, Kelvin Ho, Yaoliang YuGradient DescentStochastic Optimization Convergence

  18. A Theoretical and Experimental Study of a Novel Adaptive Learning Algorithm

    May 28, 2026Sakshi Kumari, Shyam Kumar M, Sushmitha POptimization Convergence AnalysisStochastic Optimization Convergence

  19. Stochastic Gradient Descent with Momentum is Algorithmically Stable

    May 27, 2026Yunwen Lei, Zimeng Wang, Xiaoming YuanMomentum MethodsAlgorithmic Stability

  20. Learning Theory of the SVRG: Generalization and Convergence Analysis

    May 27, 2026Yunwen Lei, Zimeng Wang, Xiaoming YuanStochastic Optimization ConvergenceVariance Reduction

  21. Probabilistic Smoothing with Ratio-Monotone Transforms for Global Optimization

    May 26, 2026Kukyoung Jang, Taehyun Cho, Junrui Zhang +2Black-Box OptimizationGlobal Optimization

  22. Global Convergence of Wasserstein Policy Gradient for Entropy-Regularized Reinforcement Learning

    May 25, 2026Zhaoyu Zhu, Rui Gao, Shuang LiOptimization Convergence AnalysisWasserstein Gradient Flows

  23. A note on convergence of Wasserstein policy optimization

    May 21, 2026David Šiška, Yufei ZhangWasserstein Gradient FlowsPolicy Optimization

  24. An Improved Adaptive PID Optimizer with Enhanced Convergence and Stability for Deep Learning

    May 21, 2026Saurabh Saini, Kapil Ahuja, Thomas Wick +1Deep Learning OptimizationStochastic Optimization Convergence

  25. LionMuon: Alternating Spectral and Sign Descent for Efficient Training

    May 19, 2026Arman Bolatov, Artem Riabinin, Nikita Kornilov +4Stochastic OptimizationSign-Based Optimization

  26. From Mean-Field Limits to Semiclassical Concentration: Global Convergence of the Canonical Evolutionary Strategy

    May 19, 2026Matías Neto, Nicolás Garay, Luis Martí +1Dynamical SystemsEvolutionary Optimization