Heavy-Tailed Stochastic Optimization

Ask about this topic

Momentum

2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 22

All topics
CardsList

  1. Decentralized SGD under Heavy-Tailed Noise: Optimal Convergence Rates and the Role of Gradient Clipping

    Oct 7, 2026Aleksandar Armacki, Haoyuan Cai, Ali H. SayedDecentralized OptimizationStochastic Gradient Descent

    Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

  2. Ordinary Nonconvex SGD under Distance-Dependent Moments: Finite-Horizon Stationarity and Nagaev Bounds

    Sep 24, 2026Wei Biao WuNonconvex Stochastic OptimizationStochastic Optimization Convergence

  3. Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and H"{o}lder Smoothness

    Sep 14, 2026Misbah Uz Zaman, Anirbit MukherjeeGradient ClippingNonconvex Stochastic Optimization

    Department of Mathematics and Statistics Indian Institute of Science Education and Research Kolkata · Department of Computer Science The University of Manchester

  4. The Convergence Behavior of Adam under Heavy-Tailed Noise

    Jul 29, 2026Yijiang PangNonconvex Stochastic OptimizationHeavy-Tailed Noise

    Michigan State University, East Lansing, USA

  5. Parameter-Free Dynamic Regret under Heavy-Tailed Noise

    Jul 29, 2026Vaneet AggarwalOnline Convex OptimizationDynamic Regret Minimization

    Purdue University

  6. Quantum Speedups for Stochastic Optimization with Heavy-Tailed Noise

    Jul 28, 2026Bin Luo, Chengchang Liu, Jonathan Allcock +2Nonconvex Stochastic OptimizationHeavy-Tailed Noise

    The Chinese University of Hong Kong · Westlake University · Tencent Quantum Laboratory

  7. Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalization

    Jul 9, 2026Ryusei Yamada, Naoki Sato, Hideaki IidukaHeavy-Tailed NoiseStochastic Optimization Convergence

    Meiji University, Japan

  8. High-Probability PL-SGD with Markovian Noise: Optimal Mixing and Tail Dependence

    Jun 24, 2026Dhruv Sarkar, Aprameyo Chakrabartty, Vaneet AggarwalStochastic OptimizationOptimization Convergence Analysis

    Indian Institute of Technology Kharagpur · Mohamed bin Zayed University of Artificial Intelligence · Purdue University

  9. Is Variational Monte Carlo Robust? Sharp Moment Thresholds and Heavy-tailed Stochastic Optimization

    Jun 24, 2026Philipp Grohs, Davide NobileStochastic OptimizationQuantum Chemistry

    Faculty of Mathematics, University of Vienna · RICAM, Austrian Academy of Sciences

  10. Open Problem: Is AdamW Effective Under Heavy-Tailed Noise?

    Jun 22, 2026Dingzhi Yu, Hongyi Tao, Yuanyu Wan +2Stochastic OptimizationHeavy-Tailed Noise

    1Nanjing University · 2Zhejiang University · 3Fudan University

  11. Federated learning with heavy-tailed gradient noise and communication noise: a variance-reduction based algorithm

    Jun 21, 2026Shengchao Zhao, Yongchao LiuStochastic OptimizationNonconvex Stochastic Optimization

    School of Mathematics, China University of Mining and Technology, Xuzhou, China · School of Mathematical Sciences, Dalian University of Technology, Dalian, China

  12. Distribution-Aware Robust Bilevel Optimization: Quantile-Guided Huber Updates in Two-Timescale Stochastic Approximation

    Jun 21, 2026Zhiyu Li, Xi Xuan, Davide CarboneStochastic OptimizationStochastic Approximation

    University of Science and Technology of China Hefei, China · City University of Hong Kong Hong Kong SAR, China · Laboratoire de Physique de l’École Normale Supérieure, Université PSL, CNRS, Sorbonne Université, Université de Paris Paris, France

  13. Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Success

    Jun 12, 2026Florian Hübler, Thomas Pethick, Suvrit SraStochastic OptimizationNonconvex Stochastic Optimization

    Department of Computer Science, ETH Zurich, Switzerland · Department of Mathematics, Technical University of Munich, Germany · Munich Center for Machine Learning (MCML)

  14. Bregman meets Lévy: Stochastic mirror descent with heavy-tailed noise in continuous and discrete time

    Jun 2, 2026Pierre-Louis Cauvin, Panayotis MertikopoulosStochastic OptimizationStochastic Differential Equations

    Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, France.

  15. In-Expectation Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise

    May 30, 2026Zijian LiuStochastic OptimizationNonconvex Stochastic Optimization

    Stern School of Business, New York University

  16. Zeroth-Order Nonconvex Nonsmooth Optimization with Heavy-Tailed Noise

    May 23, 2026Zhuanghua Liu, Luo LuoStochastic OptimizationGradient Clipping

  17. Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad

    May 18, 2026Zijian LiuStochastic OptimizationNonconvex Stochastic Optimization

    Stern School of Business, New York University

  18. Scale-Invariant Neural Network Optimization: Norm Geometry and Heavy-Tailed Noise

    May 18, 2026Jiayu Zhang, Tianyi LinStochastic OptimizationNeural Network Optimization

    Department of Industrial Engineering and Operations Research Columbia University

  19. Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters

    May 12, 2026Alexander Yukhimchuk, Mladen Kolar, Martin Takáč +1Stochastic OptimizationGradient Clipping

    MBZUAI · University of Southern California

  20. Robust stochastic first order methods in heavy-tailed noise via medoid mini-batch gradient sampling

    May 8, 2026Manojlo Vukovic, Dusan JakoveticStochastic OptimizationNonconvex Stochastic Optimization

    Faculty of Technical Sciences, University of Novi Sad, Novi Sad, Serbia · Faculty of Sciences, University of Novi Sad, Novi Sad, Serbia

  21. Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition

    May 7, 2026Sayantan Choudhury, Xiaoran Cheng, Martin Takáč +2Stochastic OptimizationMomentum Methods

    MBZUAI · Penn State University · Georgia Institute of Technology +1

  22. Accelerated stochastic first-order method for convex optimization under heavy-tailed noise

    Oct 13, 2025Chuan He, Bowen Li, Zhaosong LuStochastic OptimizationHeavy-Tailed Noise

    Department of Mathematics, Linköping University, Sweden · Department of Industrial and Systems Engineering, University of Minnesota, USA