Heavy-Tailed Noise

Ask about this topic

Overview

The field asks how to estimate, optimize, and make predictions when noise has only a finite p-th moment, or occasionally infinite variance. Common approaches include clipping and truncation, Huber losses, median-of-means estimators, and robust variants of SGD, mirror descent, and bandit algorithms. These methods are studied in settings such as nonconvex optimization, time-series and system identification, federated learning, and stochastic bandits, with guarantees expressed in terms of tail moments and dependence. A central tension is that robustness can control rare large errors but may add bias or slow convergence, while methods that adapt to unknown tail behavior can have unavoidable performance costs.

Latest papers

CardsList

34 papers3 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. StochLOB: Heavy-Tailed Score Matching for Robust Limit Order Book Trend Prediction

    Oct 15, 2026Arshia Abolghasemi, Seyed Reza Tavakoli, Parsa Naderi +7Limit Order Book ForecastingScore Matching

    Dataism Laboratory for Quantitative Finance · Sharif University of Technology · Amirkabir University of Technology +3

  2. DAG Structure Learning with Node-Varying Heavy-Tailed Disturbances

    Oct 9, 2026Robin Francis, Sundeep Prabhakar ChepuriCausal Structure LearningCausal Discovery

    Indian Institute of Science, Bangalore, India

  3. Learning Linear Systems under Heavy-Tailed Noise: A Non-Asymptotic Analysis from A Single Trajectory

    Sep 30, 2026Xiaomian Yang, Sungho ShinStochastic Linear Dynamical SystemsSample Complexity

    Department of Chemical Engineering Massachusetts Institute of Technology Cambridge, MA 02139

  4. High-Probability Guarantees for SGD under ββ-Heavy-Tailed Gradient Noise

    Sep 26, 2026Qijun Tong, Masahiro Ikeda, Ryota KawasumiStochastic Gradient DescentStochastic Optimization Convergence

    The University of Electro-Communications · The University of Osaka · Gunma University

  5. Restricted Eigenvalues Beyond Gaussian Width: Threshold Occupancy under Heavy Tails

    Sep 3, 2026Shi Fu, Huibo Xu, Qixin Zhang +1Sparse RecoveryHeavy-Tailed Noise

    Generative AI Lab, College of Computing and Data Science, Nanyang Technological University, Singapore

  6. Parameter-Free Heavy-Tailed Bandits

    Jul 31, 2026Gianmarco Genalti, Alberto Maria MetelliMulti-Armed BanditsHeavy-Tailed Noise

    Politecnico di Milano

  7. The Convergence Behavior of Adam under Heavy-Tailed Noise

    Jul 29, 2026Yijiang PangNonconvex Stochastic OptimizationHeavy-Tailed Noise

    Michigan State University, East Lansing, USA

  8. Parameter-Free Dynamic Regret under Heavy-Tailed Noise

    Jul 29, 2026Vaneet AggarwalOnline Convex OptimizationDynamic Regret Minimization

    Purdue University

  9. Quantum Speedups for Stochastic Optimization with Heavy-Tailed Noise

    Jul 28, 2026Bin Luo, Chengchang Liu, Jonathan Allcock +2Nonconvex Stochastic OptimizationHeavy-Tailed Noise

    The Chinese University of Hong Kong · Westlake University · Tencent Quantum Laboratory

  10. Heavy-Tailed Flow Matching via Random Clocks

    Jul 15, 2026Zhouhao Yang, Yezhen Wang, Kenji Kawaguchi +2Flow MatchingHeavy-Tailed Noise

    Department of Applied Mathematics and Statistics, Johns Hopkins University. · School of Computing, National University of Singapore. · Department of Computer Science, Johns Hopkins University.

  11. Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalization

    Jul 9, 2026Ryusei Yamada, Naoki Sato, Hideaki IidukaHeavy-Tailed NoiseStochastic Optimization Convergence

    Meiji University, Japan

  12. Nonlinear Bandit

    Jul 8, 2026Tianshuo Zheng, Ting Wu, Zhi-Hua Zhou +1Multi-Armed BanditsHeavy-Tailed Noise

    School of Mathematics, Nanjing University, Nanjing, 210093, China · School of Artificial Intelligence, Nanjing University, National Key Laboratory for Novel Software Technology, Nanjing, 210023, China · School of Mathematics and Physics, Xi’an Jiaotong-Liverpool University, Suzhou, 215123, China

  13. Anti-Collapse Dynamics and the Emergence of Multi-Time-Scale Learning in Recurrent Neural Networks

    Jun 28, 2026Lorenzo LiviHeavy-Tailed NoiseRecurrent Neural Networks

    OPIT – Open Institute of Technology

  14. High-Probability PL-SGD with Markovian Noise: Optimal Mixing and Tail Dependence

    Jun 24, 2026Dhruv Sarkar, Aprameyo Chakrabartty, Vaneet AggarwalStochastic OptimizationOptimization Convergence Analysis

    Indian Institute of Technology Kharagpur · Mohamed bin Zayed University of Artificial Intelligence · Purdue University

  15. Open Problem: Is AdamW Effective Under Heavy-Tailed Noise?

    Jun 22, 2026Dingzhi Yu, Hongyi Tao, Yuanyu Wan +2Stochastic OptimizationHeavy-Tailed Noise

    1Nanjing University · 2Zhejiang University · 3Fudan University

  16. Federated learning with heavy-tailed gradient noise and communication noise: a variance-reduction based algorithm

    Jun 21, 2026Shengchao Zhao, Yongchao LiuStochastic OptimizationNonconvex Stochastic Optimization

    School of Mathematics, China University of Mining and Technology, Xuzhou, China · School of Mathematical Sciences, Dalian University of Technology, Dalian, China

  17. Adam Converges in Nonsmooth Nonconvex Optimization

    Jun 21, 2026Zijian LiuHeavy-Tailed NoiseNonconvex Optimization

    Department of Technology, Operations, and Statistics Leonard N. Stern School of Business New York University New York, NY 10012, USA

  18. Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Success

    Jun 12, 2026Florian Hübler, Thomas Pethick, Suvrit SraStochastic OptimizationNonconvex Stochastic Optimization

    Department of Computer Science, ETH Zurich, Switzerland · Department of Mathematics, Technical University of Munich, Germany · Munich Center for Machine Learning (MCML)

  19. Bregman meets Lévy: Stochastic mirror descent with heavy-tailed noise in continuous and discrete time

    Jun 2, 2026Pierre-Louis Cauvin, Panayotis MertikopoulosStochastic OptimizationStochastic Differential Equations

    Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, France.

  20. Self-Regulating Annealing in Heavy-Tailed Diffusion Models

    Jun 1, 2026Keito Wakatsuki, Hideaki ShimazakiStochastic Differential EquationsDiffusion Sampling

    Graduate School of Informatics Kyoto University, Kyoto, Japan

  21. Can Entry-Wise Clipping Give Spectral Control of Stochastic Gradients?

    May 26, 2026Zitao Song, Cedar Site Bai, Zhe Zhang +2Gradient ClippingHeavy-Tailed Noise

    Department of Computer Science, Purdue University, West Lafayette, IN, USA. · School of Industrial Engineering, Purdue University, West Lafayette, IN, USA

  22. Zeroth-Order Nonconvex Nonsmooth Optimization with Heavy-Tailed Noise

    May 23, 2026Zhuanghua Liu, Luo LuoStochastic OptimizationGradient Clipping

  23. Concentration of General Stochastic Approximation Under Heavy-Tailed Markovian Noise

    May 20, 2026Shubhada Agrawal, Siva Theja Maguluri, Martin ZubeldiaHeavy-Tailed NoiseStochastic Approximation

    Indian Institute of Science · Georgia Institute of Technology · University of Minnesota

  24. Robust Subspace-Constrained Quadratic Models for Low-Dimensional Structure Learning

    May 19, 2026Zheng Zhai, Xiaohui LiHeavy-Tailed NoiseLow-Rank Approximation

    Department of Statistics, Faculty of Arts and Sciences at Beijing Normal University, Zhuhai. · School of Mathematics and Information Sciences, Yantai University.

  25. Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad

    May 18, 2026Zijian LiuStochastic OptimizationNonconvex Stochastic Optimization

    Stern School of Business, New York University

  26. Tight Complexity of Muon under Heavy-Tailed Noise

    May 18, 2026Jiayu Zhang, Tianyi LinStochastic OptimizationNeural Network Optimization

    Department of Industrial Engineering and Operations Research, Columbia University

  27. Do Heavy Tails Help Diffusion? On the Subtle Trade-off Between Initialization and Training

    May 13, 2026Hamza Cherkaoui, Hélène Halconruy, Antonio OcelloDiffusion Model SamplingHeavy-Tailed Noise

    SAMOVAR, Télécom SudParis, Institut Polytechnique de Paris, Palaiseau, France · Modal’X, Université Paris Nanterre, Nanterre, France · CREST, ENSAE Paris, Institut Polytechnique de Paris, Palaiseau, France

  28. TailedTS: Benchmark Dataset for Heavy-Tailed Time Series Prediction and Periodicity Quantification

    May 9, 2026Xinyu Chen, HanQin Cai, Lijun Ding +1Time Series ForecastingHeavy-Tailed Noise

    University of Central Florida Orlando, FL 32816 · Massachusetts Institute of Technology Cambridge, MA 02139

  29. Robust stochastic first order methods in heavy-tailed noise via medoid mini-batch gradient sampling

    May 8, 2026Manojlo Vukovic, Dusan JakoveticStochastic OptimizationNonconvex Stochastic Optimization

    Faculty of Technical Sciences, University of Novi Sad, Novi Sad, Serbia · Faculty of Sciences, University of Novi Sad, Novi Sad, Serbia

  30. A Robust Low-Rank Prior Model for Structured Cartoon-Texture Image Decomposition with Heavy-Tailed Noise

    Mar 29, 2026Weihao Tang, Hongjin HeImage RestorationHeavy-Tailed Noise

    School of Mathematics and Statistics, Ningbo University, Ningbo 315211, China.