Heavy-Tailed Distributions

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 39

All topics

Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data
Oct 6, 2026Kaito Takanami, Takashi Takahashi, Yoshiyuki KabashimaStatistical Physics of Learning Linear Regression
Graduate School of Science, The University of Tokyo · Institute for Physics of Intelligence, The University of Tokyo · Trans-Scale Quantum Science Institute, The University of Tokyo
Attention Kernels for Learning Maps Between Heavy-Tailed Measures
Sep 30, 2026Kailen Hargenrader, Edoardo Calvello, Bohan ChenSoftmax Attention Attention Mechanisms
Computing and Mathematical Sciences California Institute of Technology · Lawrence Berkeley National Laboratory University of California, Berkeley ICSI
Distributed Stochastic Approximation Algorithms and Heavy-Tailed Age of Information
Sep 23, 2026Adrian Redder, Arunselvan Ramaswamy, Holger KarlStochastic Approximation Age of Information
Dept. of Electrical Engineering and Information Technology, Paderborn University, Germany · Dept. of Industrial Engineering and Operations Research, Indian Institute of Technology Bombay, India · Hasso-Plattner Institute (HPI), Digital Engineering Faculty, University of Potsdam, Germany
Dimension-Corrected Hitting Times for Heavy-Tailed Spectral Emergence in Neural Optimizer Dynamics
Sep 14, 2026Zongmin LiuHeavy-Tailed Distributions Neural Network Training Dynamics
Stanford University
A location-invariant estimator of extremal quantile treatment effects for heavy-tailed distributions
Sep 3, 2026Xin Yu, Shuwei Huang, Jicheng Liu +4Causal Effect Estimation Heavy-Tailed Distributions
Huazhong University of Science and Technology · Tencent
Restricted Eigenvalues Beyond Gaussian Width: Threshold Occupancy under Heavy Tails
Sep 3, 2026Shi Fu, Huibo Xu, Qixin Zhang +1Sparse Recovery Heavy-Tailed Noise
Generative AI Lab, College of Computing and Data Science, Nanyang Technological University, Singapore
Fine-Tuning Generative Models for Extreme Events via CVaR-Penalized Wasserstein Gradient Flows
Aug 12, 2026Thejani Gamage, Hyemin Gu, Zhizhen Zhang +3Wasserstein Gradient Flows Generative Modeling
Department of Mathematics and Statistics, University of Massachusetts Amherst, Amherst, MA · School of Data and Information Sciences (SDIS) and Department of Mathematics, University of North Carolina at Chapel Hill, Chapel Hill, NC
Parameter-Free Heavy-Tailed Bandits
Jul 31, 2026Gianmarco Genalti, Alberto Maria MetelliMulti-Armed Bandits Heavy-Tailed Noise
Politecnico di Milano
HOMER: Huber-of-Means for Efficient and Robust Estimation in Hilbert Spaces
Jul 29, 2026Kisung You, Boram ChoHeavy-Tailed Distributions Robust Estimation
Baruch College · Yale University
Heavy-Tailed Flow Matching via Random Clocks
Jul 15, 2026Zhouhao Yang, Yezhen Wang, Kenji Kawaguchi +2Flow Matching Heavy-Tailed Noise
Department of Applied Mathematics and Statistics, Johns Hopkins University. · School of Computing, National University of Singapore. · Department of Computer Science, Johns Hopkins University.
Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalization
Jul 9, 2026Ryusei Yamada, Naoki Sato, Hideaki IidukaHeavy-Tailed Noise Stochastic Optimization Convergence
Meiji University, Japan
Model Merging as Probabilistic Inference in Fine-Tuning Parameter Space
Jul 2, 2026Long Minh Bui, Tuan Anh Le Van, Tung Phi Duc +3Energy-Based Models Heavy-Tailed Distributions
Washington State University · Hanoi University of Science and Technology
Heads, Not Backbones: Output Heads Dominate Architectures on Fat-Tailed Returns
Jun 29, 2026Sichao He, Yansong ZhangQuantitative Finance Financial Forecasting
Peking University · Academy for Advanced Interdisciplinary Studies · Beijing, China +2
Adjusted Wasserstein distances for bridging empirical and true distributions with applications to MDS
Jun 29, 2026Flor Martinez-Sermeno, Arturo Jaramillo, Johan Van HorebeekDimensionality Reduction Heavy-Tailed Distributions
The Decision Geometry of Covariance Estimation for the Global Minimum-Variance Portfolio under Heavy Tails
Jun 25, 2026Xavier FonsecaRegret Minimization Portfolio Optimization
Academy for AI, Games and Media, Breda University of Applied Sciences, Breda, The Netherlands.
High-Probability PL-SGD with Markovian Noise: Optimal Mixing and Tail Dependence
Jun 24, 2026Dhruv Sarkar, Aprameyo Chakrabartty, Vaneet AggarwalStochastic Optimization Optimization Convergence Analysis
Indian Institute of Technology Kharagpur · Mohamed bin Zayed University of Artificial Intelligence · Purdue University
Open Problem: Is AdamW Effective Under Heavy-Tailed Noise?
Jun 22, 2026Dingzhi Yu, Hongyi Tao, Yuanyu Wan +2Stochastic Optimization Heavy-Tailed Noise
1Nanjing University · 2Zhejiang University · 3Fudan University
Tail-Shape Estimation in LLM Evaluation Is Fragile: A Protocol for Diagnosing False Positives
Jun 15, 2026Luca ZhouLLM Evaluation Language Model Safety Evaluation
Sapienza University of Rome
Gradient boosting for extremes: sampling theory and application to insurance
Jun 12, 2026Stéphane Lhaut, Olivier LopezGradient-Boosted Decision Trees Heavy-Tailed Distributions
1CREST, CNRS, Ecole polytechnique, Groupe ENSAE-ENSAI, ENSAE Paris, Institut Polytechnique de Paris, Palaiseau, France
Quantum enhanced rare event discovery and sampling
Jun 4, 2026Naixu Guo, Po-Wei Huang, Qisheng Wang +4Heavy-Tailed Distributions Rare-Event Estimation
Centre for Quantum Technologies, National University of Singapore, Singapore 117543, Singapore · Mathematical Institute, University of Oxford, Oxford OX2 6GG, United Kingdom · School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, China +4
Variance Reduction for Heavy-Tailed Monetization Metrics in Ranking Experiments via Post-Stratification
Jun 2, 2026Neeti Pokharna, Olivier Jeunen, Yatharth Saraf +1Causal Effect Estimation A/b Testing
ShareChat · Bengaluru, Karnataka, India · Aampe +3
Bregman meets Lévy: Stochastic mirror descent with heavy-tailed noise in continuous and discrete time
Jun 2, 2026Pierre-Louis Cauvin, Panayotis MertikopoulosStochastic Optimization Stochastic Differential Equations
Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, France.
Self-Regulating Annealing in Heavy-Tailed Diffusion Models
Jun 1, 2026Keito Wakatsuki, Hideaki ShimazakiStochastic Differential Equations Diffusion Sampling
Graduate School of Informatics Kyoto University, Kyoto, Japan
Stop Suppressing the Tail: Causal Inference for Extreme Events
May 26, 2026Eichi UeharaCausal Effect Estimation Causal Inference
Concentration of General Stochastic Approximation Under Heavy-Tailed Markovian Noise
May 20, 2026Shubhada Agrawal, Siva Theja Maguluri, Martin ZubeldiaHeavy-Tailed Noise Stochastic Approximation
Indian Institute of Science · Georgia Institute of Technology · University of Minnesota
Tail Annealing for Heavy-Tailed Flow Matching
May 19, 2026Jean PachebatFlow Matching Generative Modeling
1CMAP, Ecole Polytechnique, Institut Polytechnique de Paris, France.
Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad
May 18, 2026Zijian LiuStochastic Optimization Nonconvex Stochastic Optimization
Stern School of Business, New York University
Markov Chain Decoders Overcome the Heavy-Tail Limitations of Lipschitz Generative Models
May 18, 2026Abdelhakim Ziani, Andras Horvath, Paolo BallariniVariational Autoencoders Generative Modeling
Université Paris Saclay, Lab. MICS, CentraleSupélec, Gif-sur-Yvette, France · Università di Torino, Torino, Italy
On Stability and Decomposition of Sample Quantiles under Heavy-Tailed Distributions
May 18, 2026Choudur LakshminarayanQuantitative Finance Heavy-Tailed Distributions
School of Business, Stevens Institute of Technology
Do Heavy Tails Help Diffusion? On the Subtle Trade-off Between Initialization and Training
May 13, 2026Hamza Cherkaoui, Hélène Halconruy, Antonio OcelloDiffusion Model Sampling Heavy-Tailed Noise
SAMOVAR, Télécom SudParis, Institut Polytechnique de Paris, Palaiseau, France · Modal’X, Université Paris Nanterre, Nanterre, France · CREST, ENSAE Paris, Institut Polytechnique de Paris, Palaiseau, France

Oct 6, 2026, cond-mat.dis-nnJ/K move · Enter open · S save

Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data

Kaito Takanami, Takashi Takahashi, Yoshiyuki Kabashima

Graduate School of Science, The University of Tokyo · Institute for Physics of Intelligence, The University of Tokyo · Trans-Scale Quantum Science Institute, The University of Tokyo

Statistical Physics of Learning Linear Regression Heavy-Tailed Distributions Empirical Risk Minimization

Open paper AskPDF arXiv

Many real-world datasets exhibit unusually large values far more frequently than predicted by Gaussian models. Heavy-tailed distributions capture this behavior, yet evaluating learning performance under them remains challenging because rare, large feature entries retain non-vanishing effects even in high dimensions. Even in the canonical setting of empirical risk minimization for linear regression with entry-wise i.i.d. symmetric

α

-stable data, a precise asymptotic characterization of prediction has been lacking. In this work, we introduce a functional order parameter that describes the random effective problem associated with each coefficient. Using the replica method, we fully characterize the generalization error in the proportional high-dimensional limit where the sample size and feature dimension diverge at a fixed ratio. Additionally, this analysis establishes a heavy-tail universality law, scaling laws relating typical errors to prediction reliability, and the Bayes-optimal prediction error. In addition to characterizing the effects of extreme entries on the learning process, our method applies broadly to other systems with persistent local heterogeneity.

Momentum

Related topics

Latest papers 39