Statistical Physics of Learning

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 21

All topics

Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data
Oct 6, 2026Kaito Takanami, Takashi Takahashi, Yoshiyuki KabashimaStatistical Physics of Learning Linear Regression
Graduate School of Science, The University of Tokyo · Institute for Physics of Intelligence, The University of Tokyo · Trans-Scale Quantum Science Institute, The University of Tokyo
An Analytical Theory of Auxiliary Learning
Sep 24, 2026Federico Milanesio, Alessandro Ingrosso, Matteo OsellaStatistical Physics of Learning Neural Network Generalization
Department of Physics and INFN University of Turin Via Giuria 1, 10125 Turin, Italy · Donders Centre for Neuroscience Radboud University Heyendaalseweg 135, 6525 AJ Nijmegen, The Netherlands
An Exploratory Replica-Overlap Probe of the Grokking Transition
Sep 22, 2026A. C. Opus, J. Q. LuStatistical Physics of Learning Neural Network Training Dynamics
Department of Physics, University of Puerto Rico, Mayagüez, PR 00680, USA
Double descent is the principle of least action
Sep 16, 2026Congzhou M ShaDouble Descent Statistical Physics of Learning
Penn Medicine Doylestown Hospital, 595 W State St, Doylestown, PA 18901, United States
A Function-Space Approach to the Statistical Mechanics of Learning Dynamics
Sep 9, 2026Yizhou Zhang, Weichen Wu, Lun Du +1Statistical Physics of Learning Representation Learning
Variational AI · The Voleon Group · Independent Researcher +1
Memory as an Energy Landscape---Hopfield
Sep 2, 2026Nima DehghaniHopfield Networks Associative Memory
McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA · The NSF AI Institute for Artificial Intelligence and Fundamental Interactions (IAIFI), Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA
Thermodynamics of Learning: A Typed Four-Component Accounting of Memory, Fit, and Value
Aug 13, 2026Akihito SudoStatistical Physics of Learning Distribution Shift
ZeroStruct Inc.
Cascading Through the Hierarchy: Regularizer-Induced Feature Detection as Phase Transitions in Deep Linear Neural Networks
Aug 6, 2026Björn Ladewig, Ibrahim Talha Ersoy, Karoline WiesnerStatistical Physics of Learning Representation Learning
Institute of Physics and Astronomy, University of Potsdam
Variational Bounds for Perceptron Learning from Structured Data
Aug 5, 2026Francesco Camilli, Pierluigi Contucci, Federica Gerace +1Statistical Physics of Learning Statistical Learning Theory
Department of Mathematics, Alma Mater Studiorum – Universit`a di Bologna, Piazza di Porta San Donato 5, 40126 Bologna, Italy
Statistical Mechanics of Learning on Product Wasserstein Manifolds
Aug 2, 2026Srinivasa Rao P Vangmayi P ReddyStatistical Physics of Learning Wasserstein Gradient Flows
Curlvee Technolabs, India; Former Scientist, C-DAC.
Broken Ergodicity and the Violation of the Fluctuation-Dissipation Theorem Lead to Generalization Beyond Overfitting in Machine Learning
Jul 5, 2026Chan Li, Nigel GoldenfeldDouble Descent Statistical Physics of Learning
Department of Physics, University of California San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA. · Halıcıoğlu Data Science Institute, University of California San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA
Explaining Machine Learning and Memorization with Statistical Mechanics
Jun 30, 2026Robin TheriaultStatistical Physics of Learning Neural Network Memorization
Faculty of Science
Informational Frustration in Neural Manifolds: Shannon Bottlenecks and the Limits of Learnability
Jun 29, 2026Srinivasa Rao P., Vangmayi P ReddyStatistical Physics of Learning Neural Network Generalization
Curlvee Technolabs, India, Former Scientist, C-DAC · Former Scientist, C-DAC · Indian Institute of Technology Madras, India
Data-Driven Energy-Based Learning via Gibbs Measures on Hierarchical Structures
Jun 29, 2026L. U. Abdullaev, F. Herrera, U. A. Rozikov +1Statistical Physics of Learning Phase Transitions
National University of Singapore, Singapore · Department of Computer Science and Artificial Intelligence, University of Granada, E-18071 Granada, Spain · V.I. Romanovskiy Institute of Mathematics, Uzbekistan Academy of Sciences, 9, Universitet str., 100174, Tashkent, Uzbekistan +3
Noise-Driven Escape from Metastable Phases explains Grokking in Deep Neural Networks
Jun 15, 2026Ibrahim Talha Ersoy, Karoline WiesnerStatistical Physics of Learning Neural Network Generalization
Complexity Science Group, Institute of Physics and Astronomy, University of Potsdam, Potsdam, Germany
A solvable model for unsupervised federated learning
Jun 11, 2026Giovanni Catania, Aurélien Decelle, Gianluca Manzan +2Statistical Physics of Learning Unsupervised Learning
Institute for Cross-disciplinary Physics and Complex Systems IFISC (CSIC-UIB), Campus Universitat Illes Balears, 07122 Palma de Mallorca, Spain. · Escuela Técnica Superior de Ingenieros Industriales, Universidad Politécnica de Madrid, Calle de José Gutiérrez Abascal 2, Madrid 28006, Spain. · GISC - Grupo Interdisciplinar de Sistemas Complejos 28040 Madrid, Spain. +5
A Boundary-Layer Mechanism for One-Third Scaling in Online Softmax Classification
May 21, 2026Marcel Kühn, Yoon Thelge, Bernd RosenowStatistical Physics of Learning Multiclass Classification
Institute for Theoretical Physics, Leipzig University · 2ScaDS.AI Dresden/Leipzig
Spherical Boltzmann machines: a solvable theory of learning and generation in energy-based models
May 9, 2026Thomas Tulinski, Simona Cocco, Rémi Monasson +1Statistical Physics of Learning Phase Transitions
Laboratoire de Physique de l’École Normale Supérieure, PSL, CNRS UMR8023, Sorbonne Université, 24 rue Lhomond, 75005 Paris, France · Institut de Physique Théorique, Université Paris-Saclay,2026 CNRS UMR3681, CEA, Gif-sur-Yvette, France
Susceptibilities and Patterning: A Primer on Linear Response in Bayesian Learning
May 8, 2026Chris Elliott, Daniel MurfetStatistical Physics of Learning Bayesian Inference
Timaeus
Can Stationary Distributions of Scale-Invariant Neural Networks Be Described by the Thermodynamics of an Ideal Gas?
Nov 10, 2025Ildus Sadrtdinov, Ekaterina Lobacheva, Ivan Klimov +3Statistical Physics of Learning Neural Network Training Dynamics
Constructor University · Mila – Quebec AI Institute · Université de Montréal +2
A statistical physics framework for optimal learning
Jul 10, 2025Francesca Mignacco, Francesco MoriStatistical Physics of Learning Neural Network Training Dynamics
1Joseph Henry Laboratories of Physics, Princeton University, Princeton, NJ 08544, USA · 2Graduate Center, City University of New York, New York, NY 10016, USA · Center of Mathematical Sciences and Applications, Harvard University, Cambridge, MA 02138, USA

Oct 6, 2026, cond-mat.dis-nnJ/K move · Enter open · S save

Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data

Kaito Takanami, Takashi Takahashi, Yoshiyuki Kabashima

Graduate School of Science, The University of Tokyo · Institute for Physics of Intelligence, The University of Tokyo · Trans-Scale Quantum Science Institute, The University of Tokyo

Statistical Physics of Learning Linear Regression Heavy-Tailed Distributions Empirical Risk Minimization

Open paper AskPDF arXiv

Many real-world datasets exhibit unusually large values far more frequently than predicted by Gaussian models. Heavy-tailed distributions capture this behavior, yet evaluating learning performance under them remains challenging because rare, large feature entries retain non-vanishing effects even in high dimensions. Even in the canonical setting of empirical risk minimization for linear regression with entry-wise i.i.d. symmetric

α

-stable data, a precise asymptotic characterization of prediction has been lacking. In this work, we introduce a functional order parameter that describes the random effective problem associated with each coefficient. Using the replica method, we fully characterize the generalization error in the proportional high-dimensional limit where the sample size and feature dimension diverge at a fixed ratio. Additionally, this analysis establishes a heavy-tail universality law, scaling laws relating typical errors to prediction reliability, and the Bayes-optimal prediction error. In addition to characterizing the effects of extreme entries on the learning process, our method applies broadly to other systems with persistent local heterogeneity.

Momentum

Related topics

Latest papers 21