Statistical Physics of Learning

Overview

The field studies learning systems as high-dimensional statistical-mechanical models, asking how data and optimization produce generalization, memorization, and distinct training regimes. Common approaches include teacher–student models, replica and dynamical mean-field methods, Gibbs distributions, and order parameters that track quantities such as overlap, loss, and representation structure. Researchers use these tools to analyze phase transitions in settings including perceptrons, associative memories, energy-based models, attention, and neural networks, alongside finite-size tests on phenomena such as grokking and double descent. A central challenge is determining when phase-transition explanations and equilibrium theories predict behavior in realistic networks trained for finite time, and which observables can reliably distinguish mechanisms.

Latest papers

CardsList

60 papers9 papers in the last four weeks, up 13% on the four weeks before. 0.1% of all new papers.

  1. Deep Learning Theory through the Lens of Statistical Physics: A Topical Review

    Nov 6, 2026Wei Huang, Youjin DengStatistical Physics of LearningNeural Network Generalization

  2. Susceptibilities for Neural Networks Learning from Physical Data

    Nov 2, 2026Rohan Hitchcock, Gary W. Delaney, Jonathan H. Manton +2Statistical Physics of LearningNeural Network Training Dynamics

  3. Finite-step statistical mechanics of scale-invariant learning: spectral equations of state and curvature transport

    Oct 27, 2026Alexander ItinStatistical Physics of LearningStochastic Gradient Descent

  4. Scaling and Renormalization in Neural Networks: An Effective Renormalization-Group Description of Grokking

    Oct 26, 2026Min Chen, Junqi Wang, Zeguan Wu +2Renormalization GroupGrokking

  5. Generalization in Learning: Eyring Formula and Predator-prey Model

    Oct 25, 2026S. V. Kozyrev, I. A. Lopatin, Alexander PechenNeural Network GeneralizationStatistical Physics of Learning

  6. Generalization properties of the token perceptron: the role of embedding and vocabulary

    Oct 23, 2026Dario Bocchi, Matteo Negri, Flavio Nicoletti +1Language Model GeneralizationRepresentation Geometry in Language Models

  7. Phase Transitions in Learning Driven by Extremal Data

    Oct 21, 2026Ethan Levien, Trevor GrandPreMultiple Instance LearningPhase Transitions in Neural Networks

  8. Sample-efficient learning with Dense Associative Memories

    Oct 19, 2026Andrew Lucas, Dmitry KrotovAssociative MemoryData-Efficient Learning

  9. Mesoscopic learning dynamics of extensive-width neural networks

    Oct 16, 2026Shu Zhou, Jean Barbier, Sebastian GoldtNeural Network Training DynamicsRepresentation Learning

  10. On the Statistical Mechanics of Attention

    Oct 12, 2026Jerry Yao-Chieh Hu, Yu-An Chen, Maojiang Su +1Transformer AttentionAttention Mechanisms

  11. On the Theoretical Limits of Learning Unconstrained Interatomic Potentials

    Oct 10, 2026Prajwal Ananth, Shuwen YueMachine Learning Interatomic PotentialsMolecular Dynamics Simulation

  12. Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data

    Oct 6, 2026Kaito Takanami, Takashi Takahashi, Yoshiyuki KabashimaStatistical Physics of LearningHigh-Dimensional Asymptotics

  13. A Flatness-Generalization Relation in the Teacher-Student Tree-Committee Machine

    Sep 25, 2026Brandon Livio Annesi, Davide Straziota, Enrico Maria MalatestaFlat MinimaNeural Network Generalization

  14. An Analytical Theory of Auxiliary Learning

    Sep 24, 2026Federico Milanesio, Alessandro Ingrosso, Matteo OsellaStatistical Physics of LearningNeural Network Generalization

  15. An Exploratory Replica-Overlap Probe of the Grokking Transition

    Sep 22, 2026A. C. Opus, J. Q. LuStatistical Physics of LearningNeural Network Training Dynamics

  16. Do Quantum Models Scale Like LLMs?

    Sep 17, 2026David S. Berman, Ying-Jer Kao, Roger G. Melko +1Neural Scaling LawsTransformer

  17. Double descent is the principle of least action

    Sep 16, 2026Congzhou M ShaDouble DescentStatistical Physics of Learning

  18. A Function-Space Approach to the Statistical Mechanics of Learning Dynamics

    Sep 9, 2026Yizhou Zhang, Weichen Wu, Lun Du +1Statistical Physics of LearningRepresentation Learning

  19. Memory as an Energy Landscape---Hopfield

    Sep 2, 2026Nima DehghaniHopfield NetworksAssociative Memory

  20. Landau theory of quenched criticality in linear in-context learning

    Aug 28, 2026Daesik Kim, Sumin Choi, Hyojae Jeon +1In-Context LearningDouble Descent

  21. Dataset Complexity Shapes Finite-Distance Loss Geometry in Neural Networks

    Aug 23, 2026Jaeyong Bae, Hawoong JeongLoss LandscapeLoss Landscape Geometry

  22. Learning reshapes power-law anisotropy in internal representations

    Aug 15, 2026Asahi Nakamuta, Jun-nosuke TeramaeNeural Representation GeometryRepresentation Learning

  23. Cascading Through the Hierarchy: Regularizer-Induced Feature Detection as Phase Transitions in Deep Linear Neural Networks

    Aug 6, 2026Björn Ladewig, Ibrahim Talha Ersoy, Karoline WiesnerStatistical Physics of LearningRepresentation Learning

  24. Variational Bounds for Perceptron Learning from Structured Data

    Aug 5, 2026Francesco Camilli, Pierluigi Contucci, Federica Gerace +1Statistical Physics of LearningStatistical Learning Theory

  25. Statistical Mechanics of Learning on Product Wasserstein Manifolds

    Aug 2, 2026Srinivasa Rao P Vangmayi P ReddyStatistical Physics of LearningWasserstein Gradient Flows

  26. Broken Ergodicity and the Violation of the Fluctuation-Dissipation Theorem Lead to Generalization Beyond Overfitting in Machine Learning

    Jul 5, 2026Chan Li, Nigel GoldenfeldDynamical Mean-Field TheoryStatistical Physics of Learning