Statistical Physics of Learning

Ask about this topic

Overview

The field studies learning systems as high-dimensional statistical-mechanical models, asking how data and optimization produce generalization, memorization, and distinct training regimes. Common approaches include teacher–student models, replica and dynamical mean-field methods, Gibbs distributions, and order parameters that track quantities such as overlap, loss, and representation structure. Researchers use these tools to analyze phase transitions in settings including perceptrons, associative memories, energy-based models, attention, and neural networks, alongside finite-size tests on phenomena such as grokking and double descent. A central challenge is determining when phase-transition explanations and equilibrium theories predict behavior in realistic networks trained for finite time, and which observables can reliably distinguish mechanisms.

Latest papers

CardsList

53 papers9 papers in the last four weeks, up 29% on the four weeks before. 0.1% of all new papers.

  1. A solvable model for unsupervised federated learning

    Jun 11, 2026Giovanni Catania, Aurélien Decelle, Gianluca Manzan +2Statistical Physics of LearningUnsupervised Learning

    Institute for Cross-disciplinary Physics and Complex Systems IFISC (CSIC-UIB), Campus Universitat Illes Balears, 07122 Palma de Mallorca, Spain. · Escuela Técnica Superior de Ingenieros Industriales, Universidad Politécnica de Madrid, Calle de José Gutiérrez Abascal 2, Madrid 28006, Spain. · GISC - Grupo Interdisciplinar de Sistemas Complejos 28040 Madrid, Spain. +5

  2. A Boundary-Layer Mechanism for One-Third Scaling in Online Softmax Classification

    May 21, 2026Marcel Kühn, Yoon Thelge, Bernd RosenowNeural Network Training DynamicsMulticlass Classification

    Institute for Theoretical Physics, Leipzig University · ScaDS.AI Dresden/Leipzig

  3. Spherical Boltzmann machines: a solvable theory of learning and generation in energy-based models

    May 9, 2026Thomas Tulinski, Simona Cocco, Rémi Monasson +1Statistical Physics of LearningPhase Transitions

    Laboratoire de Physique de l’École Normale Supérieure, PSL, CNRS UMR8023, Sorbonne Université, 24 rue Lhomond, 75005 Paris, France · Institut de Physique Théorique, Université Paris-Saclay,2026 CNRS UMR3681, CEA, Gif-sur-Yvette, France

  4. Susceptibilities and Patterning: A Primer on Linear Response in Bayesian Learning

    May 8, 2026Chris Elliott, Daniel MurfetStatistical Physics of LearningBayesian Inference

    Timaeus

  5. Distinct mechanisms underlying in-context learning in transformers

    Apr 14, 2026Cole Gibson, Wenping Cui, Gautam ReddyIn-Context LearningTransformer Training Dynamics

    Lewis-Sigler Institute for Integrative Genomics, Princeton University, Princeton, NJ 08544, USA · Department of Physics, Princeton University, Princeton, NJ 08544, USA

  6. Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory

    Apr 8, 2026Tatiana Petrova, Evgeny Polyachenko, Radu StateAssociative MemoryHopfield Networks

    Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, 29 Avenue J.F. Kennedy, L-1855 Luxembourg.

  7. Dimensional Criticality at Grokking Across MLPs and Transformers

    Apr 6, 2026Ping WangNeural Network Training DynamicsPhase Transitions in Neural Networks

    Institute of High Energy Physics, Chinese Academy of Science, 100049 Beijing, China

  8. Grokking as Dimensional Phase Transition in Neural Networks

    Apr 6, 2026Ping WangPhase Transitions in Neural NetworksGrokking

    Institute of High Energy Physics, Chinese Academy of Science, 100049 Beijing, China

  9. Non-Equilibrium Stochastic Dynamics as a Unified Framework for Insight and Repetitive Learning: A Kramers Escape Approach to Continual Learning

    Apr 5, 2026Gunn KimContinual LearningStability-Plasticity Dilemma

    Department of Physics, Sejong University, Seoul 05006, Republic of Korea

  10. Phase transition on a context-sensitive random language model with short range interactions

    Apr 1, 2026Yuma Toji, Jun Takahashi, Vwani Roychowdhury +1Language ModelingStatistical Physics of Learning

    Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Hokkaido 060-0814, Japan · Institute for Solid State Physics, The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8581, Japan · Henry Samueli School of Engineering and Applied Science, University of California, Los Angeles, California 90095

  11. From Density Matrices to Phase Transitions in Deep Learning: Spectral Early Warnings and Interpretability

    Mar 31, 2026Max Hennick, Guillaume CorlouerPhase Transitions in Neural NetworksNeural Network Training Dynamics

  12. Kernel Dynamics under Path Entropy Maximization

    Mar 29, 2026Jnaneshwar DasStatistical Physics of Learning

    School of Earth and Space Exploration, Arizona State University, Tempe, AZ 85287, USA · Earth Innovation Hub

  13. The Symmetric Perceptron: a Teacher-Student Scenario

    Mar 26, 2026Giovanni Catania, Aurélien Decelle, Suhanee KorpeStatistical Physics of LearningTeacher-Student Learning

    Departamento de Física Teórica, Universidad Complutense de Madrid, 28040 Madrid, Spain · Institute for Cross-disciplinary Physics and Complex Systems IFISC (CSIC-UIB), Campus Universitat Illes Balears, 07122 Palma de Mallorca, Spain. · Université Paris-Saclay, CNRS, INRIA Tau team, LISN, 91190 Gif-sur-Yvette, France. +2

  14. Grokking as a Falsifiable Finite-Size Transition

    Mar 25, 2026Yuda Bi, Chenyu Zhang, Qiheng Wang +1GrokkingPhase Transitions

    Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS Center), Atlanta, Georgia 30303, USA · Beijing Institute of Technology, Beijing 100081, China · Georgia Institute of Technology, Atlanta, Georgia 30332, USA

  15. Pruning-induced phases in fully-connected neural networks: the eumentia, the dementia, and the amentia

    Mar 12, 2026Haining Pan, Nakul Aggarwal, J. H. PixleyPhase Transitions in Neural NetworksNeural Network Pruning

    Department of Physics and Astronomy, Center for Materials Theory, Rutgers University, Piscataway, NJ 08854 USA · Center for Computational Quantum Physics, Flatiron Institute, New York, New York 10010, USA

  16. Marginals Before Conditionals

    Mar 10, 2026Mihir SahasrabudheNeural Network Training DynamicsPhase Transitions in Neural Networks

    School of Information Sciences, University of Illinois Urbana-Champaign

  17. Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernels

    Mar 7, 2026Tatiana PetrovaAssociative MemoryRetrieval Robustness

    Interdisciplinary Centre for Security, Reliability and Trust · Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg

  18. Thermodynamic Response Functions in Singular Bayesian Models

    Mar 5, 2026Sean PlummerSingular Learning TheoryStatistical Physics of Learning

  19. Thermodynamic Regulation of Finite-Time Gibbs Training in Energy-Based Models: A Restricted Boltzmann Machine Study

    Mar 3, 2026Görkem Can SüleymanoğluRestricted Boltzmann MachinesEnergy-Based Models

    Owner and Software Developer, Kuanka Publishing LLC, Turkey. · Ph.D. Student in Economics, Selçuk University, Institute of Social Sciences, Turkey.

  20. Detailed balance in large language model-driven agents

    Dec 10, 2025Zhuo-Yang Song, Qing-Hong Cao, Ming-xing Luo +1Statistical Physics of Learning

    School of Physics, Peking University, Beijing 100871, China · Center for High Energy Physics, Peking University, Beijing 100871, China · Beijing Computational Science Research Center, Beijing 100193, China

  21. Can Stationary Distributions of Scale-Invariant Neural Networks Be Described by the Thermodynamics of an Ideal Gas?

    Nov 10, 2025Ildus Sadrtdinov, Ekaterina Lobacheva, Ivan Klimov +3Statistical Physics of LearningNeural Network Training Dynamics

    Constructor University · Mila – Quebec AI Institute · Université de Montréal +2

  22. Toward a Physics of Deep Learning and Brains

    Sep 26, 2025Arsham Ghavasieh, Meritxell Vila-Minana, Akanksha Khurd +3Phase Transitions in Neural NetworksStatistical Physics of Learning

    Center for Complex Networks and Systems Research, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, Indiana 47408, USA · Department of Physics, Indiana University, Bloomington, Indiana 47405, USA · Institute for Advanced Study, Princeton, NJ 08540, USA

  23. A statistical physics framework for optimal learning

    Jul 10, 2025Francesca Mignacco, Francesco MoriStatistical Physics of LearningNeural Network Training Dynamics

    1Joseph Henry Laboratories of Physics, Princeton University, Princeton, NJ 08544, USA · 2Graduate Center, City University of New York, New York, NY 10016, USA · Center of Mathematical Sciences and Applications, Harvard University, Cambridge, MA 02138, USA