cs.LGNov 6, 2026

Deep Learning Theory through the Lens of Statistical Physics: A Topical Review

Authors: Wei Huang, Youjin Deng

Organizations: RIKEN Center for Advanced Intelligence Project (AIP), Tokyo, Japan · The Institute of Statistical Mathematics, Tokyo, Japan · Hefei National Research Center for Physical Sciences at the Microscale and School of Physical Sciences, University of Science and Technology of China, Hefei 230026, China · Hefei National Laboratory, University of Science and Technology of China, Hefei 230088, China · 

Abstract

The recent success of foundation models and deep learning has far outpaced our theoretical understanding of how large-scale neural networks learn and generalize. Statistical physics provides a natural language for describing such high-dimensional, nonlinear, and strongly coupled systems, offering concepts such as mean-field theory, phase transitions, energy landscapes, and stochastic dynamics. In this topical review, we adopt a statistical-physics way of thinking -- from concepts including phases and criticality, and free-energy viewpoints, to methods such as kernel and Gaussian-process limits, mean-field and dynamical mean-field theories, random-matrix tools, and Fokker-Planck formalisms, and to a mindset emphasizing universality, scaling, and tractable limits with explicit validity regimes. We survey recent progress on understanding modern deep learning through this lens, including mean-field analyses of wide networks, spectral and geometric perspectives on loss landscapes, stochastic-gradient-based training viewed as a dynamical process, and phase-transition analogies that organize phenomena such as overparameterization, generalization, and representation change. Our goal is to synthesize these threads into a coherent framework that clarifies what each approximation explains, when it is expected to hold, and how apparently competing narratives relate or diverge. Together, these perspectives form a unified physical framework that views representation learning as a collective phenomenon in high-dimensional systems, bridging deterministic and stochastic regimes of gradient-based optimization.

Figures & tables

Explore similar work

CardsList
  1. Statistical Properties of Training & Generalization

    Jun 18, 2026Itay Lavie, Noam Levi, Yonatan KahnNeural Network GeneralizationScientific ML

  2. Lecture Notes on Statistical Physics and Neural Networks

    May 7, 2026Olaf Hohmspin glassPhase Transitions

  3. There Will Be a Scientific Theory of Deep Learning

    Apr 23, 2026Jamie Simon, Daniel Kunin, Alexander Atanasov +11Neural Network Training DynamicsStatistical Learning Theory