Dynamical Mean-Field Theory

Latest papers

8 papers0 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Learning-Induced Dynamical Transition in Recurrent Neural Networks
Sep 16, 2026Varun VaidyaDynamical Systems Recurrent Neural Networks
Department of Physics, University of South Dakota, Vermillion 57069, USA
Broken Ergodicity and the Violation of the Fluctuation-Dissipation Theorem Lead to Generalization Beyond Overfitting in Machine Learning
Jul 5, 2026Chan Li, Nigel GoldenfeldDynamical Mean-Field Theory Statistical Physics of Learning
Department of Physics, University of California San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA. · Halıcıoğlu Data Science Institute, University of California San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA
How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization
May 13, 2026Leena Chennuru Vankadara, Moritz Haas, Luke Hayward +2Mixture-of-Experts Models Dynamical Mean-Field Theory
1Gatsby Computational Neuroscience Unit, University College London · 2Amazon · University of Tübingen, Tübingen AI Center
Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer
May 8, 2026Clarissa Lauditi, Cengiz Pehlevan, Blake BordelonDeep Linear Networks Neural Network Training Dynamics
John A. Paulson School of Engineering and Applied Sciences, Harvard University · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University · Center for Mathematical Sciences and Applications, Harvard University +1
ResNets of All Shapes and Sizes: Convergence of Training Dynamics in the Large-scale Limit
Mar 18, 2026Louis-Pierre Chaintron, Lénaïc Chizat, Javier MaassDynamical Mean-Field Theory Neural Network Training Dynamics
École Polytechnique Fédérale de Lausanne (EPFL), Institute of Mathematics, 1015 Lausanne, Switzerland.
Rigorous Asymptotics for First-Order Algorithms Through the Dynamical Cavity Method
Mar 15, 2026Yatin Dandi, David Gamarnik, Francisco Pernice +1Approximate Message Passing Dynamical Mean-Field Theory
Statistical Physics of Computation Laboratory, École polytechnique fédérale de Lausanne (EPFL) CH-1015 Lausanne. · Sloan School of Management, Operations Research Center and Institute of Data, Systems and Society (IDSS), MIT. · CSAIL and LIDS, MIT.
A Gaussian Comparison Theorem for Training Dynamics in Machine Learning
Mar 10, 2026Ashkan PanahiDynamical Mean-Field Theory Neural Network Training Dynamics
Department of Computer Science and Engineering, Chalmers University, Gothenburg, Sweden
High-dimensional learning dynamics of multi-pass Stochastic Gradient Descent in multi-index models
Jan 28, 2026Zhou Fan, Leda WangDynamical Mean-Field Theory Stochastic Gradient Descent
Department of Statistics and Data Science, Yale University

Sep 16, 2026J/K move · Enter open · S save

Learning-Induced Dynamical Transition in Recurrent Neural Networks

Varun Vaidya

Department of Physics, University of South Dakota, Vermillion 57069, USA

Dynamical Systems Recurrent Neural Networks Neural Network Training Dynamics Dynamical Mean-Field Theory Phase Transitions in Neural Networks

Open paper AskPDF arXiv

Learning in recurrent neural networks can fundamentally reshape their underlying dynamics, transforming initially chaotic activity into stable task-dependent behavior. We develop a non-equilibrium dynamical mean-field theory(DMFT) to describe this transition during learning. We show that a slow feedback-driven learning process generates an evolving effective feedback strength that drives the network through a transition from chaotic to stable dynamics defined by a bifurcation of the DMFT solution. By deriving the two-time correlation function throughout learning, we identify a critical feedback strength and a corresponding learning rate dependent critical time separating these regimes. The transition arises from the progressive deformation of an effective dynamical landscape by the growing learned feedback structure. Starting from the untrained state, the theory predicts the time evolution of the network output during training and shows quantitative agreement with numerical simulations.