cond-mat.dis-nnMar 29, 2023

Storage and Learning phase transitions in the Random-Features Hopfield Model

Authors: Matteo Negri, Clarissa Lauditi, Gabriele Perugini, Carlo Lucibello, Enrico M. Malatesta

Organizations: CNR-NANOTEC, Institute of Nanotechnology, Rome Unit, Piazzale Aldo Moro, 00185 Roma, Italy · Institute for Data Science and Analytics, Bocconi University

Abstract

The Hopfield model is a paradigmatic model of neural networks that has been analyzed for many decades in the statistical physics, neuroscience, and machine learning communities. Inspired by the manifold hypothesis in machine learning, we propose and investigate a generalization of the standard setting that we name Random-Features Hopfield Model. Here PP binary patterns of length NN are generated by applying to Gaussian vectors sampled in a latent space of dimension DD a random projection followed by a non-linearity. Using the replica method from statistical physics, we derive the phase diagram of the model in the limit P,N,D→∞P,N,D\to\infty with fixed ratios α=P/Nα=P/N and αD=D/Nα_D=D/N. Besides the usual retrieval phase, where the patterns can be dynamically recovered from some initial corruption, we uncover a new phase where the features characterizing the projection can be recovered instead. We call this phenomena the learning phase transition, as the features are not explicitly given to the model but rather are inferred from the patterns in an unsupervised fashion.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Hierarchical Prototype Emergence in Modern Hopfield Models

    Sep 10, 2026Aditya Cowsik, Adithya SriramNeural Network GeneralizationHopfield Networks

  2. Phase Transitions in Learning Driven by Extremal Data

    Oct 21, 2026Ethan Levien, Trevor GrandPreMultiple Instance LearningPhase Transitions in Neural Networks