cond-mat.stat-mechAug 8, 2026

High-Capacity Generalized Hopfield Networks

Authors: Victor Galitski

Organizations: Joint Quantum Institute, Department of Physics, University of Maryland, College Park, MD 20742, USA

Abstract

Generalized Hopfield networks are introduced where memories and neurons are continuous variables that lie on a Riemannian manifold. We explicitly focus on symmetric spaces associated with the special unitary groups SU(d), and use both numerical and analytical (replica) techniques to demonstrate an almost order of magnitude enhancement in critical capacity over the vector networks starting with d=3 and further rapidly growing with d. To circumvent the non-linear geometric constraints, we use a Lie algebraic method [following V. Galitski, Phys. Rev. A 84, 012118 (2011)] to exactly describe the classical neural network in terms of linear algebra in an auxiliary Hilbert space. It is shown that in contrast to the traditional Hopfield networks, memory recall in SU(d) Hopfields corresponds to neuron alignment along a top eigenvector of a spiked matrix, which is less susceptible to random matrix crosstalk than other models with continuous neuron variables. Physical platforms to realize SU(d) Hopfields are briefly discussed and physical (in addition to algorithmic) recall mechanism is demonstrated, where memory recovery occurs naturally through generalized Landau-Lifshitz-Gilbert dynamics. To illustrate SU(3) memory recall, we introduce a color (RGB) image encoding/decoding protocol and explicitly run image recovery on corrupted cues. Finally, we quantize the generalized Hopfields which are shown to reduce to Sachdev-Ye glassy type of models. Their many-body spectra generally feature two types of dark and memory bands, where the latter exhibits chaotic Wigner-Dyson level statistics that hides Hebbian data.

Explore similar work

CardsList
Jun 8, 2026q-bio.NC

Hyperbolic Neural Population Geometry Benefits Computation

Neural population geometry shapes downstream computation. Recent empirical findings in neurobiology suggest that a hyperbolic structure underlies population activity in the hippocampus. Here we provide a theoretical framework for this phenomenon. First, we propose a plausible construction of hippocampal tuning curves that statistically induces hyperbolic geometry. Next, we establish a connection between neural decoding and associative memory by demonstrating that the Modern Hopfield Network update rule computes the minimum mean-squared-error (MMSE) estimator. Finally, we introduce a novel associative memory model defined in hyperbolic space that yields significantly larger capacity than leading models. Our results suggest that animals encode spatial information as a latent hyperbolic cognitive map, improving both memory capacity and decoding accuracy.
Dennis Wu, Yi-Chun Hung, Braden Yuille +2
Hopfield NetworksAssociative MemoryHuman Visual CortexCognitive Map
AskPDF
May 23, 2026cs.LG

Beyond Fixed Points: Superpolynomial Capacity of Asymmetric Hopfield Networks

Classical Hopfield networks are limited to static patterns due to symmetric weights, whereas asymmetric networks can encode temporal sequences via limit-cycle attractors. Achieving high-capacity storage of long sequences in classical synchronous asymmetric networks, however, has remained a challenge. We present a simple and robust construction within the classical asymmetric Hopfield model with binary neurons and synchronous updates, that allows nn neurons to support exp⁡ ⁣(Ω(n/(log⁡n)2))\exp\!\big(Ω(n/(\log n)^2)\big) distinct limit-cycle attractors, each with period exp⁡ ⁣(Ω(n/log⁡n))\exp\!\big(Ω(\sqrt n/\log n)\big) and robust to random noise with flip probability up to 12−o(1)\frac12-o(1), yielding superpolynomial capacity in both the number and length of stored sequences. This is the first demonstration of such capacity for asymmetric Hopfield networks, which we obtain by combining results from combinatorics, number theory and the analysis of opinion dynamics. Our findings show that synchronous asymmetric Hopfield networks possess a sequence-memory capacity which is larger and more robust than previously recognized, demonstrating that, in both biological and artificial neural systems, robust sequence representation can be achieved through coarse architectural motifs rather than complex nonlinearities.
Aakash Kumar, Anatoly Khina, Frederik Mallmann-Trenn +1
Hopfield NetworksAttractorsCapacityFixed-Point IterationApproximation Algorithms
AskPDF
Sep 11, 2026cs.LG

Phases in a class of associative memories via hidden neurons

Associative memory in the Hopfield network is attractor dynamics in a disordered many-body system, and higher-order and exponential extensions turn its retrieval update into softmax attention. The polynomial and exponential regimes have been analyzed by different methods, with no common architecture in which to ask what fixes the storage scale. In this paper we study the bipartite architecture of Krotov and Hopfield, which we call the class HH, whose model is fixed by a Lagrangian for each layer, taking the hidden neurons as the order parameter of retrieval. At polynomial load the replica method yields the replica-symmetric phase diagrams and closed-form capacities, and the crosstalk moment is common to Ising and spherical visible neurons, so their differences come from the visible entropy. With a softmax hidden layer the load is exponential, and a copy representation maps the thermodynamics onto random-energy-model counting, with paramagnetic, condensed, and frozen phases. Heating destabilizes retrieval by quantized reassignments of attention, and typical Gaussian patterns remain metastable at every load. The regimes differ in their crosstalk statistics, central-limit at polynomial load and large-deviation at exponential load, and the class HH splits retrieval into two roles, the visible Lagrangian fixing stability and the hidden one the storage scale, two axes that may also guide the design of new Lagrangians.
Toshihiro Ota, Masato Taki
Hopfield NetworksAssociative MemoryPhase Transitions
AskPDF