Synthesis of Hopfield Neural Network: Novel Results

Classical Hopfield networks are limited to static patterns due to symmetric weights, whereas asymmetric networks can encode temporal sequences via limit-cycle attractors. Achieving high-capacity storage of long sequences in classical synchronous asymmetric networks, however, has remained a challenge. We present a simple and robust construction within the classical asymmetric Hopfield model with binary neurons and synchronous updates, that allows $n$ neurons to support $\exp\!\big(Ω(n/(\log n)^2)\big)$ distinct limit-cycle attractors, each with period $\exp\!\big(Ω(\sqrt n/\log n)\big)$ and robust to random noise with flip probability up to $\frac12-o(1)$, yielding superpolynomial capacity in both the number and length of stored sequences. This is the first demonstration of such capacity for asymmetric Hopfield networks, which we obtain by combining results from combinatorics, number theory and the analysis of opinion dynamics. Our findings show that synchronous asymmetric Hopfield networks possess a sequence-memory capacity which is larger and more robust than previously recognized, demonstrating that, in both biological and artificial neural systems, robust sequence representation can be achieved through coarse architectural motifs rather than complex nonlinearities.

The dynamics of the original Hopfield network is asynchronous (sequential) (updates the state of only one neuron per time step). In this paper, we propose a new tool and a new dynamics to reduce the processing time by updating one or more neurons simultaneously per instant while ensuring process convergence and aiming for the maximum energy decrease at each step, thus guaranteeing the shortest total processing time. From the point of view of synchronous dynamics, calculating the next network state at which energy decreases the most from the current state while ensuring convergence is itself a combinatorial optimization problem. We develop and use a new tool to solve it. We call this new tool Discrete Differential Filter (DDF) and, based upon it, we develop a new synchronous dynamics which we call SD-DDF (Synchronous Dynamics based upon Discrete Differential Filter). In this paper, we review the original asynchronous dynamics for Hopfield networks and present a new tool and a new synchronous dynamics with its theoretical justification and four computational experiments to assess the speed up in processing time empirically.

Neural population geometry shapes downstream computation. Recent empirical findings in neurobiology suggest that a hyperbolic structure underlies population activity in the hippocampus. Here we provide a theoretical framework for this phenomenon. First, we propose a plausible construction of hippocampal tuning curves that statistically induces hyperbolic geometry. Next, we establish a connection between neural decoding and associative memory by demonstrating that the Modern Hopfield Network update rule computes the minimum mean-squared-error (MMSE) estimator. Finally, we introduce a novel associative memory model defined in hyperbolic space that yields significantly larger capacity than leading models. Our results suggest that animals encode spatial information as a latent hyperbolic cognitive map, improving both memory capacity and decoding accuracy.