A Dynamical Theory of Sequential Retrieval in Input-Driven Hopfield Networks

Simone Betteti; Giacomo Baggio; Sandro Zampieri

cs.NEMar 3, 2026

A Dynamical Theory of Sequential Retrieval in Input-Driven Hopfield Networks

Authors: Simone Betteti, Giacomo Baggio, Sandro Zampieri

Organizations: The Italian Institute of Artificial Intelligence for Industry · Department of Information Engineering, Università degli Studi di Padova

Abstract

Reasoning is the ability to integrate internal states and external inputs in a meaningful and semantically consistent flow. Contemporary machine learning (ML) systems increasingly rely on such sequential reasoning, from language understanding to multi-modal generation, often operating over dictionaries of prototypical patterns reminiscent of associative memory models. Understanding retrieval and sequentiality in associative memory models provides a powerful bridge to gain insight into ML reasoning. While the static retrieval properties of associative memory models are well understood, the theoretical foundations of sequential retrieval and multi-memory integration remain limited, with existing studies largely relying on numerical evidence. This work develops a dynamical theory of sequential reasoning in Hopfield networks. We consider the recently proposed input-driven plasticity (IDP) Hopfield network and analyze a two-timescale architecture coupling fast associative retrieval with slow reasoning dynamics. We derive explicit conditions for self-sustained memory transitions, including gain thresholds, escape times, and collapse regimes. Together, these results provide a principled mathematical account of sequentiality in associative memory models, bridging classical Hopfield dynamics and modern reasoning architectures.

Figures & tables

Figure 1: Schematic illustration of the proposed two-timescale mechanism for sequential memory transitions. At the initial time, the fast state variable is at a stable equilibrium corresponding to memory

\upxi_{1}

. The slow reasoning variable

z

progressively destabilizes this equilibrium while simultaneously stabilizing the equilibrium associated with memory

\upxi_{2}

. At the escape time

T_{\mathrm{escape}}

, the equilibrium corresponding to

\upxi_{1}

loses stability and the system transitions toward

\upxi_{2}

. This transition occurs only when the gain parameter

\kappa

of the slow dynamics exceeds a critical threshold

\kappa_{\mathrm{critical}}

. We analytically characterize both

k_{\mathrm{critical}}

and the resulting escape time

T_{\mathrm{escape}}

Figure 2: Cyclic memory transitions in one- and two-timescale Hopfield models. (A) Limit-cycle structure induced by the transition matrices

Q

in the one-timescale dynamics and

A

in the two-timescale reasoning dynamics ( 7 ), with both systems initialized near memory

\upxi^{1}

. (B-C) One-timescale model: sequential retrieval proceeds through mixed memory states, with partial overlaps and unpredictable escape times across memories. (D-E) Two-timescale reasoning model: transitions occur between individual memories without mixing, with uniform, quantifiable escape times and overlaps reaching their maximal value, indicating exact memory alignment.

Figure 3: Dependence of the slow reasoning dynamics on the gain

\kappa

and the initial condition

Z_{0}

. (A) Solutions of the discrete iteration

\mathcal{P}(Z)=Z/\kappa

as a function of the gain

\kappa

. Real solutions

Z_{\pm}>1

exist only for

\kappa\geq\kappa_{\text{critical}}=4

and sequential transitions are self-sustaining only if the initial saliency weight satisfies

Z_{0}>Z_{-}

. (B) HardTanh activation function, linear in

(-1,1)

and saturated outside this interval. (C-D) Component-wise slow reasoning dynamics for different gain regimes. (C) Subcritical gain

\kappa=3

: oscillations decay and the dynamics collapse to the origin, leading to loss of retrieval. (D) Supercritical gain

\kappa=5

: oscillations persist and entrain to the stable fixed point

Z_{+}

, enabling reliable sequential memory retrieval.

Explore similar work

Redundancy Maximization as a Principle of Associative Memory Learning in Hopfield Networks
Nov 4, 2025Mark Blümel, Andreas C. Schneider, Valentin Neuhaus +5Neural Network Memorization Hopfield Networks
Complex Systems Theory, Max Planck Institute for Dynamics and Self-Organization, Göttingen, Germany · Faculty of Physics, Institute for the Dynamics of Complex Systems, University of Göttingen · Göttingen Campus Institute for Dynamics of Biological Networks, University of Göttingen, Göttingen, Germany +1
Phases in a class of associative memories via hidden neurons
Sep 10, 2026Toshihiro Ota, Masato TakiPhase Transitions Hopfield Networks
CyberAgent AI Lab, Shibuya, Tokyo 150–0002, Japan · RIKEN iTHEMS, Wako, Saitama 351–0198, Japan · Graduate School of Artificial Intelligence and Science, Rikkyo University, Toshima, Tokyo 171–8501, Japan
Learning Hippo: Multi-attractor Dynamics and Stability Effects in a Biologically Detailed CA3 Extension of Hopfield Networks
Apr 22, 2026Daniele Corradetti, Renato CorradettiDynamical Systems Neural Processes
Grupo de Física Matemática Instituto Superior Técnico Av. Rovisco Pais, 1049-001 Lisboa, Portugal · Dipartimento di Neuroscienze, Psicologia, Area del Farmaco e Salute del Bambino (NEUROFARBA) Università di Firenze, Italy22

Nov 4, 2025J/K move · Enter open · S save

Redundancy Maximization as a Principle of Associative Memory Learning in Hopfield Networks

Mark Blümel, Andreas C. Schneider, Valentin Neuhaus, David A. Ehrlich+4

Complex Systems Theory, Max Planck Institute for Dynamics and Self-Organization, Göttingen, Germany · Faculty of Physics, Institute for the Dynamics of Complex Systems, University of Göttingen · Göttingen Campus Institute for Dynamics of Biological Networks, University of Göttingen, Göttingen, Germany+1

Neural Network Memorization Hopfield Networks Associative Memory

Open paper AskPDF arXiv

Associative memory, traditionally modeled by Hopfield networks, enables the retrieval of previously stored patterns from partial or noisy cues. Yet, the local computational principles which are required to enable this function remain incompletely understood. To formally characterize the local information processing in such systems, we employ a recent extension of information theory -- Partial Information Decomposition (PID). PID decomposes the contribution of different inputs to an output into unique information from each input, redundant information across inputs, and synergistic information that emerges from combining different inputs. Applying this framework to individual neurons in classical Hopfield networks we find that below the memory capacity, the information in a neuron's activity is characterized by high redundancy between the external pattern input and the internal recurrent input, while synergy and unique information are close to zero until the memory capacity is surpassed and performance drops steeply. Inspired by this observation, we use redundancy maximization at each neuron as an information-theoretic learning goal. This dramatically increases the network's memory capacity to 1.59, a more than tenfold improvement over the 0.14 capacity of classical Hopfield networks, and also outperforming recent state-of-the-art implementations of Hopfield networks. Overall, this work establishes redundancy maximization as a new design principle for associative memories and opens pathways for new associative memory models based on information-theoretic goals.

Abstract

Explore similar work

Redundancy Maximization as a Principle of Associative Memory Learning in Hopfield Networks

Phases in a class of associative memories via hidden neurons

Learning Hippo: Multi-attractor Dynamics and Stability Effects in a Biologically Detailed CA3 Extension of Hopfield Networks