Organizations: Robust and Intelligent Autonomous Systems (RIAS) lab, The Italian Institute of Artificial Intelligence (AI4I) · Department of Information Engineering, Università degli Studi di Padova, Padova, 35131, IT · Department of Information Engineering, Padua Neuroscience Center (PNC), Università degli Studi di Padova, Padova, 35131, IT
Associative memory models classically describe retrieval as convergence to stored patterns, but the theory of sequential retrieval remains comparatively limited. We develop a two-timescale input-driven Hopfield network in which fast associative retrieval is modulated by a slow feedback variable evolving over a prescribed memory-transition graph. This yields autonomous, graph-constrained transitions while preserving analytical tractability. Using slow-fast and geometric singular perturbation theory, we characterize memory fixation via memory manifolds and identify the geometric regions that govern stability loss and escape from retrieved memories. We further derive reduced slow dynamics and explicit criteria for transition onset, self-sustained retrieval, and collapse, expressed through escape times, gain thresholds, and fixed points. Together, these results provide a tractable framework for analyzing feedback-driven sequential retrieval in Hopfield networks.
Figures & tables
Figure 1: Schematic illustration of the proposed two-timescale mechanism for sequential memory transitions. At the initial time, the fast state variable is at a stable equilibrium corresponding to memory \upxi1 . The slow variable z progressively destabilizes this equilibrium while simultaneously stabilizing the equilibrium associated with memory \upxi2 . At the escape time Tesc , the equilibrium corresponding to \upxi1 loses stability and the system transitions toward \upxi2 . This transition occurs only when the gain parameter κ of the slow dynamics exceeds a critical threshold κcrit . We analytically characterize both κcrit and the resulting escape time Tesc .
Figure 2: (A) Block diagram of the IDP Hopfield model; setting τx=1 recovers ( 1 ). (B) Block diagram of the two-timescale sequential IDP Hopfield model ( 3 ). (C) Example of a deterministic transition graph GA , where nodes represent stored memories and edges encode admissible transitions. The graph decomposes into multiple connected components (highlighted by dashed red contours), each defining a distinct memory sequence; one, selected by x(0) , is highlighted in blue. (D) One-timescale model with orthogonal memories ( κ=5 ): sequential retrieval proceeds through mixed memory states, with partial overlaps and unpredictable escape times across memories. (E-F) Two-timescale sequential IDP Hopfield model ( τx=1,τz=5,κ=5 ). (E) Orthogonal memories: transitions occur between individual memories without mixing, with uniform escape times and overlaps reaching their maximal value, indicating exact memory alignment. (F) Random memories: transitions occur without mixing, reaching maximum overlap, but escape times are no longer uniform. Numerical experiments are shown for visualization purposes with P=15 memories and a limit cycle over 5 of them, with \uppsi=tanh , and whose overlaps are highlighted in different colors in the plots.
Figure 3: (A) Sketch of the transitions between different memory manifolds in the slow-fast Hopfield model. The blue curves corresponds to the slow movement of the system close to a stable memory manifold, whereas the purple curves correspond to the fast switching between different stable memory manifolds. Each memory manifold N\upmu is decomposed into its attracting (red) and saddle-type (green) memory manifolds M\upmua , M\upmus . (B) Simulated trajectory projected onto the fast subspace in a slow-fast Hopfield model storing two memories. Continuous curves indicate the flow along memory branches N\upmu (magenta) and N\upnu (blue), whereas dashed curves indicate rapid transitions from one memory branch to another. Switching occurs after the trajectory has moved from the attracting part of the memory branches M\upnua,ε and M\upmua,ε , respectively, to the saddle-type section M\upnus,ε and M\upmus,ε . The black dots indicate the relevant point of separation between M\upnua and M\upnus , and M\upmua and M\upmus , respectively (red/green). (C) Temporal series of the variable x\upnu . Colors and line types as in panel B.
Figure 4: (A) Fixed points of the discrete map Z+=h(Z/X) with κ=15 for the SmoothF activation, with predicted Zˉ2≃6.78 (see Fig. 9 in Appendix E for \uppsi=tanh ). Real solutions Zˉ1,Zˉ2>1 emerge only for κ≥κcrit≃12.74 ; self-sustained transitions require Z0>Zˉ1 . (B) SmoothF (blue) and Tanh (green) activation functions. (C, D) Orthogonal memories with SmoothF activation. (C) Subcritical gain ( κ=9 ): oscillations decay and retrieval collapses to the origin. (D) Supercritical gain ( κ=15 ): persistent oscillations converge to the stable fixed point Zˉ2≃6.95 , close to theoretical prediction, enabling reliable sequential retrieval. (E, F) Random memories with SmoothF activation. (E) Subcritical gain: cross-talk noise sustains small-amplitude transitions. (F) Supercritical gain: transitions persist but exhibit amplitude fluctuations without entrainment to Zˉ2 . (G,H) Random memories with Tanh activation. (G) Subcritical gain ( κ=3 ) leads to collapse, with slower decay due to cross-talk. (H) Supercritical gain ( κ=5 ) yields self-sustained transitions. Panels (C-H) use N=256 , P=15 , and a directed cycle of length L=5 .
Figure 5: (A) Theoretical and numerical escape times Tesc and observed switching times Tswitch across consecutive transitions ( κ=15 , Z0=8 ). (B) Asymptotic escape time versus κ : theory and numerical estimates. (C) Mean switching delay Tswitch−Tesc versus Gaussian noise amplitude σ . (D) Probability of correct transitions in the (κ,ϵ) plane. (E) Asymptotic escape time versus κ and memory bias ϵ . (F) Asymptotic saliency amplitude versus κ and ϵ . The numerical configuration is the same as in Fig. 4 . Panels (A-C) use orthogonal memories, while (D-F) use biased random memories, with 10 independent realizations per parameter pair. In (B), asymptotic escape times are estimated over the last 8 time units of simulations of duration T=40 (approximately 5-7 transitions). Panel (C) uses Gaussian noise with standard deviation σ/N and 64 realizations per noise amplitude. Hatched regions in (D-F) indicate no successful transitions.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Graphical illustration of the two possible outcomes of the source-successor saliency dynamics. Left: no transition occurs and the dynamics decay to inactivity. Right: the successor saliency catches up with the source saliency, triggering a transition between memories.
Figure 7: Plot of the function G(Z,X) in ( 60 ) for κ=15 and Z=4 .
Figure 8: Plot of the function G(X2,X) in ( 63 ) and its intersections with the line 1/κ .
Figure 9: Discrete iteration map Z+=h(Z/X) for the \uppsi=tanh activation function and associated z -components for the case of κ=15 .
Figure 10: Numerical validation and robustness analysis for Tanh activation. (A) Theoretical and numerical escape times and observed switching times across consecutive transitions. (B) Asymptotic escape time versus κ : theory and numerical estimates. (C) Mean switching delay versus Gaussian noise amplitude. (D) Probability of correct transitions in the (κ,ϵ) plane. (E, F) Asymptotic escape time and saliency amplitude, respectively. All experiments use the same numerical configuration as Fig. 5 . Hatched regions in (D-F) indicate no successful transitions.
Figure 11: Full-sequence retrieval probability versus memory load P/N . (A, B) Growing cycles with L≃P/4 and N∈{256,512} for SmoothF and for Tanh, respectively. (C, D) Fixed cycles with L=5 and N=256 , for SmoothF and Tanh, respectively. Shaded regions indicate 95% confidence intervals across independent memory realizations.
Figure 12: Finite state machine dynamics with uncertain inputs realized by a two-timescale IDP Hopfield model. (A) Schematic of a simplified Tokyo subway graph comprising three metro lines connected by two hub stations. The metro driver (red marker) must adapt the route in response to an external rerouting request, while facing uncertainty at each hub regarding the correct branch to take. (B) Sequential retrieval of stable memory states corresponding to stations along the habitual red-black line, illustrating deterministic FSM dynamics under standard operating conditions. (C) Input- and state-dependent sequential retrieval during rerouting: external inputs bias the transition dynamics at each hub toward the green-black line (via Shinjuku station) and subsequently the blue-black line (via Tokyo station), while preserving residual uncertainty across alternative transitions.
CyberAgent AI Lab, Shibuya, Tokyo 150–0002, Japan · RIKEN iTHEMS, Wako, Saitama 351–0198, Japan · Graduate School of Artificial Intelligence and Science, Rikkyo University, Toshima, Tokyo 171–8501, Japan