Leakage and Second-Order Dynamics Improve Hippocampal RNN Replay

Josue Casco-Rodriguez; Nanda H. Krishna; Richard G. Baraniuk

cs.LGFeb 20, 2026

Leakage and Second-Order Dynamics Improve Hippocampal RNN Replay

Authors: Josue Casco-Rodriguez, Nanda H. Krishna, Richard G. Baraniuk

Organizations: Department of Electrical and Computer Engineering, Rice University, USA · Mila - Quebec AI Institute & Université de Montréal, Canada

Abstract

Biological neural networks (like the hippocampus) can internally generate "replay" resembling stimulus-driven activity. Recent computational models of replay use noisy recurrent neural networks (RNNs) trained to path-integrate. Replay in these networks has been described as Langevin sampling, but new modifiers of noisy RNN replay have surpassed this description. We re-examine noisy RNN replay as sampling to understand or improve it in three ways: (1) Under simple assumptions, we prove that the gradients replay activity should follow are time-varying and difficult to estimate, but readily motivate the use of hidden state leakage in RNNs for replay. (2) We confirm that hidden state adaptation (negative feedback) encourages exploration in replay, but show that it incurs non-Markov sampling that also slows replay. (3) We propose the first model of temporally compressed replay in noisy path-integrating RNNs through hidden state momentum, connect it to underdamped Langevin sampling, and show that, together with adaptation, it counters slowness while maintaining exploration. We verify our findings via path-integration of 2D triangular and T-maze paths and of high-dimensional paths of synthetic rat place cell activity.

Figures & tables

Panel of Figure 1 — Figure 1: Underdamped dynamics accelerate offline replay, adaptation slows it. Here we simulate a noisy RNN $r(t)$ that optimally path-integrates an Ornstein-Uhlenbeck process $s(t)$ from its velocity $s^{\prime}(t)$ . We assume $r(t)$ minimizes the loss in Equation 8 and thus evolves according to its score function $\nabla_{r(t)}\log p(r(t))$ (Equations 11 and 15 ), performing a variant of Langevin sampling when no input is given. Above, we compare three modifiers of RNN activity: the default (no modification, a.k.a. overdamped ), our proposed underdamped (momentum), and adaptation (negative feedback) dynamics. Each modifier affects the replay distribution $p(r(t))$ in different ways: underdamped sampling accelerates $p(r(t))$ towards $p(s(t))$ , decreasing the distance between them, while adaptation slows convergence of $p(r(t))$ towards $p(s(t))$ , increasing this distance.

Panel of Figure 2 — Figure 2: Leakage helps path-integration. Here we train RNNs on two tasks, ablating the leakage term. Leakage helps training, especially when losses increase with the masking difficulty $k$ (note that $k=1$ is equivalent to unmasked training, see Equation 13 ). Means are solid, standard deviations are faint.

Panel of Figure 3 — Figure 3: Underdampening and adaptation counter each other. Here we show replay from RNNs trained to path-integrate in T-maze or triangular environments. (a) Awake paths in each task form a mixture of Ornstein-Uhlenbeck processes, one for each direction of travel. Awake paths reach their endpoints and stay there. In (b) and (c) are mean replay paths (darkening over time) from T-maze and triangle tasks, simulated for the same time as awake paths. Standard deviations are faint, ideal path means are dashed. Like in Figure 1 , adaptation slows convergence towards endpoints, underdampening quickens it. Also, the two mechanisms induce deviations that negate each other.

Panel of Figure 4 — Figure 4: Underdampening improves replay fidelity in the presence of adaptation. We compute the Wasserstein distance (dissimilarity) between awake and replay path distributions ( $p(\left\{\bm{s}(t)\right\}_{t=0}^{T})$ and $p(\left\{\bm{r}(t)\right\}_{t=0}^{T})$ ), varying friction and adaptation strength (see Appendix A for details). While the two mechanisms both generally increase this distance, underdampening ( $\lambda_{v}<1$ ) decreases it if adaptation is nonzero. Like in Figure 3 , underdampening counters adaptation-induced deviations.

Panel of Figure 5 — Figure 5: Underdampening temporally compresses replay. We calculate how long it takes awake and replay trajectories to reach their endpoints. Underdampening ( $\lambda_{v}<1$ ) not only shortens this reach time, but makes it smaller than that of awake paths, temporally compressing awake activity. See Appendix F Figure 9 for mean reach times. We do not include unbiased rat trajectories because they do not have defined endpoints, but we do confirm in Appendix F Figure 10 that underdampening quickens them.

Panel of Figure 6 — Figure 6: On average, underdampening increases exploration via path length. In here and Figure 7 we simulate replay paths for more time than awake paths, varying friction and adaptation strength. Shown above, underdampening ( $\lambda_{v}<1$ ) increases the average length of replay paths. It may do so via increased replay speeds ( Appendix F Figure 10 ) or via additional path transitions ( Figure 7 ); the latter is how adaptation increases path length.

Panel of Figure 7 — Figure 7: On average, underdampening increases exploration via regions visited. Another relevant aspect of exploration is transitions not present in awake activity. (a) Here we quantify these transitions via regions visited . Each region is an area where every point is closest to the same endpoint. In our T-maze and triangle tasks, awake paths go from starting points to endpoints and stay there, visiting two regions total (starting and ending). Meanwhile, adaptation ( $b_{a}=1$ ) can make replay paths visit multiple endpoints (regions visited ¿ 2). (b) When we use adaptation and underdampening ( $\lambda_{v}<1$ ), we see that replay paths might visit (i) more, (ii) fewer, or (iii) the same regions (see Appendix F Figure 12 for more details). (c) However, on average, across several different trained models, underdampening increases regions visited, thus increasing adaptation.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Figure 8: Example decoded 2D replay trajectories in unbiased (left) and biased (right) rat tasks. Like in Figure 3 , trajectories get darker over time.

Panel of Figure 9 — Figure 9: This is another version of Figure 5 , but now with mean reach times also shown.

Panel of Figure 10 — Figure 10: Underdampening ( $\lambda_{v}<1$ ) increases mean displacement of replay trajectories, especially at early timesteps. Similarly to ( McNamee et al., 2021 ) , here we analyze the mean displacement , or distance, of replay trajectories from their starting points as a function of time. For reference, mean displacement over time is also plotted for awake trajectories. In exploration experiments, we simulate replay for $4\times$ the duration of awake trajectories. Higher mean displacements over time generally correlate with increased path length and exploration. Note that awake activity is always calculated with $b_{a}=0,\lambda_{v}=1$ , hence why awake statistics are the same across plots within the same task.

Panel of Figure 11 — Figure 11: Underdampening ( $\lambda_{v}<1$ ) increases replay trajectory variances. Like in Figure 10 , here we analyze the variance of trajectories at each timestep. Trajectories are 2D, so the variances plotted above are simply the averages of the variances along each coordinate. Underdampening increasing variance is another confirmation that underdampening complements adaptation-induced exploration.

Panel of Figure 12 — Figure 12: This is another version of Figure 7 , but this time with the specific distributions of regions visited shown as bar plots above pairs of replay trajectory sets. Recall that replay paths start at yellow circles and end at $\times$ symbols. In (a), underdampening clearly increases regions visited for several trajectories: instead of just going from $(1,0)$ to ( $0,0)$ , replay paths also continue onwards to traverse between $(0,0)$ and $(1/2,\sqrt{3}/2)$ . Meanwhile in (b), underdampening decreases regions visited for many replay trajectories: instead of traversing the whole triangle (4 regions visited), many replay paths stop in the region defined by $(1,0)$ . As for (c), underdampening did not affect regions visited since all paths started at $(1/2,\sqrt{3}/2)$ , visited $(1,0)$ , and then returned to $(1/2,\sqrt{3}/2)$ .

Panel of Figure 13 — Figure 13: Mild underdampening can improve replay fidelity in the presence of adaptation in tanh RNNs. This is another version of Figure 4 , for T-maze and triangle tasks, but with tanh RNNs.

Panel of Figure 14 — Figure 14: Underdampening increases speed in tanh RNN replay. This is another version of Figure 9 for T-maze and triangle tasks, but now with tanh RNNs.

Panel of Figure 15 — Figure 15: In tanh RNNs, underdampening may increase or decrease exploration via path length. This is an alternate version of Figure 6 but with tanh RNNs. Underdampening has mixed effects on tanh RNN replay path length because of tanh saturation.

Explore similar work

Apr 17, 2026

Impact of leaky dynamics on predictive path integration accuracy in recurrent neural networks

Experimental evidence indicates that intrinsic temporal dynamics operating across multiple time scales are closely associated with the emergence of periodic spatial activity of increasing complexity. However, how information encoded in grid-like firing patterns for path integration is processed across these intrinsic time scales remains unclear. To address this question, we introduce adaptive time scales through a leak term in recurrent neural networks (RNNs), forming leaky RNNs discretized from the continuous attractors of firing rate models. Our results demonstrate that leaky RNNs substantially enhance the emergence of well-defined and highly regular hexagonal firing patterns. Compared with vanilla RNNs lacking a leak term, the trained leaky RNNs produce more accurate position estimates while generating reliable grid-cell-like representations. Furthermore, under identical noise conditions, leaky RNNs consistently exhibit more stable dynamics and better-defined grid structures. The learned dynamics also give rise to stable torus attractors with a clear central hole, supporting robust and regular grid-like activity. Overall, the dynamic leak acts as a low-pass filtering mechanism that protects recurrent neural circuitry from noise, stabilizes network dynamics, and improves path-integration accuracy in recurrent neural networks.

Yanlin Zhang, Yan Zhang, Muhua Zheng +1

School of physics and Electronic Engineering, Jiangsu University, Zhenjiang, Jiangsu 212013, China · Teaching Department, Jiangsu University Jingjiang College, Zhenjiang 212028, Jiangsu, China

Recurrent Neural Networks

Ask PDF

Sep 16, 2026

Neural noise enables accurate internal simulation of rare events

The brain needs an accurate internal model of the world to generate predictions and guide behavior. However, it must estimate the statistical structure of the environment from limited experience. This is particularly difficult for rare events, whose observed frequencies in a limited sample may substantially under- or overestimate their true frequencies. How the brain constructs an accurate internal model despite this sampling problem remains unclear. We address this problem using a Bayesian Confidence Propagation Neural Network (BCPNN) trained on event sequences from a Markov-chain random walk with controlled event frequencies. Treating the underlying Markov structure as the ground truth, we train the network on limited sample of event sequences and then allow it to generate autonomous replay based on the learned structure. We evaluate replay fidelity at the levels of both marginal event frequencies and conditional transition structure. We find that moderate neural noise, modeled as temporally correlated random fluctuations in unit activity during replay, is critical for faithful internal simulation. Without this variability, deterministic replay systematically under- or overrepresents rare events, whereas moderate noise restores both their marginal and conditional occurrence. Moderate noise also broadens the range of parameter values that produce accurate replay, making the model more robust to parameter variation. Together, these results support noise-assisted internal simulation as a potential mechanism for compensating for sampling errors arising from limited experience. Our model also provides a testable framework for investigating how altered neural variability may impair internal-model fidelity in disorders such as Parkinson's disease.

Heng Zhang, Pawel Herman, Zenas C. Chao

International Research Center for Neurointelligence (WPI-IRCN), UTIAS, The University of Tokyo, Japan · Division of Computational Science and Technology, KTH Royal Institute of Technology, Stockholm, Sweden

Computational Cognitive Modeling Rare-Event Estimation

Ask PDF

Jun 8, 2026

Backward Coherence and Hidden-State Stability in Recurrent Neural Networks: A Quasi-Reverse-Martingale Theory

Recurrent neural networks maintain a hidden state

h_t

, but its probabilistic meaning is often unclear. We study hidden-state stability through backward coherence: the extent to which

h_t

can be reconstructed from

h_{t+1}

by a learned backward projector

g_φ

. Under contraction and summable backward drift, the hidden-state sequence forms a quasi-reverse-martingale. This yields almost-sure convergence, rates under mixing, an interpretable limiting representation, finite pathwise stopping times, and a theoretical framework for time-uniform confidence sequences. Simulations support the theory. Backward-coherence regularisation reduces the empirical quasi-martingale total

\hat Q

43

58%

, reaches stability

28

44%

earlier than an unregularised RNN, and gives tracking-error recovery consistent with geometric bounds. Additional tests confirm echo-state forgetting rates bounded by

ρ

and verify the increment-sum tube

R_t

with

100%

simultaneous coverage, although

R_t

is conservative; in practice, the defect-tail proxy

\hat Q_t

is the more useful monitor. The backward-coherence loss is also equivalent to minimising a Kullback--Leibler divergence in a Gaussian backward model, linking the method to variational inference. Extensions cover

φ

-mixing inputs, change-point tracking, and finite-sample concentration. Three real-data studies further validate the approach. On PhysioNet 2012 ICU data, the Reverse Martingale RNN (RMRNN) matches RNN mortality-prediction AUC while reaching stable representations 13 hours earlier. On FRED-MD, it reduces one-month-ahead forecast error by about fourfold under concept drift. On UCI Human Activity Recognition, it maintains lower post-transition tracking error with geometric decay. The guarantees apply under the stated assumptions; universality is not claimed.

Yuan-chin Ivan Chang

Institute of Statistical Science, Academia Sinica · Academia Road, Section 2, Nankang, Taipei 115, Taiwan

Recurrent Neural Networks

Ask PDF