Persistence, Exposure, and Credit Routing in Linear Temporal Predictive Coding

Akira Tanase; Julian Gould; Jeffrey Seely

cs.LGOct 26, 2026

Persistence, Exposure, and Credit Routing in Linear Temporal Predictive Coding

Authors: Akira Tanase, Julian Gould, Jeffrey Seely

Organizations: Sakana AI · Imperial College London

Abstract

Temporal Predictive Coding (tPC) networks offer a layer-local alternative to backpropagation. For linear tPC, we show, by applying tools from cellular sheaf theory, that temporal credit assignment is a tug-of-war between spatial displacement and temporal errors, whose ratios are modulated by persistence and exposure operators $P$ and $Q$ . Spatial inference therefore gives us the effective recurrent transition $F=PA$ , whilst future credit is filtered by $P^\ast$ . This creates a persistence--error tradeoff in spatial modes: under non-amplifying invariant recurrence, modes that preserve influence over long delays simultaneously fail to expose that influence strongly as temporal error. Recurrence can circumvent this via routing between persistent and exposed modes.

Figures & tables

Panel of Figure 1 — Figure 1: Linear temporal predictive coding architecture. Spatial maps $W_{\ell}$ predict down each time slice, whilst temporal maps $A_{\ell}$ predict forward across time. At time $t$ , the observation $z_{0,t}=o_{t}$ is fixed and inference updates the hidden activities in the shaded slice.

Figure 2: Full temporal credit learns to retain the cue in

C

and expose it in

E

at the response, whereas standard tPC does not.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: The time-slice sheaf $\mathcal{F}^{(t)}$ associated to a linear tPC network. Temporal and spatial edge-stalks are shown in red and blue respectively. The left-most column represents the layers of the network at time $t-1$ , while the black vertex stalks in the right-most column represent the layers of the network at time $t$ .

Figure 4: As in delayed recall, full temporal credit retains the digit in the carrier and exposes it at the response, whereas standard local tPC does not.

Predictive Coding

Predictive coding explains perception and learning through predictions, prediction errors, and updates to internal models.

See all 61 papers

Learning Long-Range Dependencies with Temporal Predictive Coding
Feb 20, 2026Tom Potter, Oliver RhodesPredictive Coding Recurrent Neural Networks
International Centre for Neuromorphic Systems, The University of Manchester, Manchester, United Kingdom
Topological Simplification in Predictive Coding Networks
Aug 3, 2026Adam Shaw, Jiayu Li, Michael Sperling +2Predictive Coding Topological Representation Learning
University of Southern California 3551 Trousdale Parkway, Los Angeles, CA 90089
Predictive Coding with Bayesian Priors via Proximal Gradients
Jun 6, 2026Francesco BulloPredictive Coding Markov Random Fields
Mechanical Engineering, 2325 Engineering Bldg II, UCSB, Santa Barbara, CA 93106-5070, USA

Feb 20, 2026J/K move · Enter open · S save

Learning Long-Range Dependencies with Temporal Predictive Coding

Tom Potter, Oliver Rhodes

International Centre for Neuromorphic Systems, The University of Manchester, Manchester, United Kingdom

Predictive Coding Recurrent Neural Networks Local Learning

Open paper AskPDF arXiv

Temporal Predictive Coding provides a layer-local, parallelisable mechanism for learning in recurrent systems, making it an attractive candidate for online local learning on neuromorphic and edge hardware. However, its recurrent parameter update captures only local temporal relationships, neglecting the historic influence of parameters along the latent-state trajectory, and therefore struggles to assign credit over longer temporal horizons. This work combines for the first time Temporal Predictive Coding with Real-Time Recurrent Learning (tPC-RTRL), incorporating an online influence matrix that tracks this historic effect whilst preserving the spatial and temporal locality properties valued by neuromorphic implementations. Under explicit assumptions, we prove that tPC-RTRL recovers the gradients of backpropagation-through-time exactly. Empirically, a near-equivalence holds across several tasks of varying scale and complexity, including byte-level language modelling on WikiText-103 (tPC-RTRL vs. BPTT: 1.865 vs. 1.864 validation BPC), English--French translation on a CCMatrix subset (20.23 vs. 20.29 BLEU), and a realistic nanodrone system-identification benchmark (0.506m vs. 0.505m mean position error). Finally, we show that the iterative inference mechanism used during training can be reused at deployment time to incorporate intermittent state observations, halving final-position error relative to open-loop rollout on the nanodrone task (0.402m vs. 0.805m) and suggesting a path towards unifying learning and filtering within the same computational framework.

Abstract

Predictive Coding

Explore similar work

Learning Long-Range Dependencies with Temporal Predictive Coding

Topological Simplification in Predictive Coding Networks

Predictive Coding with Bayesian Priors via Proximal Gradients