Single State Update Predictive Coding training for Time Series Forecasting and Anomaly Detection

Matteo Cardoni; Sam Leroux

cs.LGAug 25, 2026

Single State Update Predictive Coding training for Time Series Forecasting and Anomaly Detection

Authors: Matteo Cardoni, Sam Leroux

Organizations: IDLab, Department of Information and Technology, Ghent University—imec, 9052 Ghent, Belgium

Abstract

Predictive Coding (PC) is a neural learning paradigm that enables parallelizable neural network layer updates. However, the main bottleneck of PC Networks (PCN) is the sequential backwards error propagation. To tackle this, we introduce a training technique that pairs a Generative PCN with a support Encoding PCN. The two PCNs are trained in parallel to match their neural activations, without sequential propagation. We apply this to time series anomaly detection and show that our approach results in more stable, continuous, online learning.

Figures & tables

Figure 1: Guided PC training step 1. Parallel feedforward initialization of the G-PCN and the E-PCN. The initializations follow a sequential order: from

h_{0}^{t-1}

\hat{x}^{t}

for the G-PCN and form

x^{t}

\gamma_{0}^{t}

for the E-PCN. The G-PCN maps

h_{0}^{t-1}

\mu_{0}^{t}

via the layer

\theta_{0}

, that acts as temporal mapping layer.

Figure 2: Guided PC training step 2. G-PCN and E-PCN parallel state update, to minimize the Guiding Energy

\mathcal{G}

and the Inernal Energy

\mathcal{I}

at the last layer. All the the states

h_{0}^{t},\dots,h_{L-1}^{t}

and

k_{L-1}^{t},\dots,k_{0}^{t}

receive a non-zero update.

Figure 3: Guided PC training step 3. G-PCN state update to minimize the Internal Energy

\mathcal{I}

. The states

h_{0}^{t},\dots,h_{L-1}^{t}

already received a non-zero update from step 2 (Figure 2 ). Therefore, the update with respect to

\mathcal{I}

will be non-zero for all

h_{0}^{t},\dots,h_{L-1}^{t}

Figure 4: Guided PC training step 4. G-PCN and E-PCN parallel weight update. The G-PCN weights are updated to minimize

\mathcal{I}

, while the E-PCN weights are updated to minimize

\mathcal{G}

. In this way, the E-PCN is trained to match the G-PCN representation, acting as a support network.

Panel of Figure 5 — Figure 5: Average accuracy expressed as Mean Squared Error between the ground truth images $x^{t}$ and the predicted images $\hat{x}^{t}$ , in sequences with three different anomaly types. 100 experiments have been averaged per each category. The semi-transparent area indicates the standard deviation per time frame. The Guided PC obtains better stability with respect to anomalies, despite performing only one unified state update step

	G-PCN	E-PCN
level $-1$	(512, 512) $h_{0}^{t-1}\rightarrow h_{0}^{t}$ tanh	-
levels $0$ – $2$	(512, 512) $h_{i}^{t}\rightarrow\mu_{i+1}^{t}$ tanh	(512, 512) $k_{i+1}^{t}\rightarrow\gamma_{i}^{t}$ tanh
level $3$	(512, 4096) $h_{3}^{t}\rightarrow\hat{x}^{t}$ linear	(4096, 512) $x^{t}\rightarrow\gamma_{2}^{t}$ tanh

Table 1: Architectural details for the G-PCN and the E-PCN. The G-PCN forward direction goes in increasing order, while the E-PCN one goes in decreasing direction (see Figure 1 ). Every cell of the table lists the Dense layers dimensions, the layer connections and the non-linearity applied to the activation.

	SGD learning rates	Adam learning rates	Adam $\mathbf{\beta_{1},\beta_{2}}$
Guided PC	[1e-2, 5e-2, 1e-1]	[1e-4, 2.5e-4, 5e-4]	[(0, 0), (0.1, 0.1), (0.5, 0.5), (0.9, 0.9)]
Vanilla PC	[1e-4, 2.5e-4, 5e-4, 1e-3, 5e-3, 1e-2, 5e-2, 1e-1]	[1e-3, 5e-3, 1e-2, 5e-2, 1e-1]

Table 2: Hyperparameters used for design space grid search. The

\beta_{1}

and

\beta_{2}

coefficient for the Adam optimizers have been explored in couples of the same value (using the same for both the Guided PC optimizers).

	Internal Energy learning rates		Guiding Energy learning rates
	States	Weights	States	Weights
G-PCN	5e-2	2.5e-4	1e-1	-
E-PCN	-	-	1e-1	1e-4
Vanilla-PCN	5e-2	2.5e-4	-	-

Table 3: Learning rates used for Guided PC and Vanilla PC.

Guided PC	Vanilla PC
$7.3\pm 73.4$ ms	$7.2\pm 85.2$ ms

Table 4: Training time per frame averaged for 150.000 frames, from 100 experiments of 1500 frames each (

\pm

standard deviation). The required training times are similar for the two techniques because the Guided PC implementation cannot still leverage the parallelization potential.

Explore similar work

Sep 26, 2026cs.LG

Analog-Friendly Predictive Coding without Activation Derivatives

Predictive coding (PC) is a local, energy-based alternative to backpropagation (BP) whose iterative inference dynamics make it attractive for implementation on analog hardware. However, standard nonlinear PC requires evaluating the derivative of the activation function during both inference and learning, which can be difficult to realise physically. Here, we introduce \textit{activation-matched Bregman PC}, replacing standard squared-error energies with Bregman divergences matched to the activation function. This formulation eliminates activation derivatives and, when combined with inference via mirror descent, yields local inference and learning rules requiring only weighted sums, local prediction errors, state integration, and the activation function. In digital experiments, Bregman PC performs comparably to standard PC and BP on classification and generative tasks, while preserving characteristic learning dynamics of PC and its convergence to BP under stable large-model parameterisations. These results provide a more analog-friendly formulation of nonlinear PC while retaining its key computational properties.

Francesco Innocenti

Brain Network Dynamics Unit University of Oxford, UK

Predictive Coding Local Learning

Ask PDF

May 19, 2026cs.LG

Closed-form predictive coding via hierarchical Gaussian filters

Predictive coding (PC) offers a local and biologically grounded alternative to backpropagation in the training of artificial neural networks, yet to date, it remains slower, and performance degrades sharply as network depth increases. We trace both problems to a single simplification: current PC networks fix the precision matrix to the identity, discarding precision-weighted prediction errors that the variational derivation requires to be fast, local, and Bayesian. We close this gap by expressing predictive coding networks as deep hierarchical Gaussian filters (HGFs) and restore precision-weighted message passing, yielding dynamic uncertainty estimates and Hebbian-compatible update rules at every layer. The resulting networks can simultaneously learn activations, weights, and precisions under a single free-energy objective, with no global error signal, and resolve inference without requiring iterations or automatic differentiation. On FashionMNIST, our solution approaches backpropagation in epoch-level wall-clock cost while converging in fewer epochs, and outperforms it on online, data efficiency, and concept-drift tasks. We thus establish that closed-form variational inference with online precision learning provides a tractable foundation for deep predictive coding networks, retaining biological and interpretative advantages, without requiring iterative relaxation or global error signals.

Aleksandrs Baskakovs, Sylvain Estebe, Kenneth Enevoldsen +3

Center for Humanities Computing Aarhus University Aarhus, Denmark · Interacting Minds Center Aarhus University Aarhus, Denmark

Predictive Coding Variational Inference Local Learning

Ask PDF

Feb 20, 2026cs.LG

Learning Long-Range Dependencies with Temporal Predictive Coding

Temporal Predictive Coding provides a layer-local, parallelisable mechanism for learning in recurrent systems, making it an attractive candidate for online local learning on neuromorphic and edge hardware. However, its recurrent parameter update captures only local temporal relationships, neglecting the historic influence of parameters along the latent-state trajectory, and therefore struggles to assign credit over longer temporal horizons. This work combines for the first time Temporal Predictive Coding with Real-Time Recurrent Learning (tPC-RTRL), incorporating an online influence matrix that tracks this historic effect whilst preserving the spatial and temporal locality properties valued by neuromorphic implementations. Under explicit assumptions, we prove that tPC-RTRL recovers the gradients of backpropagation-through-time exactly. Empirically, a near-equivalence holds across several tasks of varying scale and complexity, including byte-level language modelling on WikiText-103 (tPC-RTRL vs. BPTT: 1.865 vs. 1.864 validation BPC), English--French translation on a CCMatrix subset (20.23 vs. 20.29 BLEU), and a realistic nanodrone system-identification benchmark (0.506m vs. 0.505m mean position error). Finally, we show that the iterative inference mechanism used during training can be reused at deployment time to incorporate intermittent state observations, halving final-position error relative to open-loop rollout on the nanodrone task (0.402m vs. 0.805m) and suggesting a path towards unifying learning and filtering within the same computational framework.

Tom Potter, Oliver Rhodes

International Centre for Neuromorphic Systems, The University of Manchester, Manchester, United Kingdom

Predictive Coding Recurrent Neural Networks Local Learning

Ask PDF