Single State Update Predictive Coding training for Time Series Forecasting and Anomaly Detection

Matteo Cardoni; Sam Leroux

cs.LGAug 25, 2026

Single State Update Predictive Coding training for Time Series Forecasting and Anomaly Detection

Authors: Matteo Cardoni, Sam Leroux

Organizations: IDLab, Department of Information and Technology, Ghent University—imec, 9052 Ghent, Belgium

Abstract

Predictive Coding (PC) is a neural learning paradigm that enables parallelizable neural network layer updates. However, the main bottleneck of PC Networks (PCN) is the sequential backwards error propagation. To tackle this, we introduce a training technique that pairs a Generative PCN with a support Encoding PCN. The two PCNs are trained in parallel to match their neural activations, without sequential propagation. We apply this to time series anomaly detection and show that our approach results in more stable, continuous, online learning.

Figures & tables

Figure 1: Guided PC training step 1. Parallel feedforward initialization of the G-PCN and the E-PCN. The initializations follow a sequential order: from

h_{0}^{t-1}

\hat{x}^{t}

for the G-PCN and form

x^{t}

\gamma_{0}^{t}

for the E-PCN. The G-PCN maps

h_{0}^{t-1}

\mu_{0}^{t}

via the layer

\theta_{0}

, that acts as temporal mapping layer.

Figure 2: Guided PC training step 2. G-PCN and E-PCN parallel state update, to minimize the Guiding Energy

\mathcal{G}

and the Inernal Energy

\mathcal{I}

at the last layer. All the the states

h_{0}^{t},\dots,h_{L-1}^{t}

and

k_{L-1}^{t},\dots,k_{0}^{t}

receive a non-zero update.

Figure 3: Guided PC training step 3. G-PCN state update to minimize the Internal Energy

\mathcal{I}

. The states

h_{0}^{t},\dots,h_{L-1}^{t}

already received a non-zero update from step 2 (Figure 2 ). Therefore, the update with respect to

\mathcal{I}

will be non-zero for all

h_{0}^{t},\dots,h_{L-1}^{t}

Figure 4: Guided PC training step 4. G-PCN and E-PCN parallel weight update. The G-PCN weights are updated to minimize

\mathcal{I}

, while the E-PCN weights are updated to minimize

\mathcal{G}

. In this way, the E-PCN is trained to match the G-PCN representation, acting as a support network.

Panel of Figure 5 — Figure 5: Average accuracy expressed as Mean Squared Error between the ground truth images $x^{t}$ and the predicted images $\hat{x}^{t}$ , in sequences with three different anomaly types. 100 experiments have been averaged per each category. The semi-transparent area indicates the standard deviation per time frame. The Guided PC obtains better stability with respect to anomalies, despite performing only one unified state update step

	G-PCN	E-PCN
level $-1$	(512, 512) $h_{0}^{t-1}\rightarrow h_{0}^{t}$ tanh	-
levels $0$ – $2$	(512, 512) $h_{i}^{t}\rightarrow\mu_{i+1}^{t}$ tanh	(512, 512) $k_{i+1}^{t}\rightarrow\gamma_{i}^{t}$ tanh
level $3$	(512, 4096) $h_{3}^{t}\rightarrow\hat{x}^{t}$ linear	(4096, 512) $x^{t}\rightarrow\gamma_{2}^{t}$ tanh

Table 1: Architectural details for the G-PCN and the E-PCN. The G-PCN forward direction goes in increasing order, while the E-PCN one goes in decreasing direction (see Figure 1 ). Every cell of the table lists the Dense layers dimensions, the layer connections and the non-linearity applied to the activation.

	SGD learning rates	Adam learning rates	Adam $\mathbf{\beta_{1},\beta_{2}}$
Guided PC	[1e-2, 5e-2, 1e-1]	[1e-4, 2.5e-4, 5e-4]	[(0, 0), (0.1, 0.1), (0.5, 0.5), (0.9, 0.9)]
Vanilla PC	[1e-4, 2.5e-4, 5e-4, 1e-3, 5e-3, 1e-2, 5e-2, 1e-1]	[1e-3, 5e-3, 1e-2, 5e-2, 1e-1]

Table 2: Hyperparameters used for design space grid search. The

\beta_{1}

and

\beta_{2}

coefficient for the Adam optimizers have been explored in couples of the same value (using the same for both the Guided PC optimizers).

	Internal Energy learning rates		Guiding Energy learning rates
	States	Weights	States	Weights
G-PCN	5e-2	2.5e-4	1e-1	-
E-PCN	-	-	1e-1	1e-4
Vanilla-PCN	5e-2	2.5e-4	-	-

Table 3: Learning rates used for Guided PC and Vanilla PC.

Guided PC	Vanilla PC
$7.3\pm 73.4$ ms	$7.2\pm 85.2$ ms

Table 4: Training time per frame averaged for 150.000 frames, from 100 experiments of 1500 frames each (

\pm

standard deviation). The required training times are similar for the two techniques because the Guided PC implementation cannot still leverage the parallelization potential.

Explore similar work

Analog-Friendly Predictive Coding without Activation Derivatives
Sep 26, 2026Francesco InnocentiPredictive Coding Local Learning
Brain Network Dynamics Unit University of Oxford, UK
Closed-form predictive coding via hierarchical Gaussian filters
May 19, 2026Aleksandrs Baskakovs, Sylvain Estebe, Kenneth Enevoldsen +3Predictive Coding Variational Inference
Center for Humanities Computing Aarhus University Aarhus, Denmark · Interacting Minds Center Aarhus University Aarhus, Denmark
Learning Long-Range Dependencies with Temporal Predictive Coding
Feb 20, 2026Tom Potter, Oliver RhodesPredictive Coding Recurrent Neural Networks
International Centre for Neuromorphic Systems, The University of Manchester, Manchester, United Kingdom

Sep 26, 2026, cs.LGJ/K move · Enter open · S save

Analog-Friendly Predictive Coding without Activation Derivatives

Francesco Innocenti

Brain Network Dynamics Unit University of Oxford, UK

Predictive Coding Local Learning

Open paper AskPDF arXiv

Predictive coding (PC) is a local, energy-based alternative to backpropagation (BP) whose iterative inference dynamics make it attractive for implementation on analog hardware. However, standard nonlinear PC requires evaluating the derivative of the activation function during both inference and learning, which can be difficult to realise physically. Here, we introduce \textit{activation-matched Bregman PC}, replacing standard squared-error energies with Bregman divergences matched to the activation function. This formulation eliminates activation derivatives and, when combined with inference via mirror descent, yields local inference and learning rules requiring only weighted sums, local prediction errors, state integration, and the activation function. In digital experiments, Bregman PC performs comparably to standard PC and BP on classification and generative tasks, while preserving characteristic learning dynamics of PC and its convergence to BP under stable large-model parameterisations. These results provide a more analog-friendly formulation of nonlinear PC while retaining its key computational properties.

Abstract

Explore similar work

Analog-Friendly Predictive Coding without Activation Derivatives

Closed-form predictive coding via hierarchical Gaussian filters

Learning Long-Range Dependencies with Temporal Predictive Coding