Predictive Coding Graphs are a Superset of Feedforward Neural Networks

Björn van Zwol

cs.LGMar 6, 2026

Predictive Coding Graphs are a Superset of Feedforward Neural Networks

Authors: Björn van Zwol

Organizations: Department of Information & Computing Sciences, Utrecht University, Utrecht, The Netherlands

Abstract

Predictive coding graphs (PCGs) are a recently introduced generalization to predictive coding networks, a neuroscience-inspired probabilistic latent variable model. Here, we prove how PCGs define a mathematical superset of feedforward artificial neural networks (multilayer perceptrons). This positions PCNs more strongly within contemporary machine learning (ML), and reinforces earlier proposals to study the use of non-hierarchical neural networks for ML tasks, and more generally the notion of topology in neural networks.

Figures & tables

Figure 1: PCGs trained with IL generalize the structure of FNNs to arbitrary graphs, including loops and non-hierarchical structures, which are not trainable using BP.

Figure 2: A PCG weight matrix partitioned into block matrices ( $N=10$ nodes, partitioned by 4 layers with 2, 3, 3, and 2 nodes/layer, respectively). In traditional ANNs trained with BP, only feedforward connections (blue and red blocks) are trainable, whereas the full matrix is trainable with IL. This figure extends fig. 4 in Salvatori et al. (2022a) .

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Mapping between PCN indices (blue nodes) and PCG indices (below nodes). Note that layers may have different widths $n_{\ell}$ . $i$ denotes PCN index within a layer, and $I_{\ell}$ is defined by ( 22 ). One weight is shown in orange using PCN indices (top) and PCG indices (bottom), cf. ( 26 ).

Predictive Coding

Predictive coding explains perception and learning through predictions, prediction errors, and updates to internal models.

See all 59 papers

Topological Simplification in Predictive Coding Networks
Aug 3, 2026Adam Shaw, Jiayu Li, Michael Sperling +2Predictive Coding Topological Representation Learning
University of Southern California 3551 Trousdale Parkway, Los Angeles, CA 90089
Closed-form predictive coding via hierarchical Gaussian filters
May 19, 2026Aleksandrs Baskakovs, Sylvain Estebe, Kenneth Enevoldsen +3Predictive Coding Variational Inference
Center for Humanities Computing Aarhus University Aarhus, Denmark · Interacting Minds Center Aarhus University Aarhus, Denmark
Augmented Lagrangian Predictive Coding
May 29, 2026Jeffrey Seely, Julian GouldBackpropagation Predictive Coding
Sakana AI

Aug 3, 2026J/K move · Enter open · S save

Topological Simplification in Predictive Coding Networks

Adam Shaw, Jiayu Li, Michael Sperling, Michael Kim+1

University of Southern California 3551 Trousdale Parkway, Los Angeles, CA 90089

Predictive Coding Topological Representation Learning

Open paper AskPDF arXiv

We study the topology of learned representations in predictive coding networks (PCNs), a neuro-inspired bidirectional architecture, using a quantitative layer-wise persistent homology analysis. We train well-performing PCNs on a synthetic classification dataset (

\geq 99.9\%

test accuracy) and on MNIST (

\geq 95\%

test accuracy), and measure how topological features change across layers for different architectures and activation functions. We find that smaller PCNs collapse connected components across layers earlier than larger models (Spearman

\unicode{x1D70C} \in [0.72, 0.79]

across activations), with model size measured as the sum of hidden-layer widths. We also observe a strong negative correlation (

\unicode{x1D70C} = -0.58

) between the depth at which simplification occurs and reconstruction error; i.e., architectures that simplify later reconstruct better. Finally, a seed-level bootstrap comparison across architectures and activations shows that PCNs consistently collapse connected components later than matched MLPs, with an average difference of

3.6

layers. These results suggest that persistent homology offers a useful quantitative lens on the compression--reconstruction tradeoff in PCNs, and that both model capacity and the recurrent, bidirectional dynamics of predictive coding inference shape when this tradeoff is resolved across layers.

Abstract

Predictive Coding

Explore similar work

Topological Simplification in Predictive Coding Networks

Closed-form predictive coding via hierarchical Gaussian filters

Augmented Lagrangian Predictive Coding