The Forward-Backward Disconnect: State Dynamics, Credit Assignment, and Biological Grounding in Neural Computation

Hadi Al Mubasher; Mariette Awad

cs.NEAug 20, 2026

The Forward-Backward Disconnect: State Dynamics, Credit Assignment, and Biological Grounding in Neural Computation

Authors: Hadi Al Mubasher, Mariette Awad

Organizations: Department of Electrical and Computer Engineering, Maroun Semaan Faculty of Engineering and Architecture, American University of Beirut, Lebanon

Abstract

A recurring pattern in neural computation is the reintroduction of dynamical and biological structure into models originally simplified for scalable optimization. Early feedforward networks reduced biological neurons to threshold or rate-like summation units, an abstraction compatible with global-gradient training at scale. Since then, forward computation has diversified: modern architectures carry recurrent state, retrieve from long contexts through attention and associative memory, compress histories through structured state-space dynamics, evolve in continuous time, settle to implicit equilibria, and communicate through sparse spikes. Training has diversified less. Scalable learning remains concentrated around backpropagation, backpropagation through time, adjoint methods, implicit differentiation, and surrogate-gradient variants. We call this asymmetry the forward-backward disconnect and develop a taxonomy spanning neural model families along three coupled axes: state-dynamics structure, credit-assignment mechanism, and biological grounding. Forward and learning grounding are treated separately, and the unit of analysis is the architecture-learning configuration rather than the architecture name alone. Across static, recurrent, attention-based, state-space, continuous-time, implicit, spiking, biologically plausible, and neuromorphic families, forward dynamics have diversified while the highest demonstrated scales remain concentrated in global or closely gradient-derived error-propagation mechanisms. Closing this disconnect requires better alignment among state dynamics, credit assignment, and computational substrate.

Figures & tables

Panel of Figure 1 — Figure 1. Timeline of selected architectural, learning-rule, and hardware-substrate milestones in neural computation, 1943–2026. Marker colors encode biological grounding under the split-axis scheme of Section 3.3 ; hardware uses a distinct gold marker because substrates do not map onto grounding tiers. The post-2010 segment is horizontally zoomed to expose the greater density of recent milestones. The figure provides illustrative historical orientation rather than quantitative evidence; Figure 2 and Section 11.1 establish the forward–backward asymmetry categorically. A three-track timeline from 1943 to 2026. The upper track shows selected architectural milestones in neural computation; the middle track shows selected learning-rule and credit-assignment milestones; the lower track shows selected hardware-substrate milestones, including neuromorphic computing introduced by Mead in 1990, the digital neuromorphic chip TrueNorth and the parallel ARM-core machine SpiNNaker in 2014, the systolic-array Tensor Processing Unit in 2017, the Loihi chip in 2018 with on-chip plasticity, Loihi 2 in 2021, and the mixed-signal BrainScaleS-2 in 2022. A visual axis break at 2010 separates a pre-2010 segment from a post-2010 segment drawn at a horizontally zoomed scale. Architecture-track and learning-track marker color encodes biological grounding under a shared color scheme: white for none or historical inspiration; teal for functional analogy on the forward axis and weak on the learning axis; olive for architectural constraint; rose for local or algorithmic; and coral for mechanistic plausibility on the forward axis and mechanistic plasticity on the learning axis. Hardware-track markers are filled gold because compute substrates do not map onto biological-grounding tiers.

Mechanism	Gradient relation	Update scope	Temporal requirement	Objective source	Update timing
Backpropagation ( Rumelhart et al., 1986 )	Exact	Global / non-local	None (static graph)	Global supervised	After forward pass
BPTT ( Werbos, 1990 )	Exact	Global / non-local	Unrolled sequence	Global supervised	After full sequence
FA / DFA / PFA ( Lillicrap et al., 2016 ; Nøkland, 2016 ; Li and Benna, 2024 )	Approximate	Broadcast error via random or learned matrices	None in feedforward case	Global supervised	Feedback pass; DFA partly parallel
Synthetic gradients ( Jaderberg et al., 2017 )	Approximate (predicted)	Module-local predictor; target derived from downstream gradient	Optional recurrent state	Global supervised	Asynchronous
Adjoint / implicit diff. ( Chen et al., 2018 ; Bai et al., 2019 )	Exact under solver / equilibrium conditions	Solver / global linear solve at equilibrium	Trajectory or fixed point	Global objective	Backward solve after forward solve
Surrogate-gradient SNN ( Neftci et al., 2019 )	Approximate derivative	Global error propagation through smoothed spikes	Spike-time trajectory	Global supervised	Backward through temporal graph

Table 1. Secondary credit-assignment dimensions applied to representative learning mechanisms. These attributes are descriptive coordinates rather than additional mutually exclusive classes; individual variants may differ. The table instantiates the multidimensional structure of the backward axis introduced above, so a new architecture–learning configuration can be located along the same coordinates.

Panel of Figure 2 — Figure 2. Joint taxonomy of neural computation as an alluvial flow. Each architecture–learning configuration is one unit of flow from a forward state-dynamics class (left column) to a credit-assignment mechanism (right column). Node heights and ribbon widths are proportional to configuration counts; ribbon colors identify the source class. Table 3 lists model families and grounding assignments; Supplementary Section S3 provides the complete configuration-level ledger. Global gradient is the largest single credit-assignment category (16/32), followed by Approximate or implicit gradient (7/32), and it receives configurations from four of the five forward classes: the categorical form of the forward–backward disconnect introduced in Section 1 . An alluvial flow diagram showing the joint taxonomy as a bipartite flow. Five colored vertical bars on the left represent forward state-dynamics classes, from top to bottom: Static (6 configurations, blue), Discrete-time or sequence (14, orange), Continuous-time (6, green), Implicit fixed-point (4, purple), Hybrid event-driven (2, red). Four gray vertical bars on the right represent credit-assignment mechanisms, from top to bottom: Global gradient (16 configurations), Approximate or implicit gradient (7), Local plasticity (6), Energy or unsupervised (3). Fourteen curved ribbons connect the left classes to the right mechanisms; each ribbon's width is proportional to how many configurations flow from that source to that destination, and each ribbon is colored by its source class. Global gradient is the largest single credit-assignment category, accounting for half of the audited configurations and receiving ribbons from four of the five forward classes, followed by Approximate or implicit gradient as the second-largest. The visual impression is a fan of colored streams on the left converging predominantly into the two upper destinations on the right, which is the visual signature of the credit-assignment concentration relative to forward-dynamics diversification.

Forward class	Global gradient	Approximate or implicit gradient	Local plasticity	Energy / unsupervised	Total
Static	4: MLP, CNN, ResNet, PINN (C01–C04)	0	1: Neocognitron (C05)	1: MLP + Forward-Forward (C06)	6
Discrete	8: RNN, LSTM, GRU, Transformer, S4, Mamba, RWKV, xLSTM (C07–C14)	2: RNN + synth. grad., RNN + RFLO (C15, C16)	2: classical Hopfield, HTM (C17, C32)	2: RBM, DBN (C18, C19)	14
Continuous	3: CTRNN, Liquid TC, Liquid closed-form (C20–C22)	1: Neural ODE (C23)	2: PC recurrent, CT-RNN + reward-mod. Hebb. (C24, C25)	0	6
Implicit	1: modern Hopfield (C26)	3: DEQ, PC local-error, PC BP-equiv. limit (C27–C29)	0	0	4
Hybrid	0	1: surrogate-gradient LIF SNN (C30)	1: STDP-trained LIF SNN (C31)	0	2
Total	16	7	6	3	32

Table 2. Configuration counts underlying Figure 2 , with the model families that populate each cell. Cell entries follow the pattern count: family list (C-IDs referencing the per-configuration ledger of Supplementary Section S3) . Row and column totals match the node heights of Figure 2 exactly.

Survey	Venue, Year	Static	Discrete / seq.	Attn / SSM	Cont. / Implicit	Spiking / hybrid	Bio learn.	Neuro- morph.
Hassabis et al. ( Hassabis et al., 2017 )	Neuron , 2017	$\circ$	$\circ$	—	—	$\circ$	$\circ$	—
Tavanaei et al. ( Tavanaei et al., 2019 )	Neural Netw. , 2019	—	—	—	—	$\bullet$	$\circ$	$\circ$
Roy et al. ( Roy et al., 2019 )	Nature , 2019	—	—	—	—	$\bullet$	$\circ$	$\bullet$
Lillicrap et al. ( Lillicrap et al., 2020 )	Nat. Rev. Neurosci. , 2020	—	—	—	—	—	$\bullet$	—
Karniadakis et al. ( Karniadakis et al., 2021 )	Nat. Rev. Phys. , 2021	$\bullet$	—	—	$\circ$	—	—	—
Schuman et al. ( Schuman et al., 2022 )	Nat. Comput. Sci. , 2022	—	—	—	—	$\bullet$	$\circ$	$\bullet$

Table 4. Coverage of recent surveys against this paper’s model-family categories.

\bullet

: substantive;

\circ

: partial; —: out of scope. The distinguishing contribution of the current survey is not broader coverage alone but the specific joint decomposition it applies: state dynamics

\times

credit assignment

\times

biological grounding

\times

substrate. This joint decomposition is what makes the forward–backward disconnect visible as a categorical pattern rather than as a scattered observation across independently organized reviews.

Panel of Figure 3 — Figure 3. Three canonical dynamical objects recovered in trained RNN state spaces by the fixed-point reverse-engineering technique of ( Sussillo and Barak, 2013 ) . Solid black dots mark fixed points; arrows indicate the local flow of the dynamics in a 2D projection $(h_{1},h_{2})$ of the hidden state. A point attractor (a) implements memory storage. A line attractor (b) implements graded integration along a one-parameter family of stable states and is the canonical mechanism for context-dependent decision making in trained RNNs and in prefrontal cortex ( Mante et al., 2013 ) . A saddle point (c) separates basins of attraction (dashed separatrices) and implements a decision boundary between two stored states. Across populations of trained networks the precise weights differ but the topology of these objects is conserved ( Maheswaranathan et al., 2019 ) . Three side-by-side phase-portrait panels in a 2D projection of recurrent-network hidden state. Panel (a) shows a point attractor: a single filled dot in the center with arrows pointing inward from all directions. Panel (b) shows a line attractor: a horizontal line of stable states with arrows pointing toward the line from above and below. Panel (c) shows a saddle point: a central dot with arrows along one axis flowing inward and along the perpendicular axis flowing outward, with dashed separatrices dividing the plane into basins of attraction.

Panel of Figure 4 — Figure 4. Substrate reconnection map. The dense-gradient path on the left ( GPU / TPU ; backpropagation training scales) and the event-driven path on the right (neuromorphic; event-driven inference scales) currently bridge only empirically (dashed gray): training runs off-chip with surrogate gradients on GPUs and deploys on neuromorphic hardware for inference. The missing bridge (dashed red) is a substrate-aligned learning rule competitive with backpropagation at deep-network scale, which Section 9 found absent. Neuromorphic hardware removes the substrate penalty for event-driven forward computation but does not supply the missing rule. A conceptual schematic in three vertical layers. Top layer: two rounded rectangles side by side. The left box, labeled GPU/TPU substrate, lists dense tensors, stored activations, batch synchronization, and global gradient transport. The right box, labeled Neuromorphic substrate, lists sparse events, local state, online updates, and memory near compute. Middle layer: two outcome boxes shaded light blue. The left says Backpropagation training scales; the right says Event-driven inference scales. Solid black arrows connect each substrate down to its corresponding outcome, labeled global gradient on the left and sparse, local on the right. A dashed gray arrow runs horizontally between the two outcome boxes labeled off-chip surrogate-gradient training, on-chip event-driven deployment, indicating the current empirical workaround. Bottom layer: a dashed red rectangle labeled Missing bridge, with subtext scalable local credit assignment and a note that no substrate-aligned learning rule surveyed in Section 9 closes this gap. Dashed red arrows from each outcome box curve down toward this missing-bridge box, visually marking the unresolved connection.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Configuration	State-dynamics class	Learning row	Forward grounding	Learning grounding
Modern Hopfield (end-to-end)	Implicit / fixed-point	Global gradient	Functional analogy	None
Classical Hopfield (Hebbian)	Discrete (fixed-point fn.)	Local plasticity	Arch. constraint	Local or algorithmic
PC: recurrent inference	Continuous-time / fixed-pt.	Local plasticity	Arch. constraint	Local/algorithmic
PC: energy-based	Continuous-time / fixed-pt.	Energy/unsupervised	Arch. constraint	Local/algorithmic
PC: local-error	Implicit / fixed-point	Approx./implicit	Funct. analogy	Local/algorithmic
PC: BP-equivalent limit	Implicit / fixed-point	Approx./implicit	Funct. analogy	Local/algorithmic

Appendix

Table 8

ID	Architecture	Learning rule	Forward class	Credit class	Fwd. gnd.	Learn. gnd.	Evidence
C01	MLP	Backpropagation	Static	Global grad.	None/Hist.	None	[ 115 ]
C02	CNN (LeNet, AlexNet)	Backpropagation	Static	Global grad.	None/Hist.	None	[ 75 , 66 ]
C03	ResNet	Backpropagation	Static	Global grad.	None/Hist.	None	[ 48 ]
C04	PINN	Backpropagation (with PDE-residual objective)	Static	Global grad.	None/Hist.	None	[ 109 ]
C05	Neocognitron	Local competitive Hebbian	Static	Local plast.	Arch. constr.	Local/alg.	[ 31 ]
C06	MLP + Forward-Forward	Layerwise goodness	Static	Energy/unsup.	None/Hist.	Local/alg.	[ 53 ]

Appendix

Table 9

Backward category	Mechanism	Count	Ledger IDs
Global gradient (16)	Ordinary reverse-mode BP	10	C01–C04, C10–C14, C26
	BPTT / reverse-mode through recurrence	4	C07–C09, C22
	Solver BP / solver BPTT	2	C20–C21
Approximate/implicit (7)	Synthetic gradients (DNI)	1	C15
	Random-feedback local online (RFLO)	1	C16
	Adjoint sensitivity	1	C23

Appendix

Table 1: Mechanism-level decomposition of the two largest backward categories. Each mechanism is identified by the exact procedure it uses, and the count reports how many of the 32 ledger configurations it accounts for. Global gradient (16 configurations total) is dominated by ordinary reverse-mode backpropagation (10); the Approximate/implicit category (7 configurations total) fragments across seven distinct mechanisms, each contributing exactly one configuration.

Family	Largest reported result	Comparison	Reference
Feedback alignment (plain)	Performs well on MNIST/CIFAR; significantly below BP on ImageNet	Gap to BP large at ImageNet	[ 7 ]
Sign-symmetry (FA variant)	Approaches BP on ImageNet with ResNet-18/AlexNet	Within a few percent of BP	[ 132 ]
Target propagation	MNIST/CIFAR competitive; ImageNet substantially below BP	No parity at ImageNet scale	[ 7 ]
Equilibrium propagation	11.7% test error on CIFAR-10 with deep ConvNets; approaches BPTT to within roughly 0.6 percentage points; downsampled ImageNet 32 $\times$ 32 in subsequent work	No full ImageNet-1k or LM result	[ 69 , 70 ]
Weight Mirror (weight-symmetry relaxation)	Learns dynamic feedback weights to align with forward weights; on ImageNet, ResNet-18 top-1 error 30.2% for Weight Mirror vs 30.1% for BP (within 0.1 pp), ResNet-50 top-1 error 23.4% for Weight Mirror vs 22.9% for BP (within 0.5 pp); the authors describe the result as “nearly matched backprop” and “learning ImageNet about as well as backprop does”	Approximately matches BP on ResNet-18 and ResNet-50 ImageNet under the paper’s training pipeline; substantially outperforms plain feedback alignment and sign-symmetry; still relies on error-broadcast machinery	[ 2 ]
Product Feedback Alignment (PFA)	On ResNet-18 ImageNet: BP 69.69%, PFA 68.46%, PFA-o 69.30% top-1 accuracy; approaches BP without explicit weight symmetry	PFA-o within roughly 0.4 percentage points of BP at ResNet-18 scale, but no ResNet-50 or GPT-class demonstration reported	[ 77 ]

Appendix

Table 2: Empirical scaling evidence anchoring the main paper’s scaling claim: strongest reported per-family results at the search cutoff, and their comparison against mainstream backpropagation.

Explore similar work

May 20, 2025

Forward Target Propagation: A Forward-Only Approach to Global Error Credit Assignment via Local Losses

Training neural networks has traditionally relied on backpropagation (BP), a gradient-based algorithm that, despite its widespread success, suffers from key limitations in both biological and hardware perspectives. These include backward error propagation by symmetric weights, non-local credit assignment, and frozen activity during backward passes. We propose Forward Target Propagation (FTP), a biologically plausible and computationally efficient alternative that replaces the backward pass with a second forward pass. FTP estimates layerwise targets using only feedforward computations, eliminating the need for symmetric feedback weights or learnable inverse functions, hence enabling modular and local learning. We evaluate FTP on fully connected networks, CNNs, and RNNs, demonstrating accuracies competitive with BP on MNIST, CIFAR10, and CIFAR100, as well as effective modeling of long-term dependencies in sequential tasks. Moreover, FTP outperforms BP under quantized low-precision and emerging hardware constraints while also demonstrating substantial efficiency gains over other biologically inspired methods such as target propagation variants and forward-only learning algorithms. With its minimal computational overhead, forward-only nature, and hardware compatibility, FTP provides a promising direction for energy-efficient on-device learning and neuromorphic computing.

Nazmus Saadat As-Saquib, A N M Nafiz Abeer, Hung-Ta Chien +3

Texas A&M University, College Station, TX 77843 · Sandia National Laboratories, Livermore, CA 94550

Neural Network Optimization Local Learning

Ask PDF

Mar 10, 2026

A Variational Latent Equilibrium for Learning in Neuronal Circuits

Brains remain unrivaled in their ability to recognize and generate complex spatiotemporal patterns. While AI is able to reproduce some of these capabilities, deep learning algorithms remain largely at odds with our current understanding of brain circuitry and dynamics. This is prominently the case for backpropagation through time (BPTT), the go-to algorithm for learning complex temporal dependencies. In this work we propose a general formalism to approximate BPTT in a controlled, biologically plausible manner. Our approach builds on, unifies and extends several previous approaches to local, time-continuous, phase-free spatiotemporal credit assignment based on principles of energy conservation and extremal action. Our starting point is a prospective energy function of neuronal states, from which we calculate real-time error dynamics for time-continuous neuronal networks. In the general case, this provides a simple and straightforward derivation of the adjoint method result for neuronal networks, the time-continuous equivalent to BPTT. With a few modifications, we can turn this into a fully local (in space and time) set of equations for neuron and synapse dynamics. Our theory provides a rigorous framework for spatiotemporal deep learning in the brain, while simultaneously suggesting a blueprint for physical circuits capable of carrying out these computations. These results reframe and extend the recently proposed Generalized Latent Equilibrium (GLE) model.

Simon Brandt, Paul Haider, Walter Senn +2

Department of Physiology, University of Bern, Bühlplatz 5, 3012, Bern, Switzerland

Temporal Credit Assignment Neural Network Training Dynamics Backpropagation Local Learning

Ask PDF

Sep 8, 2026

A Gradient-based yet Spike-Timing-Dependent Solution to the Feedback Learning Problem in Neural Microcircuits

The brain uses discrete spikes for dynamic computation, yet, how neural microcircuits (NMCs) solve temporal credit assignment using local spike timing remains a fundamental open question. Dominant spiking neural network (SNN) approaches circumvent this by approximating backpropagation through surrogate gradients, decoupling learning from biological spike timing. Here, we reformulate temporal credit assignment as a state separation problem: extracting task-required components induced by historical perturbations directly from the current neural state. This enables an online feedback learning framework for NMCs through a gradient tunneling (GT) algorithm and the lead-lag expansion technique that derives credit assignment from local synaptic spike timing, while remaining compatible with ANN-SNN hybrid architectures. Experimentally, GT-trained NMCs excel at long-timescale evidence integration and noise-robust memory retention, and perform comparably to leading SNN online learning methods on real-world benchmarks with far fewer parameters. The proposed framework addresses the two-decade-old NMC feedback learning problem and suggests a computationally plausible explanation for the brain's learning mechanisms.

Xiangnan Zhang, Jingxin Liu, Ranqi Lu +6

School of Future Technologies, Beijing Institute of Technology, Beijing, 100081, China. · Key Laboratory of Brain Health Intelligent Evaluation and Intervention (Beijing Institute of Technology), Ministry of Education, Beijing, 100081, China. · School of Medical Technology, Beijing Institute of Technology, Beijing, 100081, China. +5

Neural Processes Temporal Credit Assignment Spiking Neural Networks Spike-Timing-Dependent Plasticity Online Learning

Ask PDF