A Variational Latent Equilibrium for Learning in Neuronal Circuits

Simon Brandt; Paul Haider; Walter Senn; Federico Benitez; Mihai A. Petrovici

q-bio.NCMar 10, 2026

A Variational Latent Equilibrium for Learning in Neuronal Circuits

Authors: Simon Brandt, Paul Haider, Walter Senn, Federico Benitez, Mihai A. Petrovici

Organizations: Department of Physiology, University of Bern, Bühlplatz 5, 3012, Bern, Switzerland

Abstract

Brains remain unrivaled in their ability to recognize and generate complex spatiotemporal patterns. While AI is able to reproduce some of these capabilities, deep learning algorithms remain largely at odds with our current understanding of brain circuitry and dynamics. This is prominently the case for backpropagation through time (BPTT), the go-to algorithm for learning complex temporal dependencies. In this work we propose a general formalism to approximate BPTT in a controlled, biologically plausible manner. Our approach builds on, unifies and extends several previous approaches to local, time-continuous, phase-free spatiotemporal credit assignment based on principles of energy conservation and extremal action. Our starting point is a prospective energy function of neuronal states, from which we calculate real-time error dynamics for time-continuous neuronal networks. In the general case, this provides a simple and straightforward derivation of the adjoint method result for neuronal networks, the time-continuous equivalent to BPTT. With a few modifications, we can turn this into a fully local (in space and time) set of equations for neuron and synapse dynamics. Our theory provides a rigorous framework for spatiotemporal deep learning in the brain, while simultaneously suggesting a blueprint for physical circuits capable of carrying out these computations. These results reframe and extend the recently proposed Generalized Latent Equilibrium (GLE) model.

Figures & tables

	exponential filtering	temporal differentiation
forward- facing	discounted future $\widetilde{u}^{\text{m}}=\frac{1}{\tau^{\mathrm{m}}}\int_{t}^{\infty}u(t^{\prime})\exp\left(\frac{t-t^{\prime}}{\tau^{\mathrm{m}}}\right)\text{d}t^{\prime}$	look-ahead $\breve{u}^{\text{r}}=\left(1+\tau^{\mathrm{r}}\frac{\text{d}}{\text{d}t}\right)u(t)$
backward- facing	low-pass $\overline{u}^{\text{m}}=\frac{1}{\tau^{\mathrm{m}}}\int_{-\infty}^{t}u(t^{\prime})\exp\left(\frac{t^{\prime}-t}{\tau^{\mathrm{m}}}\right)\text{d}t^{\prime}$	look-back $\widehat{u}^{\text{r}}=\left(1-\tau^{\mathrm{r}}\frac{\text{d}}{\text{d}t}\right)u(t)$

Table 1: The temporal operators can be separated into taking the future (forward facing) or past (backward facing) into account

Figure 1: Learning a simple chain in a student-teacher setup. (a) Signal (bottom-up) and error (top-down) neurons, their incoming and outgoing firing rates and their dynamics. (b) A chain of two neurons as the simplest network structure for studying error backpropagation. (c) The network converges to a minimal loss if the backwards weights $\bm{B}$ are learned (blue) or fixed to $\bm{B}=\bm{W}^{T}$ (orange). For constant $\bm{B}$ (purple), the network does not converge. We plot a moving average of the loss to remove high-frequency oscillations. (d-f) The student (opaque) and teacher (transparent) weights. Student $\bm{W}$ converge to the teacher weights if $\bm{B}$ is learned (d) or $\bm{B}=\bm{W}^{T}$ (e) and diverge for constant $\bm{B}$ (f). The learned backward weights $\bm{B}$ (d, pink) converge to the expected value given the time constant of the neuron and the frequency of the signal. (g) compares the output of the three scenarios before learning, during learning and after learning to the target signal.

	$\hat{x}^{\mathrm{m}}(t)$	$\breve{x}^{\mathrm{m}}(t)$	$\bar{x}^{\mathrm{m}}(t)$	$\tilde{x}^{\mathrm{m}}(t)$
$\mathcal{H}(\omega)$	$1-i\omega\tau_{\mathrm{m}}$	$1+i\omega\tau_{\mathrm{m}}$	$\dfrac{1}{1+i\omega\tau_{\mathrm{m}}}$	$\dfrac{1}{1-i\omega\tau_{\mathrm{m}}}$
$\mathcal{G}(\omega)$	$\sqrt{1+(\omega\tau_{\mathrm{m}})^{2}}$	$\sqrt{1+(\omega\tau_{\mathrm{m}})^{2}}$	$\dfrac{1}{\sqrt{1+(\omega\tau_{\mathrm{m}})^{2}}}$	$\dfrac{1}{\sqrt{1+(\omega\tau_{\mathrm{m}})^{2}}}$

Table 2: The temporal operators can directly be translated into Fourier space each single frequency component

\omega

. The Fourier tranform

\mathcal{H}(\omega)

yields the gain

\mathcal{G}(\omega)

for each operator.

Figure 2: Learning in a network of neurons. (a) Forward network structure (error neurons omitted for readability). (b) Network loss over time, evaluated on multiple test signals with randomly sampled frequencies and over multiple seeds. Orange: backward weights follow exact transposes of the forward weights. Blue: backward weights learn using our local update rule ( section 2 ). (c) Training input, consisting of multiple overlayed sines of different frequency, presented to the network at different times. (d) Output of the student network learning $B$ (solid) compared to the teacher network (dashed) before training, during training and after training.

Figure 3: Temporal XOR task. (a) The network consists of two laglines that perform the temporal computation and delay the input signal and one XOR layer that does the spatial computation. The target is generated by a teacher network with weights shown along the edges. (b) The training loss when learning

\bm{B}

converges faster compared to using

\bm{B}=\bm{W}^{T}

, plotted as a moving average to remove high frequency oscillations. (c) The two input signals, each with a different frequency, for different times during the simulation. (d) The output of the model with learned

\bm{B}

(blue) compared to the target before, during and after training.

Explore similar work

The Forward-Backward Disconnect: State Dynamics, Credit Assignment, and Biological Grounding in Neural Computation
Aug 20, 2026Hadi Al Mubasher, Mariette AwadCredit Assignment Neural Network Training Dynamics
Department of Electrical and Computer Engineering, Maroun Semaan Faculty of Engineering and Architecture, American University of Beirut, Lebanon
A Gradient-based yet Spike-Timing-Dependent Solution to the Feedback Learning Problem in Neural Microcircuits
Sep 8, 2026Xiangnan Zhang, Jingxin Liu, Ranqi Lu +6Neural Processes Temporal Credit Assignment
School of Future Technologies, Beijing Institute of Technology, Beijing, 100081, China. · Key Laboratory of Brain Health Intelligent Evaluation and Intervention (Beijing Institute of Technology), Ministry of Education, Beijing, 100081, China. · School of Medical Technology, Beijing Institute of Technology, Beijing, 100081, China. +5
Dynamics and Representation Structure of Local Approximations to Gradient-Based Learning in Linear Recurrent Neural Networks
May 29, 2026Ezekiel Williams, Alexandre Payeur, Guillaume LajoieRecurrent Neural Networks Linear RNNs
Department of Mathematics and Statistics, Universit´e de Montr´eal, Canada · Mila Qu´ebec, Canada · Imperial College London, United Kingdom

Aug 20, 2026J/K move · Enter open · S save

The Forward-Backward Disconnect: State Dynamics, Credit Assignment, and Biological Grounding in Neural Computation

Hadi Al Mubasher, Mariette Awad

Department of Electrical and Computer Engineering, Maroun Semaan Faculty of Engineering and Architecture, American University of Beirut, Lebanon

Credit Assignment Neural Network Training Dynamics Backpropagation Biologically Plausible Learning

Open paper AskPDF arXiv

A recurring pattern in neural computation is the reintroduction of dynamical and biological structure into models originally simplified for scalable optimization. Early feedforward networks reduced biological neurons to threshold or rate-like summation units, an abstraction compatible with global-gradient training at scale. Since then, forward computation has diversified: modern architectures carry recurrent state, retrieve from long contexts through attention and associative memory, compress histories through structured state-space dynamics, evolve in continuous time, settle to implicit equilibria, and communicate through sparse spikes. Training has diversified less. Scalable learning remains concentrated around backpropagation, backpropagation through time, adjoint methods, implicit differentiation, and surrogate-gradient variants. We call this asymmetry the forward-backward disconnect and develop a taxonomy spanning neural model families along three coupled axes: state-dynamics structure, credit-assignment mechanism, and biological grounding. Forward and learning grounding are treated separately, and the unit of analysis is the architecture-learning configuration rather than the architecture name alone. Across static, recurrent, attention-based, state-space, continuous-time, implicit, spiking, biologically plausible, and neuromorphic families, forward dynamics have diversified while the highest demonstrated scales remain concentrated in global or closely gradient-derived error-propagation mechanisms. Closing this disconnect requires better alignment among state dynamics, credit assignment, and computational substrate.

Abstract

Explore similar work

The Forward-Backward Disconnect: State Dynamics, Credit Assignment, and Biological Grounding in Neural Computation

A Gradient-based yet Spike-Timing-Dependent Solution to the Feedback Learning Problem in Neural Microcircuits

Dynamics and Representation Structure of Local Approximations to Gradient-Based Learning in Linear Recurrent Neural Networks