Heterogeneous Time Constants Improve Stability in Equilibrium Propagation

Yoshimasa Kubo; Suhani Pragnesh Modi; Smit Patel

cs.LGMar 3, 2026

Heterogeneous Time Constants Improve Stability in Equilibrium Propagation

Authors: Yoshimasa Kubo, Suhani Pragnesh Modi, Smit Patel

Organizations: Department of Computer Science, Lakehead University

Abstract

Equilibrium propagation (EP) is a biologically plausible alternative to backpropagation for training neural networks. However, existing EP models use a uniform scalar time step dt, which corresponds biologically to a membrane time constant that is heterogeneous across neurons. Here, we introduce heterogeneous time steps (HTS) for EP by assigning neuron-specific time constants drawn from biologically motivated distributions. We show that HTS improves training stability while maintaining competitive task performance. These results suggest that incorporating heterogeneous temporal dynamics enhances both the biological realism and robustness of equilibrium propagation.

Figures & tables

Figure 1: Distributions of heterogeneous time steps ( $dt_{i}$ ) used in the hidden layer. All distributions were parameterized with $\mu=0.3$ , $\sigma=0.1$ and clamped to $[10^{-3},0.5]$ for numerical stability. The visible accumulation at $dt=0.5$ for heavy-tailed distributions results from this truncation.

Dataset	$dt_{y}$	Normal	Log-normal	Gamma	Scalar
MNIST	0.15	98.43 $\pm$ 0.03%	98.43 $\pm$ 0.04%	98.43 $\pm$ 0.04%	98.43 $\pm$ 0.04%
	0.20	98.44 $\pm$ 0.03%	98.44 $\pm$ 0.02%	98.43 $\pm$ 0.02%	98.43 $\pm$ 0.04%
	0.25	98.44 $\pm$ 0.04%	98.44 $\pm$ 0.03%	98.43 $\pm$ 0.03%	98.44 $\pm$ 0.04%
	0.30	98.46 $\pm$ 0.03%	98.46 $\pm$ 0.02%	98.45 $\pm$ 0.04%	98.43 $\pm$ 0.03%
	0.35	98.46 $\pm$ 0.04%	98.45 $\pm$ 0.03%	98.45 $\pm$ 0.04%	98.44 $\pm$ 0.04%
KMNIST	0.15	91.29 $\pm$ 0.22%	91.21 $\pm$ 0.14%	91.15 $\pm$ 0.18%	91.07 $\pm$ 0.17%

Table 1: Average test accuracy (%)

\pm

standard deviation across 10 random seeds.

Equilibrium Propagation

Equilibrium propagation trains energy-based neural networks by estimating gradients from changes between free and weakly nudged equilibrium states.

See all 17 papers

Dendritic Neural Networks with Equilibrium Propagation
May 1, 2026Yoshimasa KuboEquilibrium Propagation Neural Network Training Dynamics
Department of Computer Science Lakehead University Thunder Bay, Canada
Hybridizing Equilibrium Propagation with Ising Machines for Efficient Energy-Based Learning
Jun 8, 2026Chen-Rui Fan, Bo Lu, Xing-Yu Wu +2Equilibrium Propagation Hopfield Networks
School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China · Laboratory for Advanced Computing and Intelligence Engineering, Information Engineering University, Zhengzhou 450001, China · School of Physical Science and Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
A Variational Latent Equilibrium for Learning in Neuronal Circuits
Mar 10, 2026Simon Brandt, Paul Haider, Walter Senn +2Temporal Credit Assignment Neural Network Training Dynamics
Department of Physiology, University of Bern, Bühlplatz 5, 3012, Bern, Switzerland

May 1, 2026J/K move · Enter open · S save

Dendritic Neural Networks with Equilibrium Propagation

Yoshimasa Kubo

Department of Computer Science Lakehead University Thunder Bay, Canada

Equilibrium Propagation Neural Network Training Dynamics

Open paper AskPDF arXiv

Equilibrium propagation (EP) is a biologically plausible alternative to backpropagation (BP), but its effectiveness can degrade in deeper and more challenging learning settings. In parallel, dendritic neural networks have demonstrated improved performance and generalization when trained with BP, suggesting that structured, biologically inspired architectures may enhance learning. In this work, we investigate the integration of dendritic neural networks with equilibrium propagation using an advanced EP framework. We evaluate the proposed dendritic EP model on MNIST, Kuzushiji-MNIST (KMNIST), and Fashion-MNIST (FMNIST), considering both shallow and deeper architectures. Our results show that dendritic EP achieves performance comparable to standard EP on simple tasks, while providing consistent improvements on more challenging datasets and deeper models. In particular, dendritic EP significantly outperforms standard EP on KMNIST and FMNIST, and approaches the performance of dendritic networks trained with backpropagation through time.To further understand these improvements, we analyze the evolution of hidden states during the free phase. We observe that dendritic EP exhibits higher activation magnitudes and more distributed hidden-state activity compared to standard EP, indicating that dendritic structure alters the internal network dynamics. These findings suggest that incorporating dendritic structure can enhance the effectiveness of biologically plausible learning algorithms, especially in regimes where standard EP struggles. Our work highlights the importance of architectural design for improving biologically inspired training methods.

Abstract

Equilibrium Propagation

Explore similar work

Dendritic Neural Networks with Equilibrium Propagation

Hybridizing Equilibrium Propagation with Ising Machines for Efficient Energy-Based Learning

A Variational Latent Equilibrium for Learning in Neuronal Circuits