Hebbian Attractor Networks for Robot Locomotion

Alexander Dittrich; Fuda van Diggelen; Dario Floreano

cs.NEMar 23, 2026

Hebbian Attractor Networks for Robot Locomotion

Authors: Alexander Dittrich, Fuda van Diggelen, Dario Floreano

Organizations: Laboratory of Intelligent Systems, EPFL, Switzerland

Abstract

Biological neural networks continuously adapt and modify themselves in response to experiences throughout their lifetime - a capability largely absent in artificial neural networks. Hebbian plasticity offers a promising path toward rapid adaptation in changing environments. Here, we introduce Hebbian Attractor Networks (HAN), a class of plastic neural networks in which local weight update normalization induces emergent attractor dynamics. Unlike prior approaches, HANs employ dual-timescale plasticity and temporal averaging of pre- and postsynaptic activations to induce either co-dynamic limit cycles or fixed-point weight attractors. Using simulated locomotion benchmarks, we gain insight into how Hebbian update frequency and activation averaging influence weight dynamics and control performance. Our results show that slower updates, combined with averaged pre- and postsynaptic activations, promote convergence to stable weight configurations, while faster updates yield oscillatory co-dynamic systems. We further demonstrate that these findings generalize to high-dimensional quadrupedal locomotion with a simulated Unitree Go1 robot. These results highlight how the timing of plasticity shapes neural dynamics in embodied systems, providing a principled characterization of the attractor regimes that emerge in self-modifying networks.

Figures & tables

Fig. 2 : Dual-timescale structure of HNN. Hebbian updates are executed at a lower frequency than the forward pass (here,

\tau_{\text{hebb}}=3\times\tau_{\text{NN}}

), and the moving average of pre- and postsynaptic activations is applied (here,

M=2

Condition	MN	$M$	$f_{\text{NN}}/f_{\text{hebb}}$
(A) HNN	✗	1	1
(B) HAN ${}^{M=1}_{f_{\text{NN}}=f_{\text{hebb}}}$	✓	1	1
(C) HAN ${}^{M=1}_{f_{\text{NN}}=4\times f_{\text{hebb}}}$	✓	1	4
(D) HAN ${}^{M=10}_{f_{\text{NN}}=f_{\text{hebb}}}$	✓	10	1
(E) HAN ${}^{M=10}_{f_{\text{NN}}=4\times f_{\text{hebb}}}$	✓	10	4

TABLE I : Overview of tested HAN conditions.

Fig. 3 : Analysis of weight dynamics under three Hebbian learning conditions (A, B, E). The first row shows trajectories of 10 randomly selected plastic weights over time. The second row presents the total network plasticity, quantified as the summed

\ell_{2}

-norm of weight changes at each timestep. The third row depicts the evolution of plastic weights projected onto the first two principal components via Principal component analysis (PCA).

Algorithm	Fitness (-)
Algorithm	Swimmer	Hopper	HalfCheetah	Walker2D
PPO ‡	$134.4{{}_{\pm 1.8}}$	$2641.2{{}_{\pm 1133.1}}$	$3722.9{{}_{\pm 1842.4}}$	$4022.8{{}_{\pm 2253.7}}$
PPO	$135.7{{}_{\pm 3.5}}$	$4068.8{{}_{\pm 179.5}}$	$9644.7{{}_{\pm 1084.4}}$	$7947.8{{}_{\pm 899.6}}$
MLP †	$361.4{{}_{\pm 0.4}}$	$3470.9{{}_{\pm 195.2}}$	$3496.7{{}_{\pm 382.6}}$	$3292.3{{}_{\pm 121.6}}$
MLP	$361.9{{}_{\pm 0.3}}$	$4186.0{{}_{\pm 129.8}}$	$3168.7{{}_{\pm 115.5}}$	$4310.9{{}_{\pm 339.1}}$
GRU	$363.7{{}_{\pm 0.4}}$	$3676.8{{}_{\pm 159.5}}$	$4454.6{{}_{\pm 413.5}}$	$4066.1{{}_{\pm 418.9}}$

TABLE II : Average accumulated episodic return / fitness on Gymnasium locomotion benchmarks (

n=16

for HNN/HAN and MLP/GRU ;

n=10

for PPO ), where

\pm

captures a

95\text{\,}\mathrm{\%}

confidence interval. MLP † omits bias terms to match the parameter count of HNNs/HANs. PPO ‡ matches the number of optimization parameters of HNNs/HANs.

Fig. 4 : The non-zero frequencies of action and weight signals exhibit peaks at approximately

4\text{\,}\mathrm{Hz}

and

8\text{\,}\mathrm{Hz}

. Interrupting the Hebbian update causes a pronounced drop in step rewards during deployment for the HAN condition (B). The HNN in condition (E) shows a minor effect in step rewards.

Fig. 5 : Fitness at 1000 generations with different

f_{\text{hebb}}

and

M

(

n=16

). A window length of

M=1

corresponds to no moving average, and

f_{\text{hebb}}=20

corresponds to no reduction in update frequency.

Fig. 6 : Stepwise rewards summed weight changes during a $50\text{\,}\mathrm{s}$ deployment. A collision around $19\text{\,}\mathrm{s}$ causes a drop in reward, followed by recovery. Weight changes decrease over time, with a brief spike after the collision. The network returns to a similar weight configuration after the disturbance, indicating a stable attractor.

	Hebbian update frequency $f_{\text{hebb}}$
$M$	$1\text{\,}\mathrm{Hz}$	$5\text{\,}\mathrm{Hz}$	$10\text{\,}\mathrm{Hz}$	$20\text{\,}\mathrm{Hz}$
1	$99.9\text{\,}\mathrm{\%}$	$64.4\text{\,}\mathrm{\%}$	$21.9\text{\,}\mathrm{\%}$	$21.4\text{\,}\mathrm{\%}$
2	$99.9\text{\,}\mathrm{\%}$	$64.9\text{\,}\mathrm{\%}$	$16.1\text{\,}\mathrm{\%}$	$17.8\text{\,}\mathrm{\%}$
4	$99.9\text{\,}\mathrm{\%}$	$81.7\text{\,}\mathrm{\%}$	$55.0\text{\,}\mathrm{\%}$	$36.6\text{\,}\mathrm{\%}$
10	$99.9\text{\,}\mathrm{\%}$	$94.4\text{\,}\mathrm{\%}$	$75.7\text{\,}\mathrm{\%}$	$66.0\text{\,}\mathrm{\%}$
20	$99.9\text{\,}\mathrm{\%}$	$100.0\text{\,}\mathrm{\%}$	$100.0\text{\,}\mathrm{\%}$	$92.9\text{\,}\mathrm{\%}$

TABLE III : Ratio of HANs satisfying the convergence criterion for different

f_{\text{hebb}}

and

M

values over

1440

different evaluations.

Fig. 7 : PCA of HANs in conditions (B) and (E).

Fig. 8 : Mean foot air time versus forward velocity for HANs in conditions (B) and (E) across $60$ Unitree Go1 training runs, classified as fixed-point or limit-cycle attractors.

Fig. 9 : Mean accumulated returns with

95\text{\,}\mathrm{\%}

confidence intervals for each scenario. The left side shows training scenarios, while the right side shows unseen test scenarios. PPO significantly outperforms HANs within the training distribution, while HANs achieve higher returns on unseen test scenarios.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Training settings
Hyperparameter	Value
Number of generations	$500$ (Swimmer) / $1000$ (rest)
Repeats	$4$
Action clipping	$-1,1$
Observation normalization	Running standard normalization (Chan’s online algorithm)
Evolutionary Algorithm

Appendix

TABLE IV : Hyperparameters of controller evolution.

Controller	Architecture	Bias	Swimmer-v5	Hopper-v5	HalfCheetah-v5	Walker2d-v5
HNN / HAN	MLP (16)	✗	800	1120	1840	1840
PPO tiny	MLP (16)	✓	178	243	390	390
PPO default	MLP (64, 64)	✓	4866	5123	5702	5702
ES MLP	MLP (80)	✗	800	1120	1840	1840
ES MLP	MLP (variable)	✓	805	1113	1830	1830
ES GRU	GRU (variable)	✓	782	1137	1893	1893

Appendix

TABLE V : Number of trainable parameters for different neural controllers across Gymnasium benchmarks.

Hyperparameter	Value
Learning rate	3e-4
Number of steps per rollout ( $n_{\text{steps}}$ )	2048
Minibatch size	64
Number of epochs	10
Discount factor ( $\gamma$ )	0.99
GAE lambda	0.95

Appendix

TABLE VI : Default hyperparameters for PPO.

Parameter	Value
Population Size	$512$
Optimizer	Adam
Momentum coefficient ( $\beta_{1}$ )	$0.9$
Second moment coefficient ( $\beta_{2}$ )	$0.999$
Epsilon	$1\times 10^{-8}$
Learning Rate Schedule	Exponential Decay

Appendix

TABLE VII : Evolutionary algorithm parameters for Unitree Go1 locomotion.

Fig. 10 : Unitree Go1 reward function components.

Fig. 11 : Mean and

95\text{\,}\mathrm{\%}

confidence intervals of 4 independent rollouts of

n=16

randomly initialized training runs of static neural networks with ES.

Fig. 12 : Mean and $95\text{\,}\mathrm{\%}$ confidence intervals of 4 independent rollouts of $n=16$ randomly initialized meta-training runs of Hebbian update rules in different conditions with ES.

Fig. 13 : Mean and

95\text{\,}\mathrm{\%}

confidence intervals of

n=10

randomly initialized training runs with PPO.

Fig. 14 : Weight dynamics of HalfCheetah-v5 without max normalization.

Fig. 15 : Weight dynamics across five random weight initializations of HAN ${}^{M=1}_{f_{\text{NN}}=f_{\text{hebb}}}$ (limit-cycle attractor). Left : Time evolution of three selected plastic weights. Right : Projection onto the first two global PCA components reveals consistent oscillatory patterns across seeds.

Fig. 16 : Weight dynamics across five random weight initializations of HAN

{}^{M=10}_{f_{\text{NN}}=4\times f_{\text{hebb}}}

(fixed-point attractor). Left : Time evolution of three selected plastic weights. Right : PCA projection shows rapid convergence to a compact region in weight space regardless of initial weights.

Fig. 17 : Fitness of HNNs on HalfCheetah-v5 with weight clipping at various thresholds

\varepsilon

. Moderate clipping (

\varepsilon=5

) preserves learning, but max normalization outperforms all clipped variants.

Fig. 18 : Fitness across 16 evolutionary runs for different regularization methods. Oja’s rule does not yield competitive performance compared to max normalization.

Fig. 19 : Weight dynamics of Oja-regularized HNNs during deployment. (1) : Selected plastic weights remain bounded and non-periodic. (2) : Summed weight change decreases initially and stabilizes with high oscillations. (3) :

\ell_{2}

-distance between time instances remains high. (4) : PCA projection shows non-oscillatory, continuous drift.

Fig. 20 : Best-performing individual from each of 16 evolutionary runs on HalfCheetah-v5 for constant and co-evolved Hebbian learning rates. No clear advantage is observed for evolving the learning rate.

Size	Hidden layer size	Optimization parameters	Network parameters
XXXS	2	184	46
XXS	4	368	92
XS	8	736	184
S	16	1472	368
M	32	2944	736
L	32, 32	7040	1760

Appendix

TABLE VIII : Neural network architectures by size and parameter count.

Fig. 21 : Performance of 16 evolutionary runs per network size on HalfCheetah-v5. Left : Mean and maximum performance vs. optimization parameter count. Right : Fitness distribution across seeds for each architecture.

Panel of Figure 29 — Fig. 22 : Attractor switching on HalfCheetah-v5 during deployment. Hebbian coefficients are swapped at $16.6\text{\,}\mathrm{s}$ and $33.3\text{\,}\mathrm{s}$ , causing the network to transition between different fixed-point attractors.

Hebbian Learning

Hebbian learning studies how synapses adapt through correlations between neural activity, enabling local learning, memory, and adaptation.

See all 21 papers

Activity-Dependent Plasticity in Morphogenetically-Grown Recurrent Networks
Apr 3, 2026Sergii Medvid, Andrii Valenia, Mykola GlybovetsRecurrent Neural Networks Neuroevolution
National University of Kyiv-Mohyla Academy, Kyiv, Ukraine
Hebbian-Oscillatory Co-Learning
Feb 21, 2026Hasi HaysSparse Neural Networks Oscillatory Neural Networks
Department of Chemical Engineering, University of Arkansas, Fayetteville, AR 72701, USA
Adapting Neural Robot Dynamics on the Fly for Predictive Control
Apr 5, 2026Abdullah Altawaitan, Nikolay AtanasovRobot Dynamics Learning Robot Dynamics Identification
Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093, USA · Kuwait University

Apr 3, 2026J/K move · Enter open · S save

Activity-Dependent Plasticity in Morphogenetically-Grown Recurrent Networks

Sergii Medvid, Andrii Valenia, Mykola Glybovets

National University of Kyiv-Mohyla Academy, Kyiv, Ukraine

Recurrent Neural Networks Neuroevolution Hebbian Learning Neural Processes

Open paper AskPDF arXiv

Developmental approaches to neural architecture search grow functional networks from compact genomes through self-organisation, but the resulting networks operate with fixed post-growth weights. We characterise Hebbian and anti-Hebbian plasticity across 50,000 morphogenetically grown recurrent controllers (5M+ configurations on CartPole and Acrobot), then test whether co-evolutionary experiments -- where plasticity parameters are encoded in the genome and evolved alongside the developmental architecture -- recover these patterns independently. Our characterisation reveals that (1) anti-Hebbian plasticity significantly outperforms Hebbian for competent networks (Cohen's d = 0.53-0.64), (2) regret (fraction of oracle improvement lost under the best fixed setting) reaches 52-100%, and (3) plasticity's role shifts from fine-tuning to genuine adaptation under non-stationarity. Co-evolution independently discovers these patterns: on CartPole, 70% of runs evolve anti-Hebbian plasticity (p = 0.043); on Acrobot, evolution finds near-zero eta with mixed signs -- exactly matching the characterisation. A random-RNN control shows that anti-Hebbian dominance is generic to small recurrent networks, but the degree of topology-dependence is developmental-specific: regret is 2-6x higher for morphogenetically grown networks than for random graphs with matched topology statistics.

Abstract

Hebbian Learning

Explore similar work

Activity-Dependent Plasticity in Morphogenetically-Grown Recurrent Networks

Hebbian-Oscillatory Co-Learning

Adapting Neural Robot Dynamics on the Fly for Predictive Control