Fully online embodied learning requires synaptic adaptation to acquire new behaviors while preserving previously learned dynamics during ongoing interaction. We extend the Predictive-Coding-inspired Variational Recurrent Neural Network (PV-RNN) to continuously adapt its synaptic weights and propose Free-Energy-Gated Plasticity (FEGP), which regulates the effective learning rate according to variational free energy. In real-time physical human-robot interaction, a randomly initialized network acquired three cyclic motor patterns without offline pretraining, replay, or task-boundary signals, with all three patterns emerging in autonomous rollouts. Controlled experiments over ten randomized teaching streams and five network initializations per stream showed that FEGP substantially improved repertoire coverage and retention of previously acquired patterns after they left the recent observation window. Neither a constant learning rate matched to the gate's time-averaged effective rate nor replay of the same gain values with disrupted temporal organization reproduced these improvements. These results indicate that the temporal allocation of plasticity relative to model-environment mismatch, rather than simply its average magnitude or distribution, is critical for maintaining previously acquired behaviors during continued online learning.
Figures & tables
Figure 1: Overview of the fully online learning loop and the offline evaluation pipeline. Left: each model update incorporates the newest sensory observation into the sliding past window, performs posterior-state inference through error regression, adapts the synaptic weights through gated BPTT, and generates a future trajectory for motor control and subsequent analysis. Right: saved trajectories are segmented and classified post hoc using the evaluation pipeline described in Sec. 3.3 . The evaluation pipeline does not feed information back into the online learning loop. The numerical settings used in the experiments are listed in Table 1 .
Figure 2: Schematic illustration of Free-Energy–Gated Plasticity. The plasticity gain g(st) is a sigmoid function of the gate signal st , defined as the logarithm of the window-averaged variational free energy (Eq. 6 ). In the single-threshold formulation, one threshold λ determines the transition between lower and higher plasticity. In the hysteretic formulation, the active threshold is selected from λlow and λhigh according to the current plasticity regime. The single-threshold gate is recovered when λlow=λhigh=λ .
Component
Parameter
Value
Network
Sensory layer
softmax units, σ
10, 0.05
PV-RNN L1
d , z , τ
40, 4, 3
PV-RNN L2
d , z , τ
20, 2, 9
Both PV-RNN layers
meta-prior w
0.01
Online inference and learning
Table 1: Shared model, inference, and optimization settings.
Condition
Setting
FEGP
(λlow,λhigh)=(−9,−7) , T=0.1
Constant-plasticity baseline
g(st)=1
Single-threshold sweep
λ∈{−10.0,−9.5,…,−6.0} , T=0.1
Hysteretic-gate sweep
λlow∈{−10,−9,−8} ,
λhigh∈{−8,−7,−6}
Constant-rate control
αconst=αg(st) under FEGP
Table 2: Conditions in Experiment 1.
Figure 3: Effect of hysteresis on the temporal dynamics of the plasticity gate. (a) Representative gain trajectories under the single-threshold setting λ=−8 and the hysteretic setting (λlow,λhigh)=(−9,−7) . (b) Number of threshold crossings or regime transitions during the interaction. (c) Fraction of model updates with intermediate gain, defined by 0.1<g(st)<0.9 . The hysteretic formulation strongly reduces rapid switching and yields more clearly separated low- and high-plasticity regimes.
Condition
Cov. (%)
Ret. (%)
Δ vs. FEGP (pp)
FEGP
45.9
23.1
—
Constant
18.9
0.0
−27.0/−23.1
Matched rate
12.5
0.2
−33.4/−22.9
Permuted gain
21.3
0.5
−24.6/−22.6
Reversed gain
11.4
0.8
−34.5/−22.3
Shifted gain
17.1
2.2
−28.8/−20.9
Table 3: Repertoire coverage and retention under FEGP and its control conditions. Differences are relative to FEGP and are reported as coverage / retention in percentage points.
Figure 4: Retention of movement patterns as a function of the time elapsed since they left the 500-update observation window. The value at 0 corresponds to patterns still present within the observation window. Under FEGP, availability drops after a pattern leaves the window but remains substantially above the two constant-rate controls over longer absence intervals. In contrast, retention under constant plasticity and the matched-rate control falls to near zero shortly after the pattern leaves the window.
Figure 5: Development of the generated movement repertoire during real-time physical interaction in Experiment 2. Each row shows one independent session. Left: temporal evolution of the number of identified movement types and the unidentified-segment ratio in the open-loop rollouts generated during learning. Right: autonomous trajectories generated using parameters saved at model updates t=1,200 , 2,000 , and 6,000 . The trajectories were segmented and classified using the evaluation pipeline; segments assigned to patterns A, B, and C are color-coded, while unclassifiable segments are labeled “?”.
Setting
Precision
Recall
F1
Matching tolerance, at threshold 0.03
±3 model updates
1.000
1.000
1.000
±5 / ±10 / ±15
1.000
1.000
1.000
Detection threshold, at tolerance ±5
0.03 (operating point)
1.000
1.000
1.000
0.10
1.000
1.000
1.000
Table S1: Boundary detection against ground truth (four streams, 459 boundaries).
Stream
Boundary-step / within-cycle median
F1 ( ±5 )
Class accuracy
Unmodified
1.94
1.000
1.000
Crossfade (width 9)
1.21
1.000
1.000
Smoothed (width 5)
0.44
1.000
1.000
Time-warped ±10%
1.96
1.000
1.000
Warped + crossfade
1.25
1.000
1.000
Table S2: Boundary detection after removing the concatenation artifact.
Trajectory set
Summed score, thr. 35
Rule used
Shape distance (median)
Recorded cycles (30)
0.00
0.00
0.36
Time-reversed
0.00
0.02
0.64
Block-permuted
0.00
0.98
2.35
Two patterns spliced
0.00
1.00
3.43
Half cycle
0.80
1.00
2.62
Two cycles concatenated
0.35
1.00
2.59
Table S3: Rejection performance on recorded and control trajectories.
Boundary detection
Detector
Tol.
Prec.
Rec.
F1
All 30 cycles
±3
1.000
1.000
1.000
All 30 cycles
±5
1.000
1.000
1.000
Demos 1–5
±3
0.967
0.967
0.967
Demos 1–5
±5
1.000
1.000
1.000
Cycle classification
Table S4: Evaluation on a stream assembled only from held-out demonstrations.
Figure S1: Late-stage summary of the single-threshold sweep in Experiment 1. Left: weight-update magnitude averaged over the final 2,000 model updates as a function of λ for four representative teaching streams (Seq1–4). Right: corresponding late-stage log-scaled free energy for the same streams. Increasing λ progressively suppresses weight adaptation while residual free energy increases.
Figure S2: Composition of the open-loop rollout over online learning under the single-threshold plasticity gate in Experiment 1. Rows correspond to threshold values λ∈{−10.0,…,−6.0} , and columns show the same four representative teaching streams (Seq1–4) used throughout this section. At each online-learning update, the model’s 3,000-step rollout was segmented and classified as movement pattern A, B, C, or Unknown using the evaluation pipeline in Sec. III-C of the main text. The displayed values use a 10-update moving average. For λ≤−8 , the rollout develops and maintains all three instructed patterns, whereas plasticity is strongly suppressed for λ≥−6.5 .
Figure S3: Composition of the open-loop rollout under Synaptic Intelligence with constant plasticity in Experiment 1. Rows correspond to Synaptic Intelligence coefficients c∈{0,10−6,10−5,10−4,10−3,10−2,10−1,1.0} , and columns show the same four representative teaching streams (Seq1–4) used in Figs. S1 and S2 . Colors are as in Fig. S2 . Weak regularization produces outcomes similar to the unregularized baseline, whereas strong regularization prevents stable acquisition.