Replay in the Silent Degrees of Freedom: Continual Learning Without an Offline Phase

Yanhai Zhang; Jie Zhang; Chi Xu

cs.LGSep 9, 2026

Replay in the Silent Degrees of Freedom: Continual Learning Without an Offline Phase

Authors: Yanhai Zhang, Jie Zhang, Chi Xu

Organizations: Agency for Science, Technology and Research (A⋆STAR), Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore

Abstract

Replay-based continual learning rehearses past data either in an offline phase, during which the agent stops acting, or interleaved with the live stream, where it perturbs the computation serving the current input. An agent that learns in deployment can afford neither. Motivated by local sleep, the use-dependent off periods of individual cortical circuits in awake animals, we show that replay can instead be written into the degrees of freedom the current input leaves unused. In a network with k-winner-take-all hidden layers, confining replay updates to synapses whose presynaptic unit is silent or whose postsynaptic unit is inactive leaves the hidden computation on the current batch invariant: exactly so for silent and suppressed units, and for all but 0.3% of samples in practice. A refractory rule under which units that have just fired sit out the next competition doubles the width of this channel and carries most of the accuracy. On class-incremental split-MNIST the resulting learner, with no offline phase, matches or exceeds the best offline rehearsal schedule and outperforms experience replay, ER-ACE and unmasked interleaved replay, each re-tuned under the same micro-batch schedule. Against DER++ the comparison splits by protocol: with five epochs per task DER++ leads by 1.5 points once it runs under that schedule, and in a single pass over the stream, the regime closest to the agent deployment setting, the system leads it by 1.6 while offline rehearsal falls 15 points behind. On split CIFAR-10 it again leads offline rehearsal and experience replay, but trails ER-ACE and DER++ by two to three points. The construction is not tied to the local learner: on a backprop network with k-winner-take-all hidden layers under the same schedule, refractory rotation adds half a point in five epochs and three in a single pass, and isolation again costs nothing on top of it.

Figures & tables

Panel of Figure 1 — Figure 1: Mechanism. (a) During each waking batch, replay from the episodic buffer is written only into synapses whose update is invisible to the current input, exactly so for silent presynaptic and suppressed units (Propositions 1 – 2 ); refractory rotation forces recently used units into the consolidable set (Corollary 1 ). (b) Two internal signals: unit-level homeostatic pressure triggers replay bursts, and a relative-novelty gate decides when rotation runs. The default system runs rotation always on and replays one micro-batch at every waking batch; the pressure trigger and the novelty gate are the schedulers studied under sparse replay budgets and lower activity fractions.

Panel of Figure 2 — Figure 2: Consolidation schedules. Ticks are waking batches. The offline night pauses the stream; local sleep consolidates during it, in replay bursts confined to asleep units.

Panel of Figure 3 — Figure 3: Training data flow. Each waking batch is inferred under the current suppression mask; its awake sets define the synapse and unit masks for the replay micro-batch applied in the same step and the rotation mask for the next batch. The masked step confines the velocity and the decay to the mask; the readout row is the one deliberately plastic exception. Unit-level pressure decides when replay bursts run.

Figure 4: Buffer size. Split-MNIST sequential accuracy against the buffer size

K

for the system and the replay references (held-out split, six seeds).

Figure 5: Left (isolation): final split-MNIST accuracy vs. replay batch size (default substrate, heavy-ball SGD); the number of updates is fixed within each series (one per waking batch;

20

per epoch for the night, whose samples therefore fall with its batch). The full system is flat down to

8

samples and loses

1.9

4

(

12.7

2

, Appendix H ); rotation with unmasked replay (dashed) trails it by

0.1

0.3

, within seed noise, and fails severely in half the seeds at batch

2

; without rotation, unmasked and isolated replay and the offline night all degrade. Right (timing): accuracy vs. number of waking replay batches. Trickles and clocked bursts fail alike; pressure-triggered bursts (star) keep most of the accuracy at a third of the events; surprise triggering fires no events in this regime.

	Online	Anytime	Worst	Stability gap	Churn
isolated replay $+$ rotation	$97.7\pm 0.1$	$94.9\pm 0.3$	$66.2\pm 4.4$	$\mathbf{1.5\pm 0.3}$	$\mathbf{0.33\pm 0.04}$
rotation, unmasked replay	$97.8\pm 0.1$	$95.4\pm 0.1$	$69.7\pm 2.9$	$2.3\pm 1.1$	$0.62\pm 0.06$
isolated replay, no rotation	$97.6\pm 0.1$	$94.1\pm 0.3$	$63.9\pm 1.9$	$16.9\pm 1.8$	$0.30\pm 0.03$
unmasked, no rotation	$95.9\pm 1.3$	$92.9\pm 1.4$	$32.9\pm 26.0$	$15.9\pm 7.3$	$4.39\pm 2.15$
offline rehearsal	$87.6\pm 0.2$	$83.3\pm 0.7$	$33.1\pm 4.9$	$38.4\pm 0.2$	—
no replay	$91.3\pm 0.7$	$43.0\pm 0.1$	$18.7\pm 0.2$	$51.8\pm 0.0$	—

Table 1: Measured while the stream runs (split-MNIST,

K{=}1000

, held-out split; six seeds for the first two rows, three for the rest). Churn is undefined for schedules that do not replay during the stream.

Figure 6: Left: accuracy on the classes seen so far, measured every

50

waking batches; vertical lines are task switches and bands are one standard deviation over seeds. Offline rehearsal recovers the old tasks only when its night arrives, so for most of a task the stream is served by a network that has not yet been repaired. Right: the fraction of the current batch whose prediction the replay update changes, on a logarithmic axis, for the four cells of the isolation and rotation square; every seed is drawn.

Backprop, $512$ – $256$	lr	Sequential	i.i.d. (static)	Seq., single pass
$+$ $k$ -WTA, interleaved replay (unmasked)	$3{\cdot}10^{-5}$	$91.7\pm 0.4$	$97.2\pm 0.1$	$89.3\pm 0.6$
$+$ $k$ -WTA, isolated replay	$10^{-4}$	$84.7\pm 2.6$	$97.6\pm 0.1$	$81.9\pm 2.1$
$+$ $k$ -WTA, interleaved replay $+$ rotation	$10^{-4}$	$92.2\pm 0.3$	$96.7\pm 0.3$	$92.7\pm 0.3$
$+$ $k$ -WTA, isolated replay $+$ rotation	$10^{-4}$	$92.1\pm 0.3$	$96.6\pm 0.1$	$92.8\pm 0.2$
ER	$3{\cdot}10^{-5}$	$90.8\pm 0.6$	—	$87.8\pm 0.6$
ER-ACE	$3{\cdot}10^{-4}$	$89.7\pm 0.4$	—	$90.5\pm 0.6$

Table 2: Backprop learners under our schedule (

512

–

256

, Adam with weight decay

10^{-3}

, waking batches of

16

with one

16

-sample replay micro-batch each,

K{=}1000

; held-out split, six seeds; the learning rate of every row re-selected on the development split). Upper block:

k

-WTA hidden layers at the same

10\%

fraction as the local learner. Middle block: the published baselines of Table 3 , unchanged except for the schedule. Lower block: the local system, for reference.

Variant	Sequential	i.i.d. (static)	Seq., single pass
full system : isolated replay $+$ always-on rotation (heavy-ball SGD)	$\mathbf{91.6\pm 0.3}$	$94.1\pm 0.2$	$\mathbf{91.8\pm 0.3}$
rotation gated by relative novelty, committed to bouts	$91.8\pm 0.3$	$94.4\pm 0.3$
rotation gated per batch	$87.5\pm 1.6$	$94.7\pm 0.1$
$+$ two-timescale anchor ( $\lambda{=}\mu{=}3{\cdot}10^{-4}$ ), rotation always on	$91.7\pm 0.0$	$93.8\pm 0.3$
$-$ rotation (same mask, Eq. 6 , natural silence only)	$88.0\pm 0.7$	$95.4\pm 0.2$	$86.4\pm 1.0$
$-$ isolation, $-$ rotation (unmasked interleaved replay)	$85.4\pm 3.5$	$93.8\pm 5.0$	$83.0\pm 14.9$

Table 3: Component ablation of the consolidation mechanism on the default substrate (

512

–

256

10\%

; backprop references dense, at the widths selected in Section 4.4 ). Sequential

=

final accuracy on split-MNIST; static

=

i.i.d. MNIST (the axis that must not degrade). Dashes: combination not applicable; blank: not run. Single pass

=

one epoch per task, otherwise identical. Held-out split; six seeds, except the gate, anchor, optimiser and unmasked-Adam rows (three).

Figure 7: The operating frontier at the

10\%

fraction under heavy-ball SGD: split-MNIST sequential accuracy against i.i.d. static accuracy (headline points six seeds). Blue: non-dominated by seed means (the bout-committed gate and the silent mask); purple: dominated; held-out split. The top cluster (always-on rotation, bout gate, anchor, masked Adam) lies within seed noise of one another; the frontier trades sequential for static accuracy through the silent mask, the unmasked rotation-free control is dominated by it once its collapsed seed is counted, and the adaptive-

\lambda

point (Section 7 ) is dominated on this split.

Figure 8: Left: the substrate decomposition on feature CIFAR-10, transplanting the local network’s ingredients into BP+ER one at a time (each cell at its better of two learning rates; three seeds). Right: spiking inference, i.i.d. MNIST accuracy against the idealised 45-nm FP32 arithmetic-energy proxy for

T_{s}\in\{4,8,16,32\}

, with the dense (square) and event-driven (diamond) rate references; held-out split; the default is monotone in

T_{s}

, the rotation-free control dips

0.4

T_{s}{=}32

	fixed $\lambda$ (tuned)		controller (drive)
	seq.	static	seq.	static
split-MNIST	$91.6\pm 0.3$	$94.1\pm 0.2$	$90.6\pm 0.6$	$93.5\pm 4.3$
split CIFAR-10	$28.4\pm 0.8$	$28.0\pm 1.2$	$26.2\pm 1.1$	$30.1\pm 0.2$
feature CIFAR-10	$42.3\pm 0.6$	$42.4\pm 1.1$	$39.1\pm 0.4$	$43.6\pm 0.6$
split CIFAR-100	$1.2\pm 0.1$	$1.0\pm 0.1$	$3.2\pm 0.8$	$4.1\pm 0.6$
+ features	$1.1\pm 0.3$	$1.1\pm 0.2$	$5.2\pm 1.6$	$6.4\pm 0.6$

Table 4: One dataset-independent ratio target replaces per-dataset decay tuning. Sequential / static accuracy (%) under the tuned fixed decay (

10^{-3}

on the image benchmarks;

10^{-4}

with

3\times

targets as the CIFAR-100 rescue) against the drive-referenced controller of ( 10 ) at the same

\rho=0.25

everywhere; held-out split, three seeds (six on split-MNIST and in the fixed-decay split CIFAR-10 cell); the full system (refractory rotation, isolated replay,

K{=}1000

) in every cell.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Channel	Condition	Invisible to $X$	Source
presynaptic silent	$j\notin\mathcal{A}^{\ell-1}(X)$	for any magnitude	Prop. 1
postsynaptic naturally asleep, active pre	$i\notin\mathcal{A}^{\ell}(X)$ , $s^{\ell}_{i}=0$	under the margin $\tau^{\ell}_{k,b}$	Prop. 2
postsynaptic suppressed	$s^{\ell}_{i}=1$	for any magnitude	Cor. 1

Appendix

Table 5: The three consolidation channels opened by the synapse mask.

Substrate	Accuracy	Syn. ops
dense backpropagation MLP (autograd reference)	$97.7\pm 0.1$	1.00
local burst-error learner, unconstrained	$97.8\pm 0.1$	0.85
$+$ bounded error, Dale’s law, sign-concordant feedback (no transport)	$97.1\pm 0.0$	0.85
$+$ $k$ -WTA $10\%$	$97.0\pm 0.2$	0.80
$+$ relaxation $T{=}5$	$97.0\pm 0.1$	0.80
$+$ $30\%$ distance-dependent connectivity	$96.5\pm 0.2$	0.24

Appendix

Table 6: Substrate ablation on i.i.d. MNIST (

256

–

128

hidden core, three seeds; development phase, official test set). “Syn. ops” is the fraction of the dense MLP’s synaptic operations in one forward pass. Upper block: constraints added cumulatively; lower block: one ingredient replaced at its stage.

Figure 9: Rotation: price, trade-off, and gate. Left: on i.i.d. MNIST the novelty gate restores no-rotation accuracy. Middle: on split-MNIST rotation is required and the gate keeps it. Right: on split CIFAR-10 (grayscale) rotation is a gain on both axes.

Figure 10:

K=200

mechanism triangulation. Isolated rotating replay beats the unmasked trickle, diversity helps only during wake, and the irreducible advantage of the night at tiny buffers is its interference-free window.

Figure 11: Isolation across replay batch sizes (held-out split). Left: split-MNIST accuracy against replay batch size for isolated replay (the full system) and for rotation with unmasked replay, every seed shown; at batch

2

no unmasked mean is drawn because three of six seeds fail severely (values at the axis floor). Right: the seed-paired margin of isolation with a 95% bootstrap interval at the batch sizes where the unmasked run does not collapse.

	development set	held-out split 1	held-out split 2
always-on rotation (default)	$92.7\pm 0.3$	$91.6\pm 0.3$	$91.8\pm 0.4$
static	$94.7\pm 0.3$	$94.1\pm 0.2$	$94.1\pm 0.6$
bout-committed gate	$92.3\pm 0.2$	$91.8\pm 0.3$	$92.0\pm 0.6$
rotation alone (unmasked replay)	$92.1\pm 0.5$	$91.5\pm 0.3$	$91.8\pm 0.6$
static	$94.7\pm 0.1$	$93.7\pm 0.3$	$93.7\pm 0.3$
silent mask (no rotation)	$88.8\pm 0.8$	$88.0\pm 0.7$	$88.6\pm 1.3$

Appendix

Table 7: Development-set (official test split) numbers beside the held-out numbers and a second held-out split. Sequential accuracy, with the static axis on its own row where the paper reports it; six seeds on the reported split, three on the development set and on the second split. Lower blocks: single pass, and raw split CIFAR-10.

after task	T1	T2	T3	T4	T5
1	$99.8$
2	$98.7$	$97.2$
3	$97.7$	$92.7$	$97.0$
4	$97.5$	$89.6$	$92.9$	$97.5$
5	$96.7$	$88.1$	$89.7$	$91.5$	$91.8$

Appendix

Table 8: Per-task accuracy matrix of the headline configuration on split-MNIST (held-out split, mean over six seeds): row

=

after training task

i

, column

=

accuracy on task

j

. Forgetting

F

, the mean of the diagonal minus the last row over tasks

1

–

4

, is

6.4\pm 0.6

; the drop is concentrated on tasks

2

–

3

and the first task is retained.

Replay-Based Continual Learning

Replay-based continual learning preserves prior-task knowledge by revisiting stored or generated experience while adapting models to new data.

See all 134 papers

Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Tasks
Jun 7, 2026Anthony Bazhenov, Jean Erik Delanois, Giri P. KrishnanReplay-Based Continual Learning Memory Consolidation
Khoury College of Computer Sciences, Northeastern University, Boston, MA · Department of Medicine, University of California, San Diego, La Jolla, CA, USA · ARTISAN, Georgia Institute of Technology, Atlanta, GA USA
Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning
Aug 12, 2026Tieliang Gong, Zhongbo Zhang, Wen Wen +1Information-Theoretic Generalization Bounds Replay-Based Continual Learning
Tieliang Gong, Zhongbo Zhang and Wen Wen are with the School of Computer Science and Technology, Xi’an Jiaotong University, China · Yong-Jin Liu is with Department of Computer Science and Technology, Tsinghua University, China
Beyond Class Marginals: Bounding Rehearsal Gaps without Freezing Class Co-occurrence
Sep 22, 2026Congren Dai, Nat Roongjirarat, Fei YeClass-Incremental Learning Replay-Based Continual Learning
Imperial College London · King’s College London · University of Electronic Science and Technology of China

Jun 7, 2026J/K move · Enter open · S save

Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Tasks

Anthony Bazhenov, Jean Erik Delanois, Giri P. Krishnan

Khoury College of Computer Sciences, Northeastern University, Boston, MA · Department of Medicine, University of California, San Diego, La Jolla, CA, USA · ARTISAN, Georgia Institute of Technology, Atlanta, GA USA

Replay-Based Continual Learning Memory Consolidation Catastrophic Forgetting

Open paper AskPDF arXiv

One of the critical limitations of artificial neural networks is their lack of ability to continually learn: training on new tasks often leads to interference and forgetting of the previous ones. While several algorithms have been proposed to protect old memories from interference, they are typically applied during or immediately after each new episode of training. In contrast, humans and animals can learn continuously, acquiring multiple new memories during active learning before consolidating all of them into long-term storage. Here we show that multiple new tasks can be trained sequentially before an unsupervised sleep-like replay phase is applied to partially restore performance across all previously learned tasks. Our study further suggests that task-specific information remains resilient to new training but decays gradually as network is trained on new tasks. These findings point to novel principles for developing a broad range of continual learning AI solutions.

Abstract

Replay-Based Continual Learning

Explore similar work

Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Tasks

Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning

Beyond Class Marginals: Bounding Rehearsal Gaps without Freezing Class Co-occurrence