In Two Minds about Lifelong Learning: Exploring Hemispheric Redundancy and Specialisation in Neural Models

Benjamin Smith; Levin Kuhlmann; Kaushik Roy; Gideon Kowadlo

cs.LGAug 20, 2026

In Two Minds about Lifelong Learning: Exploring Hemispheric Redundancy and Specialisation in Neural Models

Authors: Benjamin Smith, Levin Kuhlmann, Kaushik Roy, Gideon Kowadlo

Organizations: Data Science and AI, Monash University, Wellington Road, Clayton, 3800, Victoria, Australia · CSIRO Robotics, CSIRO, Australia · Cerenaut, Victoria, Australia

Abstract

Persistent intelligent systems require the ability to learn continually, but current machine learning approaches face significant challenges in this area compared to biological learning systems. Machine learning algorithms typically trade off retention of previously learned information and adaptation to new or changing data patterns. When continual learning capabilities are absent, algorithms must undergo retraining using the entire data set, an approach that becomes impractical when original training data are unavailable due to storage constraints, financial or computational costs, or privacy restrictions. However, biological animals can learn continually, without experiencing catastrophic forgetting. This paper attempts to build a high-level framework for how animals learn and preserve knowledge by modelling neural components and states that are known to be related to memory consolidation. We focus on three concepts: experience replay, REM sleep, and bilaterality. We propose 4MAS (4 Module Awake/Sleep), a novel macroarchitecture demonstrating how machine learning models might benefit from asymmetric hemispheres, each with their own long- and short-term memory mechanisms, and how a period of sleep between incremental learning tasks might benefit memory consolidation. Finally, we present results showing that our architecture achieves statistical improvements over established generative replay baselines on Split-MNIST (98.3%) and Split-Fashion-MNIST (84.9%), while reducing forgetting by more than half relative to the strongest replay baseline and demonstrating monotonic capacity scaling on Split-CIFAR-100 (29.29%).

Figures & tables

Figure 1: LTM inputs and outputs: (a) LTM training on image-label pairs for each hemisphere during awake and sleep phases. (b) Sample generation with temperature $\tau$ from each hemisphere provides two synthetic input sources for subsequent LTM training tasks. (c) During inference, both hemispheres process inputs and the highest confidence classification is selected.

Figure 2: MNIST datasets are split into 5 tasks of 2 classes each for the Class-IL scenario: (a) Split-MNIST, (b) Split-Fashion-MNIST.

Figure 3: Awake/sleep training phases: (a) Awake training: LTMs learn new task data interleaved with STM samples. (b) Sleep training: LTMs are fine-tuned on contralateral representations.

Panel of Figure 4 — Figure 4: 2-dimensional mapping of the LTM’s latent space after training. $\tau$ scales the sampling radius in units of the prior standard deviation. High temperature $\tau>1$ increases the diversity of output representations (16 latent dimensions used in final configurations).

Dataset	Method	Final Acc	Forg.	BWT	FWT ${}_{\text{probe}}$ ( $\Delta$ )	Drift
		(%) $\uparrow$	(%) $\downarrow$	(%) $\uparrow$	( $\%$ ) $\uparrow$	$\downarrow$
Split-MNIST	Joint (145M)	$98.14\pm 0.12$	—	—	—	—
	4MAS (Our method, 145M)	$98.30\pm 0.04$	$1.2$	$-1.2$	$91.5\;(-5.7)$	$1.0$
	B-IR (5M)	$93.5\pm 0.2$	$6.4\pm 0.2$	$-6.4\pm 0.2$	$98.7\;(+1.5)$	$0.2\pm 0.0$
	GR (186M)	$94.50\pm 0.49$	$8.9\pm 0.4$	$-8.9\pm 0.4$	$97.6\;(+0.5)$	$42.1\pm 1.8$
	LwF (1.9M)	$24.2\pm 0.4$	$0.0$	$0.1$	$96.7\;(-0.5)$	$15.2$

Table 1: Class-IL evaluation across Split-MNIST, Split-Fashion-MNIST, and Split-CIFAR-100 benchmarks. For each method, we report performance across capacity sweeps, with total trainable parameter counts indicated in parentheses. Best values (excluding the Joint upper bound) are in bold. Metrics display mean and standard error of the mean (

\pm\text{SEM}

) across random seeds after learning all tasks.

\text{FWT}_{\text{probe}}

denotes representation linear probe accuracy on future tasks and relative transfer (

\Delta

) above random initialisation.

Panel of Figure 5 — Figure 5: (a) Performance variation with sleep phase learning rates (LR) as a percentage of awake phase LR, Split-Fashion-MNIST. (b) Dominance scores categorised as RH (<50%), symmetrical (50%), or LH (>50%). Memory consolidation during sleep phase increased LH dominance, while new task learning during awake phase increased RH dominance; produced by lateralisation techniques (class representation bias by age in LH and asymmetric generator temperatures).

Panel of Figure 6 — Figure 6: Fashion-MNIST: (a) Hemispheric accuracy for each task. (b) $\tau$ -asymmetry results, with and without sleep training phase.

		$\tau$ - right
		1	2	3	4	5
$\tau$ - left	1	0.6301	0.7122	0.7445	0.7132	0.6984
	2	0.7021	0.6895	0.6662	0.6342	0.6178
	3	0.6768	0.6116	0.5907	0.5595	0.5504
	4	0.6593	0.6128	0.5561	0.5551	0.5233
	5	0.6453	0.576	0.5417	0.555	0.5148

Table 2: LH and RH generator temperature vs Split-Fashion-MNIST accuracy. Degraded performance observed for symmetrical temperatures.

		CCS - RH
		90%	70%	50%	30%	10%	0%	LSCC
CCS - LH	90%	0.7562	0.7662	0.7668	0.7668	0.7536	0.7505	0.7656
	70%	0.7618	0.7724	0.7753	0.7725	0.7607	0.7589	0.7796
	50%	0.7695	0.7706	0.7733	0.7667	0.7648	0.7573	0.7779
	30%	0.7703	0.7718	0.7758	0.7627	0.7565	0.7247	0.7799
	10%	0.7673	0.7704	0.7675	0.7509	0.7351	0.727	0.7757

Table 3: Comparison of mechanisms for selecting Fashion-MNIST memories for STM buffer storage. Storing latent space centroids via K-Means clustering (LSCC) produced the most consistent results on Split-Fashion-MNIST, whereas highly asymmetric confidence selection (

L=0.1/R=0.5

) (CCS) achieved peak performance on the more complex Split-CIFAR-100 benchmark.

Panel of Figure 7 — Figure 7: (a) Biasing class representation by recency; weighting training heavily toward older classes significantly boosts early-class accuracy with only minor performance degradation on recent classes. (b) Effect of STM memory buffer size on accuracy; gains from larger buffers plateau after 200 samples, while also diminishing the relative impact of the sleep phase.

Figure 8: Capacity scaling curves comparing our bilateral 4MAS architecture against the unilateral B-IR baseline across (a) Split-MNIST, (b) Split-Fashion-MNIST, and (c) Split-CIFAR-100. The plots show final accuracy as a function of total model parameters (in millions).

Figure 9: Ablation performance for Split-MNIST and Split-Fashion-MNIST. Configurations including the sleep phase consistently perform best on both datasets, where sleep informed hemispheric confidence levels lead to system accuracy exceeding individual hemisphere accuracies.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Explored Values
Model Capacity (Parameters)	248K, 477K, 736K, 970K, 4.9M, 9.9M, 11.8M, 16.0M, 26.3M, 39.2M, 54.5M, 72.4M, 92.7M
Short-Term Memory Size ( $N_{stm}$ )	50 per hemisphere (10 samples per class)
VAE Generator Dimension	140, 250, 360, 450, 1450, 2250, 2500, 3000, 4000, 5000, 6000, 7000, 8000
VAE Latent Space Dimension	16, 32, 64, 128, 256
Awake-Phase Learning Rate ( $LR_{awake}$ )	$2\times 10^{-4}$ (Adam)
Sleep-Phase Learning Rate ( $LR_{sleep}$ )	$2\times 10^{-5}$ (Adam)

Appendix

Table 4: Hyperparameter search space for Split-MNIST and Split-Fashion-MNIST.

Optuna Study Name	Trials	Best Class-IL (%)	Key Parameters and Configurations
fourmas_cifar100_v1	89	0.00	Default VAE, latent dim 64, LR 0.0014, SGD
fourmas_cifar100_v2	34	-	Task-IL scenario ( $28.82\%$ Task-IL), latent dim 128, buffer 2000, Adam
fourmas_cifar100_v3	10	4.11	Class-IL, buffer 5000, VAE temp 1.15 (LH) / 1.54 (RH)
fourmas_cifar100_v4	26	7.63	Class-IL, buffer 5000, SWS/REM epochs 75/50, LR 0.0022
fourmas_cifar100_multi_arch	4	3.77	Pre-trained tap layer 2 feature extractor, latent dim 64
fourmas_cifar100_scratch_sota	15	6.85	Scratch training (no ImageNet), buffer 10000, latent dim 128

Appendix

Table 5: Best configurations and parameters across Split-CIFAR-100 Optuna sweeps.

ID	Model Type	SupCon Weight	LH Freeze	BSS Active	Class-IL (%)	Task-IL (%)
1	vae	0.0	False	False	1.52	12.04
2	vae	0.0	False	True	1.02	9.61
3	vae	0.0	True	False	1.19	11.17
4	vae	0.0	True	True	1.13	9.26
5	vae	0.5	False	False	2.13	17.33
6	vae	0.5	False	True	2.01	14.90

Appendix

Table 6: Grid search performance over generative model types, supervised contrastive (SupCon) weights, left-hemisphere parameter freezing, and BSS active gating.

Selection Strategy (LH / RH)	Left Centroid ( $L$ )	Right Centroid ( $R$ )	Pre-training Tasks	Synaptic Intelligence (SI)	Class-IL Acc (%)
CCS (Symmetric Baseline)	0.5	0.5	1	No	21.38
CCS (Symmetric)	0.5	0.5	2	No	25.34
CCS (Symmetric)	0.5	0.5	3	No	29.18
CCS (Symmetric)	0.5	0.5	4	No	30.46
CCS (Symmetric)	0.5	0.5	5	No	29.01
CCS (Left-Skewed)	0.7	0.3	1	No	21.78

Appendix

Table 7: Split-CIFAR-100 performance across selection strategies (CCS, LSCC, Hybrid), pre-training task horizons, and local SI regularisation.

Model Size	Architecture	Total Parameters	Class-IL Acc (%)	Forgetting (%)
14M Scale	4MAS (Ours, Scaled-down)	14.1M	13.16	25.8
	B-IR (Standard Baseline)*	13.3M	21.0	61.1
35M Scale	4MAS (Ours, Scaled-down)	34.5M	18.41	25.8
	B-IR (Ours, Scaled-down)	35.8M	23.01	58.9
70M Scale	4MAS (Ours, Champion)	70.0M	$24.45\pm 0.32$	$25.8\pm 1.2$
	B-IR (Ours, Scaled)	62.7M	$26.27\pm 1.37$	$52.82\pm 1.97$

Appendix

Table 8: Split-CIFAR-100 Class-IL performance across parameter capacity scales.

Figure 10: Continual learning post-REM accuracy trajectories across class scale (10 to 100 classes) for top VAE configurations and Quality-Gated replay regimes, showing mean performance and standard deviation error bars across random seeds.

Online Continual Learning

Online continual learning adapts models to sequential data streams while preserving useful knowledge and limiting forgetting.

See all 106 papers

Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Tasks
Jun 7, 2026Anthony Bazhenov, Jean Erik Delanois, Giri P. KrishnanReplay-Based Continual Learning Memory Consolidation
Khoury College of Computer Sciences, Northeastern University, Boston, MA · Department of Medicine, University of California, San Diego, La Jolla, CA, USA · ARTISAN, Georgia Institute of Technology, Atlanta, GA USA
NeuroSynth: A Biologically Inspired Continual Reinforcement Learning Architecture for Mitigating Catastrophic Forgetting
Jul 24, 2026Yash KiniContinual Learning Continual RL
James Madison High School Vienna, Virginia, USA
Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
Jun 2, 2026Ali Behrouz, Farnoosh Hashemi, Adel Javanmard +1Continual Learning for LLMs Persistent Memory for Language Models
Google

Jun 7, 2026J/K move · Enter open · S save

Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Tasks

Anthony Bazhenov, Jean Erik Delanois, Giri P. Krishnan

Khoury College of Computer Sciences, Northeastern University, Boston, MA · Department of Medicine, University of California, San Diego, La Jolla, CA, USA · ARTISAN, Georgia Institute of Technology, Atlanta, GA USA

Replay-Based Continual Learning Memory Consolidation Catastrophic Forgetting

Open paper AskPDF arXiv

One of the critical limitations of artificial neural networks is their lack of ability to continually learn: training on new tasks often leads to interference and forgetting of the previous ones. While several algorithms have been proposed to protect old memories from interference, they are typically applied during or immediately after each new episode of training. In contrast, humans and animals can learn continuously, acquiring multiple new memories during active learning before consolidating all of them into long-term storage. Here we show that multiple new tasks can be trained sequentially before an unsupervised sleep-like replay phase is applied to partially restore performance across all previously learned tasks. Our study further suggests that task-specific information remains resilient to new training but decays gradually as network is trained on new tasks. These findings point to novel principles for developing a broad range of continual learning AI solutions.

Abstract

Online Continual Learning

Explore similar work

Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Tasks

NeuroSynth: A Biologically Inspired Continual Reinforcement Learning Architecture for Mitigating Catastrophic Forgetting

Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories