Organizations: School of Data Science, Fudan University, 220 Handan Road, Shanghai, 200433, China · Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, 220 Handan Road, Shanghai, 200433, China
Lifelong learning enables artificial systems to adapt to changing environments. To achieve this, an effective artificial learner should dynamically reorganize past and new experiences, maintaining stability while enabling flexible knowledge integration. However, existing efforts primarily mitigate catastrophic forgetting, lacking the ability to adaptively update memory akin to biological systems. Here, by integrating the full memory life cycle of engram neurons into artificial learners, we propose a framework that constructs a structured memory from gradient representations, which regulates parameter updates and guides memory management. Through comprehensive evaluations on four benchmarks, our approach shows promising lifelong learning capability, maintaining stability over long task sequences and supporting flexible adaptation. Beyond the performance gains, our approach offers a computational perspective on robust and flexible lifelong learning, echoing mechanisms observed in biological memory systems.
Figures & tables
Figure 1: Illustration of the problem in lifelong learning (LL) and how biological intelligence solves this problem: (a) Biological memory adaptively balances old and new contexts through the lifecycle of engrams. Encoding involves CREB-regulated excitability that biases the allocation of neurons into sparse memory engrams. Synaptic plasticity underlies both consolidation and forgetting by stabilizing relevant connections and weakening or reorganizing others. Retrieval follows the encoding specificity principle, whereby cues overlapping with the original memory encoding trace preferentially reactivate the corresponding synaptic connection. CREB: cAMP response element-binding protein. (b) Plasticity dominated lifelong learning methods face severe catastrophic forgetting of previous contexts, whereas stability dominated learning restricts updating of new knowledge for new contexts. By contrast, stability-plasticity balanced LL methods adapt well to new knowledge and consolidate important prior knowledge.
Figure 2: Overview of the proposed AdaMLearn framework: (a) Overall workflow of the proposed AdaMLearn framework across the lifelong learning stream. (b) Diagram of how the model retrieves memory during training. The initial gradient obtained from backpropagation (treated as the cue) is orthogonally projected (matching) onto the old memory (treated as the trace). The residual gradient component lies in the new memory, enabling the model to learn new knowledge without compromising previously acquired knowledge. (c) Diagram of the encoding process. we design a core set optimization procedure. In this procedure, representative samples are chosen so that the loss landscape they produce closely matches that of the full dataset. This ensures that the most important memories of the context are preserved. (d) Illustration of the proposed process in gradient space. For input data from Context B, AdaMLearn projects features onto both the old (Context A) and new (Context B) gradient subspaces, applies SVD, then consolidates high-singular-value components and forgets low-singular-value ones.
Figure 3: Overall performance in visual classification tasks: (a) Lifelong learning performance is evaluated from multiple perspectives, including sensitivity to task ordering, cross-domain transferability, and robustness to higher-resolution inputs. Colors represent the category of different LL methods. All results are averaged over five runs with different random seeds. (b) The learner’s performance on all previously learned tasks at each stage of the full task sequence is reported, averaged over five runs. (c) Plasticity is assessed by measuring the test accuracy on a new task immediately after its initial learning, with experiments conducted on CIFAR-100. (d) Comparison with other replay-based lifelong learning methods shows that our approach maintains strong performance across a wide range of buffer size settings. Notably, even when the number of available samples is small, it does not suffer from catastrophic forgetting.
Figure 4: Achieving lifelong adaptive memorization within the framework of synaptic plasticity: (a) Integration of consolidation and forgetting within the neural synaptic plasticity. The color and intensity of nodes reflect the activation level of neurons. Learning context B introduces new potential synapses that partially weaken previously consolidated connections from context A. (b) Ablation experiments evaluate average accuracy and backward transfer. For AdaMLearn (M1), although synaptic remodeling increases the degree of forgetting, it leads to an overall improvement in average accuracy. All results are averaged over five runs. (c) Effects of consolidation and forgetting on memory efficiency. Without consolidation and forgetting (M4), memory grows rapidly and eventually saturates the budget. For AdaMLearn (M1), memory usage remains low and stable across tasks, adaptively converging to compact memory budgets that reflect efficient and context-aware resource allocation in CIFAR scenarios. The memory usage is computed at the level of the final linear hidden layer of the ANN. A value of 100% indicates that subsequent learning does not alter the parameters of this layer.
Figure 5: Implementing the selective encoding of contextual memory: (a) Effects of the core set selection procedure. Varying the fraction of core set selection demonstrates that our encoding process yields substantial benefits across a wide range of selection ratios. The baseline (Random) means selecting samples randomly. (b) The t-SNE plot of the CIFAR-100 dataset illustrates the representative samples chosen for each class after core set optimization. Samples selected as core set are marked with distinctive shapes, whereas the unselected ones appear as small circular points.
Figure 6: Applying retrieval during LL: (a) Effects of the retrieval process on preserving prior knowledge. Without any constraint (SGD), the model rapidly forgets previously acquired knowledge when learning a new task. When perturbations are introduced during retrieval (Disturb), causing the parameter update direction to deviate from being strictly orthogonal to the old memory, the stability degrades.
Layer
Channels
Kernel
Stride
Padding
Dropout
Input
3
-
-
-
-
Conv1 + BN
128
4×4
1
-
0.2
MaxPool1
128
2×2
2
-
-
Conv2 + BN
256
3×3
1
-
0.2
MaxPool2
256
2×2
2
-
-
Conv3 + BN
512
3×3
1
-
0.5
Supplementary Table 1: Neural network architecture (AlexNet) for CIFAR-100.
Layer
Channels
Kernel
Stride
Padding
Dropout
Input
3
-
-
-
-
Conv1 + ReLU + LRN
40
5×5
1
2
-
MaxPool1
40
3×3
2
1
-
Conv2 + ReLU + LRN
100
5×5
1
2
-
MaxPool2
100
3×3
2
1
-
Flatten
6400
-
-
-
-
Supplementary Table 2: Neural network architecture (LeNet) for CIFAR-superclass.
Layer
Channels
Kernel
Stride
Padding
Dropout
Conv1 + BN + ReLU
Cin→Cout
3×3
s
1
-
Conv2 + BN
Cout→Cout
3×3
1
1
-
Shortcut
Cin→Cout
1×1 (if needed)
s
0
-
ReLU (post-addition)
-
-
-
-
-
Supplementary Table 3: Structure of BasicBlock used in ResNet18.
Layer
Channels
Kernel
Stride
Padding
Dropout
Input
C
-
-
-
-
Conv1 + BN + ReLU
64
3×3
1
1
-
Stage 1: BasicBlock × 2
Block1
64
3×3
1
1
-
Block2
64
3×3
1
1
-
Stage 2: BasicBlock × 2
Supplementary Table 4: Neural network architecture (ResNet18(32×32)) for FIVE benchmark.
Layer
Channels
Kernel
Stride
Padding
Dropout
Input
3
-
-
-
-
Conv1 + BN + ReLU
64
3×3
1
1
-
Stage 1: BasicBlock × 2
Block1
64
3×3
1
1
-
Block2
64
3×3
1
1
-
Stage 2: BasicBlock × 2
Supplementary Table 5: Neural network architecture (ResNet18(84×84)) for Mini-imagenet.
Modern machine learning models are deployed in diverse, non-stationary environments where they must continually adapt to new tasks and evolving knowledge. Continual fine-tuning and in-context learning are costly and brittle, whereas neural memory methods promise lightweight updates with minimal forgetting. However, existing neural memory models typically assume a single fixed objective and homogeneous information streams, leaving users with no control over what the model remembers or ignores over time. To address this challenge, we propose a generalized neural memory system that performs flexible updates based on learning instructions specified in natural language. Our approach enables adaptive agents to learn selectively from heterogeneous information sources, supporting settings, such as healthcare and customer service, where fixed-objective memory updates are insufficient.
Large language model agents are expected to continuously adapt to new tasks and environments over their lifetime by reusing past experience. However, existing memory-based agents struggle to transfer reusable experience across environments and suffer from catastrophic forgetting as experience accumulated. To address these challenges, we propose LifeMem, a lifelong learning framework that enables agents to transfer knowledge across multiple environments. During learning, LifeMem clusters accumulated interaction trajectories based on underlying workflows to extract reusable skills. When solving a new task at inference time, the agent recalls relevant skills and trajectories to guide actions. To validate our method, we conduct experiments across 10 environments and over 13k tasks with 2k newly annotated interaction trajectories. Results show that LifeMem enables effective experience reuse in lifelong learning, achieving both reduced forgetting on learned tasks and superior cross-task transfer. Further analysis reveals that task streaming impacts learning, while consolidating structurally similar trajectories within memory boosts performance.
Yuli Qiu, Yutong Li, Wei Su +5
School of Computer Science and Technology, Beijing Institute of Technology · School of Computer Science and Engineering, Beihang University · Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology +1
Artificial Intelligence (AI) systems often perform well on isolated tasks but struggle under continual learning conditions, where training on new tasks can overwrite previously acquired knowledge, a failure mode known as catastrophic forgetting. Biological learning systems reduce this interference through complementary memory processes involving rapid hippocampal encoding and slower cortical consolidation. This study introduces NeuroSynth, a brain-inspired continual reinforcement learning architecture designed to mitigate catastrophic forgetting through a dual-pathway consolidation mechanism. NeuroSynth separates rapid task acquisition from long-term retention using distinct "plan" and "habit" pathways combined with replay and knowledge distillation. NeuroSynth was evaluated against Proximal Policy Optimization (PPO) and Elastic Weight Consolidation (EWC) across three sequential navigation tasks with changing goal locations in a non-revisitation continual learning setting. Across six independent seeds, NeuroSynth preserved substantially more early-task knowledge than PPO after sequential training, achieving 18.00% Task A success rate compared to 0.33% for PPO (p = 0.014929, Cohen's d = 1.49) and 35.33% Task B success rate compared to 0.00% for PPO (p = 0.002376, Cohen's d = 2.31). NeuroSynth also demonstrated higher final Task C performance than EWC, achieving 9.00% compared to 2.00% (p = 0.226643, Cohen's d = 0.56), indicating a moderate but not statistically significant advantage. These findings suggest that biologically inspired consolidation mechanisms may improve the stability-plasticity balance in continual reinforcement learning systems.