Continual Learning
Overview
Continual Learning studies how models adapt to sequences of tasks or non-stationary data without losing useful earlier knowledge, a challenge known as catastrophic forgetting. Common approaches replay selected examples, regularize updates to protect important parameters, isolate tasks with adapters or separate subnetworks, or manage knowledge through prompts, memory, and model expansion. Methods are tested on class- and domain-incremental image benchmarks such as CIFAR and ImageNet, as well as task-free streams and applications in language, robotics, and healthcare. A central tension is balancing retention with the capacity to learn, while evaluations increasingly question assumptions such as known task boundaries and seek to measure transfer, recovery, resource use, and performance under realistic shifts.
Latest papers
323 papers26 papers in the last four weeks, down 24% on the four weeks before. 0.2% of all new papers.
Effect of Spherical Parameter Space on Number of Non-Forgetting Tasks and Capacity of Continual Learning
Engram Neuron Inspired Adaptive Memorization for Lifelong Learning
LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs
Continual Learning without Continual Training
MAGIC: Topology-Aware Analytic Graph Few-Shot Class-Incremental Learning
Gated Target Propagation for Compositional Generalization in Continual Learning
VoiceWeaver: Staged Learning of Structured Controls for Expressive Speech and Sound-Event Generation
PaLoRA: Paced Low-Rank Adaptation for Continual Learning
One-Step Curvature Probes Miss the Fitting Operator: Retained Capacity and Terminal Null-Space Correction for Continual Learning
What Should World Models Forget? Stratified Retention for Continual Adaptation
Task-Oriented Rank Adaptation for Continual Learning in Text Classification
MLoRA:Continual Low-Rank Adaptation via Relative-Response Subspace Selection and Momentum Orthogonalization
MLoRA, which jointly controls capacity allocation and utilization. First, historical whitening identifies input directions with strong current-task response relative to accumulated historical response, yielding a task-adaptive basis that remains fixed during training. Second, approximate polar orthogonalization of momentum updates reduces spectral concentration within theselected space. An orthonormal basis connects these mechanisms by transferring the factor-update spectrum exactly tothe induced weight update. We establish a max--min characterization of exact subspace selection and derive cumulative spectral bounds under controlled cross-step anisotropy. Across five class-incremental benchmarks and eight incremental settings, MuLoRA achieves the highest mean accuracy in 15 of 16 reported metrics.