Authors: Andrea Cossu, Federico Giannini, Giacomo Ziffer, Alessio Bernardo, Alexander Gepperth, Emanuele Della Valle, Barbara Hammer, Davide Bacciu
Organizations: Computer Science Department, University of Pisa, Largo Bruno Pontecorvo, 3, Pisa, 56127, Italy · DEIB, Politecnico di Milano, Via Giuseppe Ponzio, 34, Milano, 20133, Italy · Motus ml, Milano, Italy · University of Applied Sciences Fulda, Germany · Bielefeld University, Germany
Continual Learning (CL) and Streaming Machine Learning (SML) study the ability of agents to learn from a stream of non-stationary data. Despite sharing some similarities, they address different and complementary challenges. While SML focuses on rapid adaptation after changes (concept drifts), CL aims to retain past knowledge when learning new tasks. After a brief introduction to CL and SML, we discuss Streaming Continual Learning (SCL), an emerging paradigm providing a unifying solution to real-world problems, which may require both SML and CL abilities. We claim that SCL can i) connect the CL and SML communities, motivating their work towards the same goal, and ii) foster the design of hybrid approaches that can quickly adapt to new information (as in SML) without forgetting previous knowledge (as in CL). We conclude the paper with a motivating example and a set of experiments, highlighting the need for SCL by showing how CL and SML alone struggle in achieving rapid adaptation and knowledge retention.
Figures & tables
Figure 1: Overview of the structure of this manuscript. We start with an introduction to non-stationary environments, followed by a short background on streaming and continual learning, before discussing the streaming continual learning paradigm together with an empirical example motivating why it is needed.
Figure 2: Illustrations of various concept drift types. The red line represents the theoretical decision boundary of the problem that the model must fit. 2(b) depicts a real drift where the decision boundary shifts. 2(c) represents a zoom-in virtual drift that concentrates on a subset of the initial feature space. 2(d) shows a pure expansionary virtual drift introducing a new, previously unseen feature subspace. 2(e) displays a drift involving both familiar and new subspaces without altering the original decision boundary. Finally, 2(f) illustrates a combination of virtual and real drift, resulting in changes to the decision boundary within the previously known subspace.
Figure 3: An RL control loop: as a reaction to actions a(t) taken by the agent, it receives observations o(t) and rewards r(t) .
Figure 4: Virtual and real drift in Continual Reinforcement Learning (CRL). In this example scenario, a robot is rewarded or punished when touching certain objects, depending on form, color, and displayed symbol. Left: an object that should be touched, giving +10 reward. Middle: virtual drift, introducing a different object that should not be touched (or else: +10 reward). Right: real drift, changing the reward previously obtained for touching the object to punishment.
Figure 5: Summary of SML and CL, with some examples of popular state-of-the-art approaches and popular libraries. We refer to existing reviews for an in-depth treatment of either field.
Figure 6: Illustration of digits 1 , 7 , 0 , and 8 using a seven-segment display representation. Each digit is encoded as a binary vector over seven dimensions (x1,…,x7) , where each xi corresponds to one segment (on = 1, off = 0). Although some digits share subsets of active segments (e.g., 1 and 7 ), their full encodings place them in distinct regions of the input space. This simplified scenario reflects the nature of virtual drifts in our experiments, where new digits introduce non-overlapping patterns in the input distribution. While MNIST uses grayscale pixel values in a high-dimensional space, the same principle holds: each digit activates a different subset of features, occupying a unique region in the space.
Figure 7: Prequential evaluation results averaged over the ten executions (considering Cohen’s Kappa score). In the virtual drift scenario ( 8(a) ), the Naive classifier performs well. ER and AGEM obtain slightly lower performance, since they balance between plasticity and stability. In the real drift scenarios ( 8(b) ), the Naive classifier and AGEM perform equally, since the AGEM strategy has no impact. ER has strong difficulties to learn the new concept since it tries to mitigate forgetting. ARF is plastic in both scenarios.
Figure 8: CL learning evaluation. After each drift, the models are tested over the current and all the previous test sets, and the average Cohen’s Kappa score is computed. Finally, results are averaged over the ten executions. On the virtual drift scenario ( 8(a) ), ER and AGEM are stable, while ARF and the Naive classifier catastrophically forget previous concepts. On the virtual drift scenario ( 8(b) ), all the models cannot avoid forgetting. This is due to the fact that the concepts are in contradiction, and the model cannot solve both of them jointly.
Virtual drift scenario
Real drift scenario
Kavg
BWT
Kavg
BWT
AGEM
0.89 ± 0.06
-0.13 ± 0.10
0.24 ± 0.01
-0.99 ± 0.05
ARF
0.40 ± 0.09
-0.83 ± 0.14
0.24 ± 0.00
-1.06 ± 0.00
ER
0.94 ± 0.02
-0.04 ± 0.02
0.27 ± 0.02
-0.85 ± 0.08
Naive
0.46 ± 0.09
-0.77 ± 0.14
0.24 ± 0.01
-1.00 ± 0.04
Table 1: Mean and standard deviation of the CL metrics over the ten executions using Cohen’s Kappa score.
Developing effective predictive models becomes challenging in dynamic environments that continuously produce data and constantly change. Continual Learning (CL) and Streaming Machine Learning (SML) are two research areas that tackle this arduous task. We put forward a unified setting that harnesses the benefits of both CL and SML: their ability to quickly adapt to non-stationary data streams without forgetting previous knowledge. We refer to this setting as Streaming Continual Learning (SCL). SCL does not replace either CL or SML. Instead, it extends the techniques and approaches considered by both fields. We start by briefly describing CL and SML and unifying the languages of the two frameworks. We then present the key features of SCL. We finally highlight the importance of bridging the two communities to advance the field of intelligent systems.
Federico Giannini, Giacomo Ziffer, Andrea Cossu +1
DEIB, Politecnico di Milano, Milano, 20133, Italy · Computer Science Department, University of Pisa, Pisa, Italy
Concept drift, temporal dependence, and catastrophic forgetting represent major challenges when learning from data streams. While Streaming Machine Learning and Continual Learning (CL) address these issues separately, recent efforts in Streaming Continual Learning (SCL) aim to unify them. In this work, we introduce MAGIC Net, a novel SCL approach that integrates CL-inspired architectural strategies with recurrent neural networks to tame temporal dependence. MAGIC Net continuously learns, looks back at past knowledge by applying learnable masks over frozen weights, and expands its architecture when necessary. It performs all operations online, ensuring inference availability at all times. Experiments on synthetic and real-world streams show that it improves adaptation to new concepts, limits memory usage, and mitigates forgetting.
Federico Giannini, Sandro D'Andrea, Emanuele Della Valle
The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting. This objective rests on a pervasive, often unstated assumption: that a lifelong learner should approximate the Joint-Task Learning (JTL) solution and retain all previously acquired knowledge. We challenge this retention-centered premise, arguing that in non-stationary environments prioritizing retention can impede real-time adaptation. Shifting the focus to the Average Lifelong Error (ALE), we formalize CL as an online optimization problem governed by the interaction between environmental and learning dynamics. We introduce Transfer Efficiency as a quantitative measure of the tension between Instability, the bias inherited from conflicting past experience, and Transient Error, the optimization cost of learning new tasks from scratch. Under mild convergence conditions, holding across linear and neural network models, this decomposition yields a Critical Task Duration: a closed-form threshold beyond which historical knowledge transitions from a warm-start advantage to an optimization liability whenever retention induces a positive stationary bias. We validate these theoretical predictions on continual image classification and reinforcement learning benchmarks. Finally, by connecting continual learning to the online learning framework of predictable sequences, we show that JTL is only one instance of a broader family of objectives, and we propose a new general class of continual learning algorithms, which we call Predictive Continual Learning. Predictive CL algorithms optimize expected future performance under an explicit, dynamically updated model of future tasks. As a proof of concept, we analyze a Window algorithm that interpolates between JTL and Independent-Task Learning (ITL), outperforming both under controlled distributional drift.
ETH AI Center, Department of Computer Science, ETH Zürich · Mila - Quebec AI Institute, School of Computer Science, McGill University · Mila - Quebec AI Institute +1