cs.LGNov 7, 2026

STABLE: A Continual Learning Optimizer with Adaptive Drift Control

Authors: Xiaoyan Li, Lanpei Li, Massimo Coppola, Vincenzo Lomonaco, Antonio Carta

Organizations: Department of Computer Science University of Pisa · Department of AI, Data and Decision Sciences Luiss Guido Carli University · Institute of Information Science and Technologies “Alessandro Faedo” (ISTI) National Research Council of Italy (CNR)

Abstract

Continual learning suffers from catastrophic forgetting, leading to the loss of past knowledge and unstable adaptation across task streams. Existing approaches typically address this issue through replay buffers, architectural changes, or additional regularization terms. We argue that forgetting is a consequence of unstable learning dynamics, which may benefit from an optimizer-level solution. The main challenge is that learning dynamics are time-dependent, as demonstrated by empirical phenomena such as the stability gap. First, we formally prove that, for distance-based classifiers, bounded parameter drift guarantees bounded margin erosion under a local score-stability condition. Based on this insight, we propose STABLE, a primal-dual optimizer with adaptive drift control. STABLE enforces a drift budget by increasing the dual variable when the budget is violated and decreasing it otherwise. This results in high stability without penalizing plasticity. Experiments on long-horizon unimodal streams and CLIP-based multimodal streams show that vanilla PEFT baselines trained with STABLE match or outperform specialized continual learning methods. In the multimodal setting, STABLE further preserves zero-shot transfer, matching or exceeding dedicated multimodal continual learning methods. Code is available at https://github.com/atlas-luiss/STABLE.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Theoretical Foundations of Continual Learning via Drift-Plus-Penalty

    Jun 7, 2026Nazreen Shah, Govinda Arya, Bharath B. N. +1Continual LearningReplay-Based Continual Learning

  2. To Retain or to Adapt? Generalizing Continual Learning

    Jul 6, 2026Giulia Lanzillotta, Mandana Samiei, Doina Precup +2Continual LearningOnline Continual Learning

  3. LargeMonitor: Monitoring Online Task-Free Continual Learning via Large Pretrained Models

    Jun 8, 2026Mingqi Yuan, Xiaoquan Sun, Shihao Luo +1Distribution ShiftConcept Drift