Online Versatile Incremental Learning: Towards Class and Domain-Agnostic Adaptation at Any Time

Jae-Ho Lee; Min-Yeong Park; Jun-Yeong Moon; Jung Uk Kim; Gyeong-Moon Park

cs.CVSep 29, 2026

Online Versatile Incremental Learning: Towards Class and Domain-Agnostic Adaptation at Any Time

Authors: Jae-Ho Lee, Min-Yeong Park, Jun-Yeong Moon, Jung Uk Kim, Gyeong-Moon Park

Organizations: Korea University, Seoul, Republic of Korea · Kyung Hee University, Yongin, Republic of Korea

Abstract

Continual learning enables vision systems to adapt to ever-changing data distributions. Despite significant advances, existing approaches fail to capture continuous and concurrent shifts in classes and domains, a critical capability for real-world deployment. This work introduces Online VIL (Online Versatile Incremental Learning), a novel scenario where class concepts and visual domains evolve simultaneously online without explicit boundaries. To better adapt to the challenges of such dynamic environments that more closely resemble real-world conditions, we propose a novel framework TopFlow, Topology preservation with Flow matching representation that contains two complementary mechanisms: Domain-agnostic Flow Matching (DFM) and Global Topology Preservation (GTP). DFM guides the model to have domain-agnostic representations by integrating the geodesic flow kernel into contrastive learning. In contrast, GTP maintains the global structure of the feature space without explicitly storing past examples. Our extensive experiments demonstrate that TopFlow effectively addresses the limitations of existing methods within the Online VIL scenario, achieving state-of-the-art performance in challenging Online VIL. The proposed methods suggest potential directions for building continual learning systems in realistic dynamic environments. Our implementation code is available at https://github.com/KU-VGI/Online-VIL.

Figures & tables

Panel of Figure 1 — Figure 1 : Conceptual comparison of (a) Class Incremental Learning (CIL), (b) Domain Incremental Learning (DIL), (c) Versatile Incremental Learning (VIL), and (d) Online Versatile Incremental Learning (Online VIL, Ours). Red lines indicate the scopes observable at once, while black dashed lines depict transitions across time.

Panel of Figure 2 — Figure 2 : Analysis of layer-wise knowledge on pre-trained ViT for data with changing distributions (CORe50). (a) t-SNE visualization of features from certain layers of the pre-trained ViT. The color indicates the domain. (b) The t-SNE visualization of the last layer with class-wise color. (c) Accuracy of linear probing for class and domain classification from different layers.

Figure 3 : Architecture overview. TopFlow comprises two components: Domain-agnostic Flow Matching (DFM) and Global Topology Preservation (GTP). DFM promotes domain-agnostic representation accumulation, while GTP preserves the global feature topology.

Method	iDigits		CORe50		CLEAR100
Method	$A_{\text{AUC}}$	$A_{\text{Last}}$	$A_{\text{AUC}}$	$A_{\text{Last}}$	$A_{\text{AUC}}$	$A_{\text{Last}}$
Upper-bound	-	87.18 $\pm$ 0.13	-	91.66 $\pm$ 0.25	-	94.36 $\pm$ 0.28
Lower-bound	13.45 $\pm$ 0.62	12.71 $\pm$ 3.34	3.39 $\pm$ 0.10	3.24 $\pm$ 1.09	2.38 $\pm$ 0.32	2.66 $\pm$ 0.55
EWC [ 11 ]	20.07 $\pm$ 2.84	14.67 $\pm$ 3.61	18.60 $\pm$ 4.64	16.07 $\pm$ 1.56	23.61 $\pm$ 3.11	19.93 $\pm$ 1.33
LwF [ 16 ]	19.61 $\pm$ 3.50	15.38 $\pm$ 1.04	25.27 $\pm$ 3.77	21.97 $\pm$ 4.22	23.80 $\pm$ 2.24	21.70 $\pm$ 4.19
CODA-P [ 27 ]	23.96 $\pm$ 4.74	20.62 $\pm$ 3.47	54.06 $\pm$ 5.32	48.88 $\pm$ 2.92	28.82 $\pm$ 5.77	25.61 $\pm$ 3.52

Table 1 : Experimental results with proposed Online VIL scenarios. We used bold and underlined as brief indications of the best and the second best, respectively.

Buffer Size	Method	iDigits		CORe50		CLEAR100
Buffer Size	Method	$A_{\text{AUC}}$	$A_{\text{Last}}$	$A_{\text{AUC}}$	$A_{\text{Last}}$	$A_{\text{AUC}}$	$A_{\text{Last}}$
500	ER [ 25 ]	59.43 $\pm$ 6.24	48.70 $\pm$ 2.51	74.77 $\pm$ 4.85	72.25 $\pm$ 2.27	73.92 $\pm$ 3.93	71.49 $\pm$ 3.20
	RM [ 1 ]	55.02 $\pm$ 5.37	51.73 $\pm$ 2.05	81.06 $\pm$ 3.90	70.41 $\pm$ 3.17	72.42 $\pm$ 4.64	72.93 $\pm$ 2.05
	CLIB [ 12 ]	57.38 $\pm$ 4.16	52.63 $\pm$ 3.38	75.06 $\pm$ 5.81	71.93 $\pm$ 1.06	68.39 $\pm$ 5.25	66.92 $\pm$ 1.52
	OCM [ 9 ]	57.40 $\pm$ 3.60	52.88 $\pm$ 2.52	75.29 $\pm$ 3.10	72.66 $\pm$ 1.93	77.80 $\pm$ 3.25	75.10 $\pm$ 2.42
	CBA [ 33 ]	58.05 $\pm$ 4.39	54.28 $\pm$ 3.07	81.92 $\pm$ 4.04	81.02 $\pm$ 1.39	75.26 $\pm$ 4.08	74.47 $\pm$ 1.82

Table 2 : Results of OnlineVIL scenarios using replay buffer sizes 500 and 2000.

DFM	GTP	$A_{\text{AUC}}$	$A_{\text{Last}}$
Baseline		58.30	52.84
$\checkmark$		64.13	64.57
	$\checkmark$	63.95	65.28
$\checkmark$	$\checkmark$	64.51	66.20

Table 3 : Ablation study for DFM and GTP on CORe50.

Method	$A_{\text{Last}}$
CODA-P	49.13
+ DFM	52.88
+ GTP	53.17
+ DFM, GTP	54.62
PEC	45.99
+ DFM	46.64

Table 6 : Performance comparison of different methods and variants.

Method	iDigits		CORe50		DomainNet
Method	Avg. Acc $\uparrow$	Forgetting $\downarrow$	Avg. Acc $\uparrow$	Forgetting $\downarrow$	Avg. Acc $\uparrow$	Forgetting $\downarrow$
ICON	70.67 $\pm$ 2.13	13.49 $\pm$ 2.09	79.61 $\pm$ 1.73	8.19 $\pm$ 1.47	49.08 $\pm$ 2.11	17.88 $\pm$ 2.40
TopFlow (Ours)	73.33 $\pm$ 1.72	12.14 $\pm$ 4.20	79.77 $\pm$ 1.47	8.03 $\pm$ 1.19	49.62 $\pm$ 1.82	17.30 $\pm$ 3.09

Table 7 : The Effectiveness of TopFlow in Offline VIL Scenarios.

Figure 4 : t-SNE visualization of output features over tasks with GTP and without GTP.

Explore similar work

DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models
May 8, 2026Mengxin Qin, Xiang Zhang, Xi Wang +3Vision-Language Model Adaptation Class-Incremental Learning
School of Electronic Engineering, Xidian University Xi’an 710071, China
Continual Learning without Continual Training
Oct 7, 2026Nikita Narayanan, Ritham Majumdarr, Sonali ParbhooParameter-Efficient Adaptation Singular Learning Theory
Imperial College London
Remembering by Reconstructing: Domain Incremental Learning With Test-Time Training on Video Streams
May 29, 2026Jonathan Swinnen, Tinne TuytelaarsClass-Incremental Learning Streaming Video
ESAT, KU Leuven, Leuven, Belgium

May 8, 2026, cs.CVJ/K move · Enter open · S save

DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models

Mengxin Qin, Xiang Zhang, Xi Wang, Kun Wei+2

School of Electronic Engineering, Xidian University Xi’an 710071, China

Vision-Language Model Adaptation Class-Incremental Learning Continual Learning Plasticity

Open paper AskPDF arXiv

Continual learning enables vision-language models to accumulate knowledge and adapt to evolving tasks without retraining from scratch. However, in multi-domain task-incremental learning, large domain shifts intensify the stability-plasticity dilemma. Most existing methods rely on fixed architectures with statically allocated parameters, which limits adaptation to new domains and aggravates catastrophic forgetting. To address these challenges, we propose DIMoE-Adapters, a Dynamic Incremental Mixture-of-Experts Adapters framework that introduces a dynamic expert evolution paradigm to balance stability and plasticity. This paradigm is implemented through two collaborative components: Self-Calibrated Expert Evolution (SCEE) and Prototype-Guided Expert Selection (PGES). SCEE constructs and evolves a sparse expert pool through expert optimization dynamics, improving plasticity while reducing redundant capacity. PGES controls expert utilization based on the pool shaped by SCEE, improving stability across both previously encountered and unseen tasks. Extensive experiments show that DIMoE-Adapters outperforms previous state-of-the-art methods across various settings.

Abstract

Explore similar work

DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models

Continual Learning without Continual Training

Remembering by Reconstructing: Domain Incremental Learning With Test-Time Training on Video Streams