Residual SODAP: Residual Self-Organizing Domain-Adaptive Prompting with Structural Knowledge Preservation for Continual Learning

Gyutae Oh; Jungwoo Bae; Jitae Shin

cs.LGMar 13, 2026

Residual SODAP: Residual Self-Organizing Domain-Adaptive Prompting with Structural Knowledge Preservation for Continual Learning

Authors: Gyutae Oh, Jungwoo Bae, Jitae Shin

Organizations: Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea

Abstract

Continual learning (CL) suffers from catastrophic forgetting, which is exacerbated in domain-incremental learning (DIL) where task identifiers are unavailable and storing past data is infeasible. While prompt-based CL (PCL) adapts representations with a frozen backbone, we observe that prompt-only improvements are often insufficient due to suboptimal prompt selection and classifier-level instability under domain shifts. We propose Residual SODAP, which jointly performs prompt-based representation adaptation and classifier-level knowledge preservation. Our framework combines $α$ -entmax sparse prompt selection with residual aggregation, data-free distillation with pseudo-feature replay, prompt-usage--based drift detection, and uncertainty-aware multi-loss balancing. Across three DIL benchmarks without task IDs or extra data storage, Residual SODAP achieves state-of-the-art AvgACC/AvgF of 0.850/0.047 (DR), 0.760/0.031 (Skin Cancer), and 0.995/0.003 (CORe50).

Figures & tables

Figure 1: PCL exhibits classifier-level forgetting. We apply the cross-composition (backbone $\times$ classifier) diagnostic of [ 21 ] to a [ 14 ] -based baseline under the skin-cancer DIL setting. Residual SODAP mitigates performance degradation caused by classifier instability and improves overall accuracy. The experimental setup and interpretation are provided in Sup. 0.A – 0.B .

CL-Type [Ref.]	Method	DR		Skin Cancer		CORe50
		AvgACC $\uparrow$	AvgF $\downarrow$	AvgACC $\uparrow$	AvgF $\downarrow$	AvgACC $\uparrow$	AvgF $\downarrow$
PCL [ 14 ]	OS-Prompt	0.666 $\pm$ 0.062	0.132 $\pm$ 0.003	0.682 $\pm$ 0.041	0.135 $\pm$ 0.075	0.982 $\pm$ 0.004	0.012 $\pm$ 0.012
PCL [ 14 ]	OS-Prompt++	0.769 $\pm$ 0.027	0.113 $\pm$ 0.006	0.725 $\pm$ 0.032	0.063 $\pm$ 0.013	0.983 $\pm$ 0.001	0.014 $\pm$ 0.002
PCL [ 31 ]	Coda-Prompt	0.688 $\pm$ 0.079	0.140 $\pm$ 0.044	0.713 $\pm$ 0.040	0.041 $\pm$ 0.013	0.974 $\pm$ 0.041	0.056 $\pm$ 0.011
PCL [ 42 ]	Dual-prompt	0.467 $\pm$ 0.159	0.291 $\pm$ 0.065	0.637 $\pm$ 0.065	0.012 $\pm$ 0.039	0.975 $\pm$ 0.002	0.036 $\pm$ 0.001
Arch-CL [ 47 ]	MoE-Adapters	0.716 $\pm$ 0.141	0.080 $\pm$ 0.092	0.597 $\pm$ 0.165	0.040 $\pm$ 0.016	0.949 $\pm$ 0.041	0.043 $\pm$ 0.023

Table 1: Performance comparison with continual learning baselines on two medical imaging benchmarks: DR (APTOS [ 11 ]

\rightarrow

DDR [ 18 ]

\rightarrow

DRD [ 5 ] ) and Skin Cancer (ISIC [ 28 ]

\rightarrow

HAM [ 4 ]

\rightarrow

DERM7 [ 12 ] ), and a general continual learning benchmark CORe50 [ 22 ] . Best results are in bold .

Figure 3: Stage-wise prompt selection with PUDD-triggered pool expansion (60 $\rightarrow$ 84 $\rightarrow$ 94) and sparse $\alpha$ -entmax routing on the DR dataset. Corresponding results on the other datasets are provided in Sup. 0.E .

Figure 4: Prompt usage and optimization dynamics on the DR dataset: (a) balanced selection, (b) no redundancy after expansion, (c–d) uncertainty-weighted loss rebalancing across stages. Corresponding results for the other datasets are provided in Sup. 0.E .

Components

Metric

vs. Full (pp)

Method

SODAP

PUDD

Distill

Pseudo

\boldsymbol{\mathcal{L}_{div}}

\boldsymbol{\mathcal{L}_{norm}}

AvgACC

\uparrow

AvgF

\downarrow

\Delta

AvgACC

\uparrow

\Delta

AvgF

\downarrow

Full Model

✓

0.850

\pm

0.001

0.047

\pm

0.003

+0.0

(1) Prompting / drift handling

w/o Query Enhancer (QE)

✓

0.808

\pm

0.013

0.032

\pm

0.007

-4.2

-1.5

w/o PUDD

✓

0.841

\pm

0.004

0.028

\pm

0.009

-0.9

-1.9

w/o Residual SODAP (SODAP)

✓

0.831

\pm

0.004

0.027

\pm

0.003

-1.9

-2.0

Table 2: Sequential ablation on the DR continual learning benchmark (APTOS

\rightarrow

DDR

\rightarrow

DRD). We remove one component at a time from the full model.

\Delta

reports the absolute change from the full model in percentage points (pp):

\Delta\mathrm{AvgACC}=100\times(\mathrm{AvgACC}-0.850)

and

\Delta\mathrm{AvgF}=100\times(\mathrm{AvgF}-0.047)

(higher AvgACC and lower AvgF are better). ✓ indicates the component is enabled.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Normal (0)	NPDR (1)	PDR (2)
APTOS [ 11 ]	1805	1562	295
DDR [ 18 ]	6266	5343	913
DRD [ 5 ]	5000	8608	708

Appendix

Table 3: Dataset statistics for the DR benchmarks after remapping to three classes (Normal, NPDR, PDR).

Dataset	Benign	Malignant
ISIC [ 28 ]	584	584
HAM [ 4 ]	1113	1113
DERM7 [ 12 ]	252	252

Appendix

Table 4: Distribution of Skin Cancer datasets used for training

Class	Category	Stage 1	Stage 2	Stage 3	Stage 4	Stage 5	Stage 6	Stage 7	Stage 8	Stage 9	Stage 10	Stage 11
0	Plug adapters	1,064	1,066	1,066	1,064	1,066	1,064	1,066	1,064	1,066	1,066	1,066
1	Mobile phones	1,048	1,052	1,052	1,052	1,052	1,051	1,049	1,050	1,052	1,052	1,047
2	Scissors	1,070	1,067	1,067	1,069	1,064	1,070	1,070	1,066	1,067	1,067	1,070
3	Light bulbs	1,041	1,042	1,040	1,039	1,033	1,039	1,039	1,040	1,043	1,041	1,041
4	Cans	1,039	1,038	1,040	1,041	1,041	1,038	1,040	1,040	1,039	1,039	1,038
5	Glasses	1,064	1,059	1,065	1,065	1,058	1,064	1,065	1,062	1,066	1,065	1,064

Appendix

Table 5: Per-class training sample counts for each session in CORe50 [ 22 ] . Each session serves as one stage in our 11-stage domain-incremental continual learning setup (70/15/15 split).

Hardware
Category	Setting
GPU	2 $\times$ NVIDIA V100
RAM	256 GB
Optimization
Optimizer	AdamW
Learning rate	1e-3

Appendix

Table 6: Experimental Settings.

Method	Hyperparameter	Value
CODA-Prompt	Pool size ( coda_pool_size )	100
	Prompt length ( coda_prompt_length )	4
	Ortho. reg. ( coda_ortho_mu )	0.1
DualPrompt	Prompt length ( dual_prompt_length )	4
	Pool size ( dual_pool_size )	100
	Top- $k$ ( dual_top_k )	5

Appendix

Table 7: Key hyperparameters for continual learning baselines.

Figure 5: Results of cross-stage backbone–classifier compositions ( $3\times 3$ ) under the skin-cancer DIL protocol. The baseline visualization follows the prompt-based continual learning method of [ 14 ] .

Figure 6: Results of cross-stage backbone–classifier compositions ( $3\times 3$ ) under the skin-cancer DIL protocol. This figure visualizes our method using the same cross-composition protocol; the corresponding baseline visualization based on [ 14 ] is provided in Sup. Fig. 5 .

Figure 7: Residual SODAP prompt pool analysis: (a) stage-wise prompt count ( $60\rightarrow 84\rightarrow 94$ ), (b) prompt selection ratio on 100 random samples, and (c) prompt cosine similarity.

Figure 8: Supplementary analysis on the skin-cancer benchmark.

Figure 9: Supplementary analysis on the CORe50 benchmark.

Figure 10: Supplementary analysis on the CORe50 benchmark.

Continual Learning

Continual Learning studies how models acquire knowledge from changing data streams while retaining or selectively revising what they learned before.

See all 294 papers

Information-Theoretic Decoupled Prompt Tuning for Continual Learning
Aug 13, 2026Yunfei Zhang, Wen Wen, Tieliang Gong +1Continual Learning for VLMs Continual Learning for LLMs
National Engineering Lab for Big Data Analytics, School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an 710049, China
GAP-Prompt: Gated Adaptive Prompting for Efficient Continual Learning
Aug 24, 2026Trung-Anh Dang, Duy-Cuong Bui, Ngoc-Son Vu +2Continual Learning Prompt Learning
LIFO UR 4022 Université d’Orléans, INSA CVL, Orléans, 45067, France · ETIS CY Cergy Paris University, ENSEA, CNRS, Cergy, 95000, France · LIST3N Université de Technologie de Troyes, Troyes, 10010, France
Learning Probabilistic Prompt for Continual Learning
Jul 6, 2026Hyekang Park, Sanghoon Lee, Geon Lee +2Class-Incremental Learning Prompt Learning
1Yonsei University, Seoul, South Korea · 2Samsung Electronics, Suwon, South Korea

Aug 13, 2026J/K move · Enter open · S save

Information-Theoretic Decoupled Prompt Tuning for Continual Learning

Yunfei Zhang, Wen Wen, Tieliang Gong, Weizhan Zhang

National Engineering Lab for Big Data Analytics, School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an 710049, China

Continual Learning for VLMs Continual Learning for LLMs VLM Adaptation

Open paper AskPDF arXiv

Continual learning (CL) aims to incrementally acquire knowledge from sequential data while avoiding catastrophic forgetting. Recently, prompt tuning has attracted increasing attention as an efficient approach for adapting pre-trained models to CL tasks. However, existing prompt design paradigms commonly suffer from retrieval dependence and classifier bias, which make model adaptation sensitive to prompt selection and bias predictions toward newly arrived classes. To address these challenges, we propose Decoupled Prompt Tuning for Continual Learning (DPT4CL), which decouples the CLIP textual prompt into a task-shared prompt distribution and class-specific prompts. The task-shared prompt distribution is derived by optimizing an Information Bottleneck objective to facilitate cross-task knowledge transfer and alleviate classifier bias, while class-specific prompts enhance inter-class separability without relying on explicit prompt retrieval. Furthermore, we establish a unified excess risk bound from an information-theoretic perspective, providing theoretical support for the robust generalization and forgetting mitigation of the proposed framework. Extensive experiments on standard CL benchmarks demonstrate that DPT4CL achieves state-of-the-art performance. The source code is available at https://github.com/Cloudfly-Z/DPT4CL

Abstract

Continual Learning

Explore similar work

Information-Theoretic Decoupled Prompt Tuning for Continual Learning

GAP-Prompt: Gated Adaptive Prompting for Efficient Continual Learning

Learning Probabilistic Prompt for Continual Learning