GAP-Prompt: Gated Adaptive Prompting for Efficient Continual Learning

Trung-Anh Dang; Duy-Cuong Bui; Ngoc-Son Vu; Christel Vrain; Vincent Nguyen

cs.LGAug 24, 2026

GAP-Prompt: Gated Adaptive Prompting for Efficient Continual Learning

Authors: Trung-Anh Dang, Duy-Cuong Bui, Ngoc-Son Vu, Christel Vrain, Vincent Nguyen

Organizations: LIFO UR 4022 Université d’Orléans, INSA CVL, Orléans, 45067, France · ETIS CY Cergy Paris University, ENSEA, CNRS, Cergy, 95000, France · LIST3N Université de Technologie de Troyes, Troyes, 10010, France

Abstract

Continual learning faces the persistent challenge of catastrophic forgetting, where sequential task updates degrade previously acquired knowledge. While prompt-based methods integrated with pre-trained models offer a compelling solution by freezing the backbone, they often rely on static, task-level prompting strategies that overlook fine-grained intra-task diversity. In this paper, we propose Gated Adaptive Prompting (GAP-Prompt), a novel method that introduces instance-level adaptability to the prompting process. GAP-Prompt consists of three synergistic modules: (1) instance-conditioned gating, which dynamically determines optimal prompt injection layers for each individual image; (2) dynamic knowledge fusion, which performs instance-aware aggregation of current and historical prompts, enabling knowledge integration across tasks; and (3) shared prompt distillation, which anchors foundational knowledge in early shared layers to mitigate forgetting. Extensive evaluations on CIFAR-100, ImageNet-R, and CUB-200 benchmarks demonstrate that GAP-Prompt consistently achieves state-of-the-art performance. Notably, on the fine-grained CUB-200 dataset, GAP-Prompt reaches 87.29% accuracy, approaching the joint training upper bound (88.00%) and outperforming existing methods by a significant margin.

Figures & tables

Figure 1: Conceptual comparison of different prompt gating mechanisms, i.e., determining where prompts are injected into multi-head self-attention (MSA) layers, used in existing prompt-based CL methods. (a) Static gating prepends prompts to a fixed, pre-defined subset of layers for all tasks and instances. (b) Task-conditioned gating learns a specific layer-wise topology per task but remains fixed for all instances within that task. (c) Instance-conditioned gating (ours) dynamically computes soft gate values for prompts at each layer for every input image via a learnable task-specific gating module. The varying intensity of each position illustrates our model’s ability to adaptively weight layer-wise prompts based on the fine-grained visual context of each instance.

Figure 2: Overall architecture of GAP-Prompt. It partitions prompts into shared and task-specific components and comprises three main modules. Shared prompt distillation - SPD (green boxes and $\mathcal{L}_{SPD}$ ) stabilizes foundational knowledge in the shared prompt space. For task-specific prompts, instance-conditioned gating - ICG (blue bars Soft Gates) generates image-specific soft weights that drive dynamic knowledge fusion - DKF (pink boxes), which combines current and historical prompts into a unified representation. In inference, the query $\mathbf{q}$ adaptively retrieves and aggregates prompt-gate pairs for each input.

Method	CIFAR-100		ImageNet-R		CUB-200
Method	Avg Acc( $\uparrow$ )	Forgetting( $\downarrow$ )	Avg Acc( $\uparrow$ )	Forgetting( $\downarrow$ )	Avg Acc( $\uparrow$ )	Forgetting( $\downarrow$ )
Joint training	91.38	–	82.06	–	88.00	–
LAE ( Gao et al., 2023 )	86.09 $\pm$ 0.55	6.22 $\pm$ 0.17	73.88 $\pm$ 0.43	5.08 $\pm$ 0.33	-	-
InfLoRA ( Liang and Li, 2024 )	86.91 $\pm$ 0.34	4.22 $\pm$ 0.37	76.13 $\pm$ 0.44	5.22 $\pm$ 0.48	-	-
EASE ( Zhou et al., 2024 )	87.28 $\pm$ 0.71	4.81 $\pm$ 0.42	75.82 $\pm$ 0.54	5.83 $\pm$ 0.37	-	-
L2P ( Wang et al., 2022b )	83.86 $\pm$ 0.28	7.35 $\pm$ 0.38	61.57 $\pm$ 0.66	9.73 $\pm$ 0.47	71.22 $\pm$ 0.54	10.68 $\pm$ 0.61

Table 1: Performance comparison with other methods on CIFAR-100, ImageNet-R, and CUB-200 under the 10-task class-IL setting. Best results are in bold .

Pre-trained model	Method	CIFAR-100	ImageNet-R
iBOT-1K	L2P	$75.57\pm 0.41$	$60.97\pm 0.26$
	DualPrompt	$76.63\pm 0.05$	$61.51\pm 1.05$
	CODA-Prompt	$79.11\pm 1.02$	$66.56\pm 0.68$
	RainBowPrompt	$76.20\pm 0.34$	$65.82\pm 0.52$
	GAP-Prompt (Ours)	$\mathbf{79.90\pm 0.57}$	$\mathbf{69.29\pm 0.32}$
DINO-1K	L2P	$70.65\pm 0.57$	$57.40\pm 0.23$

Table 2: Average accuracy on CIFAR-100 and ImageNet-R with self-supervised pre-trained models iBOT-1K and DINO-1K. Best results are in bold .

Figure 3: Comparison between our method (GAP-Prompt) and task-based adaptive prompting (RainbowPrompt ( Hong et al., 2025 ) ) on the dataset CUB-200, 10 tasks, CIL setting. (a) Accuracy after learning the first task; (b) Average accuracy (AA) after learning the fifth task; (c) AA after learning the final task; (d) Forgetting of the model after the final task.

Training Strategy	Avg Acc ( $\uparrow$ )	Forgetting ( $\downarrow$ )
Baseline	84.78 $\pm$ 0.80	4.56 $\pm$ 0.73
+ Instance-Conditioned Gating (ICG)	87.82 $\pm$ 0.37	4.05 $\pm$ 0.22
+ Dynamic Knowledge Fusion (DKF)	88.94 $\pm$ 0.46	3.73 $\pm$ 0.31
+ Shared Prompt Distillation (SPD)	89.24 $\pm$ 0.28	3.03 $\pm$ 0.44

Table 3: Component-wise ablation of GAP-Prompt on the CIFAR-100 dataset

Method	Avg Acc ( $\uparrow$ )	Forgetting ( $\downarrow$ )
Baseline ( fixed prompting )	84.78 $\pm$ 0.80	4.56 $\pm$ 0.73
Baseline + Task-based adaptive prompting (TAP)	86.62 $\pm$ 0.30	5.68 $\pm$ 0.26
Baseline + Instance-conditioned gating (ICG)	87.82 $\pm$ 0.37	4.05 $\pm$ 0.22
RainbowPrompt ( Hong et al., 2025 )	87.22 $\pm$ 0.53	4.81 $\pm$ 0.41
GAP -Prompt	89.24 $\pm$ 0.28	3.03 $\pm$ 0.44

Table 4: Comparison between ICG and TAP on CIFAR-100. Bold highlights the best results.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Description
$E$	Number of training epochs
$\tilde{\eta}$	Learning rate
$\lambda_{match}$	Coefficient of $\mathcal{L}_{match}$ in the final loss
$\lambda_{SPD}$	Coefficient of $\mathcal{L}_{SPD}$ in the final loss
$bsz$	Batch size
$N_{\mathbf{p}}$	Task-specific expert prompt length

Appendix

Table 5: Hyperparameters used in GAP-Prompt

Dataset	Hyperparameter
All	$N_{\mathbf{p}}:20$ , $N_{\mathbf{s}}:6$ , $\lambda_{match}:1.0$ ,
	$\tau_{start}:5.0$ , $\tau_{end}:0.1$ , $\zeta:0.1$
CIFAR-100	$\tilde{\eta}:0.005$ , $E:20$ , $bsz:128$ , $\lambda_{SPD}:0.1$
ImageNet-R	$\tilde{\eta}:0.003$ , $E:50$ , $bsz:64$ , $\lambda_{SPD}:0.01$
CUB-200	$\tilde{\eta}:0.005$ , $E:20$ , $bsz:128$ , $\lambda_{SPD}:0.1$

Appendix

Table 6: Selected hyperparameters in GAP-Prompt

Gating module	CIFAR-100		ImageNet-R		CUB-200
Gating module	Avg Acc( $\uparrow$ )	Forgetting( $\downarrow$ )	Avg Acc( $\uparrow$ )	Forgetting( $\downarrow$ )	Avg Acc( $\uparrow$ )	Forgetting( $\downarrow$ )
MLP	88.28 $\pm$ 0.20	3.69 $\pm$ 0.18	76.27 $\pm$ 0.36	4.35 $\pm$ 0.33	86.32 $\pm$ 0.36	4.10 $\pm$ 0.11
Linear	89.24 $\pm$ 0.28	3.03 $\pm$ 0.44	78.72 $\pm$ 0.35	3.12 $\pm$ 0.54	87.29 $\pm$ 0.45	3.68 $\pm$ 0.27

Appendix

Table 7: Comparison of gating module architectures in GAP-Prompt (Linear and MLP) on CIFAR-100, ImageNet-R, and CUB-200 under the 10-task class-IL setting. The best results are highlighted in bold .

Figure 4: Convergence and optimization stability of DKF. We visualize the training loss curves during task 2, task 5, and task 10 training on different datasets. The curves decrease smoothly without severe oscillations, suggesting that input-specific dynamic fusion does not destabilize optimization.

Aspect	CODA-Prompt ( Smith et al., 2023 )	RainbowPrompt ( Hong et al., 2025 )	GAP-Prompt
Weight/gate type	Input-conditioned component weight	Task-conditioned insertion gate	Instance-conditioned activation gate
Weight/gate semantics	Component composition coefficient	prompt insertion decision	prompt activation strength
Historical knowledge use	Frozen accumulated components	Evolved accumulated prompts	Selective current/history fusion
Inference behavior	Dense component assembly	Task-level evolved prompts	On-the-fly input-specific fusion
Sparsity	Dense weighting	Task-level selection	Thresholded sparse activation

Appendix

Table 8: Comparison of GAP-Prompt with CODA-Prompt and RainbowPrompt

Dataset	Task querying accuracy ( $\uparrow$ )	Avg Acc ( $\uparrow$ )
CIFAR-100	55.75	89.24
ImageNet-R	44.69	78.72
CUB-200	44.30	87.29

Appendix

Table 9: Task querying accuracy (%) of the task-key retrieval module in GAP-Prompt after learning the final task under the 10-task class-IL setting.

\uparrow

indicates that higher values are better

Dataset	Correct query (%, $\uparrow$ )	Wrong query (%, $\uparrow$ )	Avg Acc (%, $\uparrow$ )
CIFAR-100	94.35	81.91	89.24
ImageNet-R	90.65	68.99	78.72
CUB-200	92.75	82.57	87.29

Appendix

Table 10: Classification accuracy conditioned on task querying correctness. This analysis separates degradation due to incorrect task querying from degradation due to class-level forgetting or confusion

Dataset	$\Delta t=0$ ( $\uparrow$ )	$\Delta t>0$ ( $\uparrow$ )	$\Delta t<0$ ( $\uparrow$ )	Avg Acc ( $\uparrow$ )
CIFAR-100	94.35	82.90	83.50	89.24
ImageNet-R	90.65	71.41	66.38	78.72
CUB-200	92.75	84.32	81.20	87.29

Appendix

Table 11: Sensitivity to the direction of task querying errors.

\Delta t=0

indicates correct task querying,

\Delta t>0

indicates over-estimation, and

\Delta t<0

indicates under-estimation. All values are reported in percentage (%).

Dataset	Tasks learned	Avg candidate gates	Active gates	Active ratio
CIFAR-100	2	14.4	5.39	37.5%
CIFAR-100	5	33.1	17.03	51.5%
CIFAR-100	10	60.3	29.61	49.1%
ImageNet-R	2	14.2	9.94	70.2%
ImageNet-R	5	29.7	17.08	57.4%
ImageNet-R	10	49.9	19.18	38.5%

Appendix

Table 12: Average number of active task-layer gates after inference thresholding. Candidate and active gates are averaged over all test samples. Active gates and active ratio indicate the effective sparsity of DKF after thresholding

Dataset	Tasks learned	ViT only ( $\downarrow$ )	GAP-Prompt ( $\downarrow$ )	Overhead ( $\downarrow$ )	Overhead ratio ( $\downarrow$ )
CIFAR-100	2	6.62ms	19.87ms	13.25ms	200.0%
CIFAR-100	5	6.62ms	24.17ms	17.55ms	265.1%
CIFAR-100	10	6.62ms	30.77ms	24.15ms	364.9%
ImageNet-R	2	6.61ms	19.59ms	12.98ms	196.4%
ImageNet-R	5	6.62ms	23.71ms	17.09ms	258.1%
ImageNet-R	10	6.64ms	28.86ms	22.22ms	334.9%

Appendix

Table 13: Inference latency across task increments (ms/image). “Overhead” denotes the additional inference time over the frozen ViT backbone.

\downarrow

indicates that lower values are better.

Method	Extra stored params	Extra inference params	Candidate set at inference	Active prompts/layers	GFLOPs ( $\downarrow$ )
ViT only	0M	0M	–	–	33.70
DualPrompt	1.10M	1.10M	Retrieved task prompt	Fixed	67.44
CODA-Prompt	3.99M	3.99M	Accumulated components	Dense component assembly	67.44
RainbowPrompt	6.44M	1.85M	Evolved task prompts	Task-level gates	67.46
GAP-Prompt	1.36M	1.36M	$1:\hat{t}(\mathbf{x})$	Thresholded instance gates	67.39

Appendix

Table 14: Inference efficiency comparison with prompt-based CL methods on CIFAR-100 under the 10-task class-IL setting. GFLOPs are measured under the same hardware and batch size, and averaged over the test set for GAP-Prompt due to input-dependent routing and thresholded DKF.

Continual Learning

Continual Learning studies how models acquire knowledge from changing data streams while retaining or selectively revising what they learned before.

See all 294 papers

Learning Probabilistic Prompt for Continual Learning
Jul 6, 2026Hyekang Park, Sanghoon Lee, Geon Lee +2Class-Incremental Learning Prompt Learning
1Yonsei University, Seoul, South Korea · 2Samsung Electronics, Suwon, South Korea
Information-Theoretic Decoupled Prompt Tuning for Continual Learning
Aug 13, 2026Yunfei Zhang, Wen Wen, Tieliang Gong +1Continual Learning for VLMs Continual Learning for LLMs
National Engineering Lab for Big Data Analytics, School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an 710049, China
Residual SODAP: Residual Self-Organizing Domain-Adaptive Prompting with Structural Knowledge Preservation for Continual Learning
Mar 13, 2026Gyutae Oh, Jungwoo Bae, Jitae ShinContinual Learning Domain-Incremental Learning
Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea

Jul 6, 2026J/K move · Enter open · S save

Learning Probabilistic Prompt for Continual Learning

Hyekang Park, Sanghoon Lee, Geon Lee, Jongyoun Noh+1

1Yonsei University, Seoul, South Korea · 2Samsung Electronics, Suwon, South Korea

Class-Incremental Learning Prompt Learning Continual Learning Visual Prompt Learning

Open paper AskPDF arXiv

Continual learning aims to progressively learn from a sequence of tasks, each containing a disjoint subset of classes, while preserving previously learned knowledge. Prompt-based continual learning methods propose to learn a small set of parameters, i.e., prompts, by associating them with a query feature of an input image. These methods optimize the prompts, attempting to represent diverse patterns of images. However, we have observed that existing prompt-based methods suffer from a prompt collapse problem, that is, the prompts tend to be highly similar to each other, thereby failing to capture the diverse data distributions in continual learning scenarios. To address this issue, we propose in this paper a novel prompt-based continual learning framework that captures diverse patterns of images across a sequence of tasks. To this end, we model each prompt as a probabilistic distribution and construct a mixture of these distributions, from which we sample diverse prompts. This enables our model to effectively capture highly diverse image distributions in the continual learning process. We also present a distribution regularization loss to prevent abrupt changes in the prompt distributions throughout the training process. We show extensive experimental results for continual learning on standard benchmarks, including ImageNet-R, CIFAR-100, and CUB-200, demonstrating the effectiveness of our framework.

Abstract

Continual Learning

Explore similar work

Learning Probabilistic Prompt for Continual Learning

Information-Theoretic Decoupled Prompt Tuning for Continual Learning

Residual SODAP: Residual Self-Organizing Domain-Adaptive Prompting with Structural Knowledge Preservation for Continual Learning