Beyond Prompt Degradation: Prototype-guided Dual-pool Prompting for Incremental Object Detection

Yaoteng Zhang; Zhou Qing; Junyu Gao; Qi Wang

cs.CVMar 2, 2026

Beyond Prompt Degradation: Prototype-guided Dual-pool Prompting for Incremental Object Detection

Authors: Yaoteng Zhang, Zhou Qing, Junyu Gao, Qi Wang

Organizations: Northwestern Polytechnical University

Abstract

Incremental Object Detection (IOD) aims to continuously learn new object categories without forgetting previously learned ones. Recently, prompt-based methods have gained popularity for their replay-free design and parameter efficiency. However, due to prompt coupling and prompt drift, these methods often suffer from prompt degradation during continual adaptation. To address these issues, we propose a novel prompt-decoupled framework called PDP. PDP innovatively designs a dual-pool prompt decoupling paradigm, which consists of a shared pool used to capture task-general knowledge for forward transfer, and a private pool used to learn task-specific discriminative features. This paradigm explicitly separates task-general and task-specific prompts, preventing interference between prompts and mitigating prompt coupling. In addition, to counteract prompt drift resulting from inconsistent supervision where old foreground objects are treated as background in subsequent tasks, PDP introduces a Prototypical Pseudo-Label Generation (PPG) module. PPG can dynamically update the class prototype space during training and use the class prototypes to further filter valuable pseudo-labels, maintaining supervisory signal consistency throughout the incremental process. PDP achieves state-of-the-art performance on MS-COCO (with a 9.2% AP improvement) and PASCAL VOC (with a 3.3% AP improvement) benchmarks, highlighting its potential in balancing stability and plasticity. The code and dataset are released at: https://github.com/zyt95579/PDP\_IOD/tree/main

Figures & tables

Figure 1: Comparison of prompt-based methods. (a) Use task ID to isolate prompt, which forces task-general prompt to be relearned at each task. (b) Missing annotations cause prompt tokens to drift. (c) The shared prompt pool continuously optimizes task-general prompt, and category prototypes guide the generation of pseudo labels for old categories.

Figure 2: Overview of our framework at incremental step $t$ . Given an image $x$ , the query function generates a content-aware query representation by adaptively computing query weights via a ranking function $F_{\psi}$ and performing weighted aggregation. Subsequently, prompts are retrieved from both the shared and private pool and injected into the decoder layer. In parallel, the teacher model $\Phi_{t-1}$ generates a set of candidate bounding boxes, where potentially valuable ones are projected into the feature space to compute their similarity with class prototypes. This process yields a set of refined, high-quality pseudo-labels to guide the training of the student model $\Phi_{t}$ .

Figure 3: The pseudo-label generation process of PPG at stage $t$ . PPG dynamically updates the prototypes of new task classes while keeping the old class prototypes frozen. The teacher model $\Phi_{t-1}$ produces a set of candidate detections, where high-confidence predictions are directly regarded as reliable samples. For low-confidence candidates, similarity matching with frozen old class prototypes is performed, and those exceeding a predefined threshold are also considered reliable. Finally, both types of samples are merged to generate high-quality pseudo-labels.

Method	Task 1	Task 2			Task 3			Task 4
Method	mAP@C	mAP@P	mAP@C	mAP@A	mAP@P	mAP@C	mAP@A	mAP@P	mAP@C	mAP@A
ORE-EBUI [ 17 ]	61.4	56.5	26.1	40.6	37.8	23.7	33.7	33.6	26.3	31.8
OW-DETR [ 11 ]	71.5	62.8	27.5	43.8	45.2	24.4	38.5	38.2	28.1	33.1
PROB [ 58 ]	73.4	66.3	36.0	50.4	47.8	30.4	42.0	42.6	31.7	39.9
CL-DETR [ 24 ]	–	–	–	–	–	–	–	–	–	39.2
ERD [ 7 ]	–	–	–	–	–	–	–	–	–	35.4

Table 1: Comparison with state-of-the-art methods on MS-COCO under the multi-step IOD setting, the second-best results are underlined .

Method	40+40			70+10
Method	AP	AP 50	AP 75	AP	AP 50	AP 75
ABR [ 27 ]	34.5	57.8	35.2	31.1	52.9	32.7
FasterILOD [ 31 ]	20.6	40.1	–	21.3	39.9	–
CL-DETR [ 24 ]	42.0	60.1	45.9	40.4	58.0	43.9
PseudoRM [ 48 ]	25.3	44.4	–	–	–	–
MMA [ 4 ]	33.0	56.6	34.6	30.2	52.1	31.5

Table 2: Performance comparison under the 40+40 and 70+10 IOD settings, the second-best results are underlined .

Method	10+10			15+5			19+1
Method	mAP@P	mAP@C	mAP@A	mAP@P	mAP@C	mAP@A	mAP@P	mAP@C	mAP@A
ILOD [ 35 ]	63.2	63.2	63.2	68.3	58.4	65.8	65.8	62.7	68.2
Faster ILOD [ 31 ]	69.8	54.5	64.2	71.6	56.9	67.9	68.9	61.1	68.5
ORE-EBUI [ 17 ]	60.4	68.8	64.7	71.8	58.7	68.5	69.4	61.0	68.8
OW-DETR [ 11 ]	63.5	67.9	65.7	72.2	58.9	69.4	70.2	62.0	70.4
PROB [ 58 ]	66.0	67.2	66.5	73.2	60.3	70.1	73.9	48.5	72.6

Table 3: Comparison of different methods on the PASCAL VOC dataset under three IOD settings (10+10, 15+5, and 19+1), the second-best results are underlined .

PP	SP	PPG	$L_{ddl}$	Task1	Task2			Task3			Task4
PP	SP	PPG	$L_{ddl}$	mAP@C	mAP@P	mAP@C	mAP@A	mAP@P	mAP@C	mAP@A	mAP@P	mAP@C	mAP@A
✓				78.6	66.4	57.1	60.8	52.7	57.3	53.5	46.0	52.5	46.0
✓	✓		✓	79.0	74.9	58.9	65.6	63.5	59.8	61.6	56.9	52.5	55.1
✓		✓		79.2	77.2	58.8	66.7	64.5	58.9	62.0	59.9	55.2	58.3
✓	✓	✓		78.9	77.2	59.5	67.0	65.2	60.0	62.7	61.0	55.3	59.0
✓	✓	✓	✓	79.1	77.2	59.9	67.4	65.7	60.5	63.2	61.3	55.8	59.4

Table 4: Ablation study of Private Pool, Share Pool, and PPG modules.

$\theta_{l}$	$\theta_{h}$	$\theta_{s}$	Task 1	Task 2			Task 3			Task 4
$\theta_{l}$	$\theta_{h}$	$\theta_{s}$	mAP@C	mAP@P	mAP@C	mAP@A	mAP@P	mAP@C	mAP@A	mAP@P	mAP@C	mAP@A
–	0.5	–	79.2	76.9	57.3	65.8	63.1	58.0	60.6	57.6	53.8	56.1
0.2	0.5	0.5	79.1	77.2	59.9	67.4	65.7	60.5	63.2	61.3	55.8	59.4
0.2	0.5	0.6	79.1	77.1	59.6	67.0	65.3	60.1	62.8	61.3	55.9	59.5
0.2	0.5	0.7	79.1	77.2	60.0	67.3	65.4	60.1	62.8	61.3	55.7	59.4

Table 5: The effect of different threshold parameters

\theta_{l}

\theta_{h}

, and

\theta_{s}

Figure 4: Ablation on shared and private pool sizes. Performance of PDP under different $(N_{s},N_{p})$ configurations on the COCO incremental detection benchmark, reported in $mAP@C$ , $mAP@P$ , and $mAP@A$ across sequential tasks.

Figure 5: Visualization of old-class detection results on the PASCAL VOC dataset.

Method	Params (M)	Training FLOPs (G)	Inference FLOPs (G)
MD-DETR	64.9	166.2 (12.3GB GPU)	166.17
PDP (Ours)	66.4	332.4 (13.4GB GPU)	166.18

Table 6: Model parameters and FLOPs comparison.

Method	Baseline	$AP$	$AP_{50}$
CL-DETR ${}_{\mathrm{CVPR\,^{\prime}23}}$	Deformable-DETR	35.3	–
SSDGR ${}_{\mathrm{CVPR\,^{\prime}24}}$	Deformable-DETR	41.1	59.5
DCA ${}_{\mathrm{AAAI\,^{\prime}25}}$	Deformable-DETR	40.3	54.1
PDP (Ours)	Deformable-DETR	42.1	60.7

Table 7: Results under the

40+20\times 2

setting on COCO.

Method	All Categories			Old Categories
Method	$AP$	$AP_{50}$	$AP_{75}$	$AP$
Fine-tune	4.2	–	–	0.7
Upper bound	46.2	65.2	50.0	47.4
PDP (Ours)	42.9	61.1	47.1	43.8

Table 8: Performance under the

70+10

setting on COCO.

Object Detection

Object detection locates and classifies objects in images or video, balancing localization accuracy, robustness, and inference speed.

See all 605 papers

Symbiosis-Inspired Knowledge Distillation for Incremental Object Detection
Jul 15, 2026Mingyue Zeng, De Cheng, Zhipeng Xu +3Class-Incremental Learning Object Detection
State Key Laboratory of Integrated Services Networks, School of Telecommunications Engineering, Xidian University, Xi’an, China · School of Electronic Engineering, Xidian University, Xi’an, China
Focus, Align, and Sustain: Counteracting Gradient Dilution in Incremental Object Detection
Jun 13, 2026Aoting Zhang, Dongbao Yang, Chang Liu +2Class-Incremental Learning Detection Transformer
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · Nankai University +2
C $^2$ Path: Class-Conditional Pathway Decoupling for Vision-Language Incremental Object Detection
Aug 22, 2026Lecheng Xu, Feifei Shao, Ouyangzi Ye +5Continual Object Detection Object Detection
Zhejiang University · The Hong Kong University of Science and Technology · Zhejiang Tobacco Monopoly Administration

Jul 15, 2026J/K move · Enter open · S save

Symbiosis-Inspired Knowledge Distillation for Incremental Object Detection

Mingyue Zeng, De Cheng, Zhipeng Xu, Huaijie Wang+2

State Key Laboratory of Integrated Services Networks, School of Telecommunications Engineering, Xidian University, Xi’an, China · School of Electronic Engineering, Xidian University, Xi’an, China

Class-Incremental Learning Object Detection Knowledge Distillation Catastrophic Forgetting

Open paper AskPDF arXiv

Incremental object detection (IOD) aims to extend detectors to new categories while retaining previously acquired knowledge. Existing methods often adopt a class incremental learning perspective, separating feature spaces to sharpen decision boundaries. However, this separation-oriented paradigm may overlook object symbiosis in detection, where co-occurrence and occlusion introduce spatial and semantic dependencies that benefit from shared representations. Ignoring these dependencies distorts the shared representations, exacerbates confusion between old and new classes, and accelerates catastrophic forgetting. To address this, we propose Symbiosis-Inspired Knowledge Distillation (SIKD), which explicitly leverages object symbiosis at two complementary levels. Spatial Symbiosis Distillation (SpSD) focuses on symbiotic regions where the old model responds with high overlap to objects in the new task. It preserves generalizable old class cues, suppresses class-specific bias and redundancy, and distills the refined evidence to the new model at matched spatial locations with slot-aligned supervision. Semantic Symbiosis Distillation (SeSD) maintains class level structure by forming confidence weighted prototypes for old classes and aligning their inter class soft ranks over the old class logits, which stabilizes the semantic topology during adaptation. Extensive experiments demonstrate the effectiveness and superiority of the proposed method.

Abstract

Object Detection

Explore similar work

Symbiosis-Inspired Knowledge Distillation for Incremental Object Detection

Focus, Align, and Sustain: Counteracting Gradient Dilution in Incremental Object Detection

C2^22Path: Class-Conditional Pathway Decoupling for Vision-Language Incremental Object Detection

C $^2$ Path: Class-Conditional Pathway Decoupling for Vision-Language Incremental Object Detection