Few-Shot Incremental 3D Object Detection in Dynamic Indoor Environments

Yun Zhu; Jianjun Qian; Jian Yang; Jin Xie; Na Zhao

cs.CVApr 9, 2026

Few-Shot Incremental 3D Object Detection in Dynamic Indoor Environments

Authors: Yun Zhu, Jianjun Qian, Jian Yang, Jin Xie, Na Zhao

Organizations: Nanjing University of Science and Technology · Nanjing University · Singapore University of Technology and Design

Abstract

Incremental 3D object perception is a critical step toward embodied intelligence in dynamic indoor environments. However, existing incremental 3D detection methods rely on extensive annotations of novel classes for satisfactory performance. To address this limitation, we propose FI3Det, a Few-shot Incremental 3D Detection framework that enables efficient 3D perception with only a few novel samples by leveraging vision-language models (VLMs) to learn knowledge of unseen categories. FI3Det introduces a VLM-guided unknown object learning module in the base stage to enhance perception of unseen categories. Specifically, it employs VLMs to mine unknown objects and extract comprehensive representations, including 2D semantic features and class-agnostic 3D bounding boxes. To mitigate noise in these representations, a weighting mechanism is further designed to re-weight the contributions of point- and box-level features based on their spatial locations and feature consistency within each box. Moreover, FI3Det proposes a gated multimodal prototype imprinting module, where category prototypes are constructed from aligned 2D semantic and 3D geometric features to compute classification scores, which are then fused via a multimodal gating mechanism for novel object detection. As the first framework for few-shot incremental 3D object detection, we establish both batch and sequential evaluation settings on two datasets, ScanNet V2 and SUN RGB-D, where FI3Det achieves strong and consistent improvements over baseline methods. Code is available at https://github.com/zyrant/FI3Det.

Figures & tables

Figure 1: Comparison between incremental 3D object detection (In3Det) and our FI3Det setting. (a) In3Det methods assume abundant annotated samples for novel classes, limiting their applicability in dynamic embodied environments. (b) In contrast, FI3Det requires only a few annotated novel samples, enabling efficient and scalable incremental perception.

Figure 2: Correlation between base and novel category objects. In indoor 3D scenes, novel category objects tend to appear alongside base category objects.

Figure 3: Overview of our few-shot incremental 3D object detection model . The model consists of two parts: base training and incremental learning. In the base stage, we introduce a VLM-guided unknown object learning module that uses 2D VLMs to generate unknown objects, thereby improving the perception of unknown objects. In the incremental stage, we propose a gated multimodal prototype imprinting module that builds 2D semantic and 3D geometric prototypes for efficient adaptation to novel categories.

Figure 4: Visualization comparison of features. In (b), the baseline produces similar feature object across different objects and inconsistent features within the same object. In contrast, with the unknown object learning module, our method (c) generates more discriminative inter-object features and maintains intra-object consistency.

Methods	1-way 1-shot			1-way 5-shot			9-way 1-shot			9-way 5-shot
Methods	Base	Novel	All	Base	Novel	All	Base	Novel	All	Base	Novel	All
Baseline	71.47	-	-	71.47	-	-	72.77	-	-	72.77	-	-
Imprinting [ 33 ]	71.47	1.81	67.62	71.47	0.23	67.72	72.77	6.52	39.64	72.77	7.10	39.94
IL-DETR [ 11 ]	69.78	0.03	65.91	65.63	0.35	62.00	65.77	6.02	35.90	67.05	13.82	40.43
SDCOT++ [ 58 ]	67.75	0.05	63.99	62.12	0.09	58.68	35.87	1.35	18.61	28.30	7.77	18.03
AIC3DOD [ 5 ]	67.44	0.07	63.69	70.54	4.59	66.88	71.66	8.94	40.30	69.97	15.43	42.70

Table 1: Batch incremental 3D object detection performance on ScanNet V2 [ 9 ] . Results are reported under 1-way/9-way and 1-shot/5-shot settings. Bold indicates the best performance, and underline indicates the second best.

Methods	1-way 1-shot			1-way 5-shot			5-way 1-shot			5-way 5-shot
Methods	Base	Novel	All	Base	Novel	All	Base	Novel	All	Base	Novel	All
Baseline	62.37	-	-	62.37	-	-	61.58	-	-	61.58	-	-
Imprinting [ 33 ]	62.37	0.18	56.15	62.37	1.61	56.29	61.58	4.70	33.14	61.58	4.32	32.95
IL-DETR [ 11 ]	61.81	0.05	55.63	61.72	0.02	55.55	58.27	0.74	29.50	58.90	0.25	29.57
SDCOT++ [ 58 ]	54.11	0.09	48.71	51.19	0.11	46.08	48.95	0.90	24.93	46.67	1.10	23.88
AIC3DOD [ 5 ]	58.13	0.05	52.32	58.83	0.02	52.95	60.53	2.35	31.44	58.28	0.88	29.58

Table 2: Batch incremental 3D object detection performance on SUN RGB-D [ 37 ] . Results are reported under 1-way/5-way and 1-shot/5-shot settings. Bold indicates the best performance, and underline indicates the second best.

Method	ScanNet V2									SUN RGB-D
	Task1			Task2			Task3			Task1			Task2
	Base	Novel	All	Base	Novel	All	Base	Novel	All	Base	Novel	All	Base	Novel	All
Baseline	72.77	-	-	72.77	-	-	72.77	-	-	61.58	-	-	61.58	-	-
Imprinting [ 33 ]	72.75	3.08	55.35	72.76	9.20	47.34	72.76	7.47	40.12	62.23	4.37	40.53	61.62	5.85	33.74
IL-DETR [ 11 ]	62.53	6.43	48.51	36.18	16.64	28.36	14.88	14.04	14.46	58.53	0.07	36.61	53.50	0.40	26.95
SDCOT++ [ 58 ]	43.38	0.82	32.75	15.11	10.24	13.17	7.61	0.59	4.10	47.30	0.16	29.62	22.43	0.03	11.23

Table 3: Sequential incremental results on ScanNet V2 [ 9 ] and SUN RGB-D [ 37 ] . Results are reported on 9-way 5-shot and 5-way 5-shot settings, respectively. Bold indicates the best performance, and underline indicates the second best.

No.	UOM	UOW	GPI	Base	Novel	All
1				71.81	14.09	68.60
2	✓			72.73	25.43	70.10
3	✓	✓		72.83	32.46	70.61
4	✓		✓	72.73	28.94	70.30
5	✓	✓	✓	72.84	38.48	70.94

Table 4: Ablation study of key components including UOM, UOW, GPI on ScanNet V2.

Figure 5: Qualitative comparison on the ScanNet V2 [ 9 ] (upper) and SUN RGB-D [ 37 ] (below). The red dashed circles highlight novel object categories are“ window ” and “ toilet ”.

Figure 6: Ablation of different components in UOM and UOW.

No.	$\bm{\alpha}^{*}$	$\bm{\gamma}$	Base	Novel	All
1			72.83	32.46	70.61
2	✓		72.83	36.58	70.84
3		✓	72.87	34.68	70.73
4	✓	✓	72.84	38.48	70.94

Table 5: Ablation study of GPI components on ScanNet V2.

Figure 7: Performance of different hyper-parameters.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Method	1-way			9-way
Method	Base	Novel	All	Base	Novel	All
Baseline	71.47	-	-	72.77	-	-
Ours ∗ [ 57 ]	72.71	55.52	71.75	74.61	69.63	72.12
Ours ∗ +UOL	74.26	59.76	73.45	75.40	71.91	73.66

Appendix

Table 6: Batch incremental 3D object detection results with fully labeled novel objects on ScanNet V2. All models are based on the TR3D baseline. Results are reported under 1-way and 9-way configurations. “Base” denotes base classes, “Novel” denotes novel classes, and “All” indicates the overall mean [email protected].

Figure 8: Statistical analysis of the number of instances for each category in ScanNet V2 and SUN RGB-D.

Method	ScanNet V2						SUN RGB-D
	9-way 1-shot			9-way 5-shot			5-way 1-shot			5-way 5-shot
	Base	Novel	All	Base	Novel	All	Base	Novel	All	Base	Novel	All
Baseline	72.77	6.52	39.64	72.77	7.10	39.94	61.58	4.70	33.14	61.58	4.32	32.95
YOLO-World [ 4 ]	72.43	28.09	50.76	72.44	26.38	49.91	61.77	16.24	39.01	61.77	23.97	42.87
GroundDINO (ours) [ 22 ]	72.27	30.81	51.54	72.28	30.23	51.26	62.49	15.27	38.88	62.49	26.81	44.65

Appendix

Table 7: Batch few-shot incremental 3D object detection results with different Vision-Language Models (VLMs) on ScanNet V2 and SUN RGB-D. Results are reported under 9-way/5-way and 1-shot/5-shot configurations. “Base” denotes base classes, “Novel” denotes novel classes, and “All” indicates the overall mean [email protected].

Methods	ScanNet V2						SUN RGB-D
	9-way 1-shot			9-way 5-shot			5-way 1-shot			5-way 5-shot
	Base	Novel	All	Base	Novel	All	Base	Novel	All	Base	Novel	All
Imprinting [ 33 ]	66.84	4.33	38.90	66.84	10.94	38.90	66.69	0.86	33.77	66.69	0.62	31.10
IL-DETR [ 11 ]	61.36	6.50	33.93	57.64	20.12	38.88	65.65	0.09	32.87	63.36	0.19	31.77
SDCOT++ [ 58 ]	48.26	5.38	23.82	27.38	19.41	23.40	60.86	0.02	30.44	51.51	0.06	25.78
AIC3DOD [ 5 ]	66.82	5.14	35.98	66.97	10.09	38.53	66.72	0.05	33.39	66.51	0.07	33.29

Appendix

Table 8: Batch few-shot incremental 3D object detection performance on ScanNet V2 [ 9 ] and SUN RGB-D [ 37 ] . Results are reported under 9-way/5-way and 1-shot/5-shot configurations. “Base” denotes base classes, “Novel” denotes novel classes, and “All” indicates the overall mean [email protected]. The base and novel categories are divided according to the number of object instances in each class.

Figure 9: Qualitative comparison on the ScanNet V2 [ 37 ] . The red dashed circles highlight novel object categories “ sofa ”, “ refrigerator ”, and “ window ”.

Figure 10: Qualitative comparison on the SUN RGB-D [ 37 ] . The red dashed circles highlight novel object categories“ table ” and “ dresser ”.

3D Object Detection

3D object detection identifies objects and estimates their locations, sizes, and orientations from images, point clouds, radar, or fused sensors.

See all 181 papers

SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation
Mar 6, 2026Vishal Thengane, Zhaochong An, Tianjin Huang +5Few-Shot Semantic Segmentation 3D Semantic Segmentation
University of Surrey, UK · University of Wollongong, Australia · University of Copenhagen, Denmark +2
WildDet3D: Scaling Promptable 3D Detection in the Wild
Apr 9, 2026Weikai Huang, Jieyu Zhang, Sijun Li +153D Object Detection Monocular 3D Object Detection
Allen Institute for AI · University of Washington · Johns Hopkins University +2
Breaking the Model Forgetting Cycle in Long-Incremental 3D Object Detection
Jul 16, 2026Peisheng Qian, Jie Xu, Xulei Yang +1Class-Incremental Learning Continual Object Detection
Singapore University of Technology and Design · Institute for Infocomm Research, A*STAR, Singapore

Mar 6, 2026J/K move · Enter open · S save

SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation

Vishal Thengane, Zhaochong An, Tianjin Huang, Son Lam Phung+4

University of Surrey, UK · University of Wollongong, Australia · University of Copenhagen, Denmark+2

Few-Shot Semantic Segmentation 3D Semantic Segmentation Class-Incremental Learning Prototype Learning

Open paper AskPDF arXiv

Incremental Few-Shot (IFS) segmentation aims to learn new categories over time from only a few annotations. Although widely studied in 2D, it remains underexplored for 3D point clouds. Existing methods suffer from catastrophic forgetting or fail to learn discriminative prototypes under sparse supervision, and often overlook a key cue: novel categories frequently appear as unlabelled background in base-training scenes. We introduce SCOPE (Scene-COntextualised Prototype Enrichment), a plug-and-play background-guided prototype enrichment framework that integrates with any prototype-based 3D segmentation method. After base training, a class-agnostic segmentation model extracts high-confidence pseudo-instances from background regions to build a prototype pool. When novel classes arrive with few labelled samples, relevant background prototypes are retrieved and fused with few-shot prototypes to form enriched representations without retraining the backbone or adding parameters. Experiments on ScanNet and S3DIS show that SCOPE achieves SOTA performance, improving novel-class IoU by up to 6.98% and 3.61%, and mean IoU by 2.25% and 1.70%, respectively, while maintaining low forgetting. Code is available https://github.com/Surrey-UP-Lab/SCOPE.

Abstract

3D Object Detection

Explore similar work

SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation

WildDet3D: Scaling Promptable 3D Detection in the Wild

Breaking the Model Forgetting Cycle in Long-Incremental 3D Object Detection