Mitigating Performance Discrepancy in Cross-Domain 3D Class-Incremental Learning

Jinge Ma; Gautham Vinod; Bruce Coburn; Jui-Feng Chi; Siddeshwar Raghavan; Fengqing Zhu

cs.CVSep 4, 2026

Mitigating Performance Discrepancy in Cross-Domain 3D Class-Incremental Learning

Authors: Jinge Ma, Gautham Vinod, Bruce Coburn, Jui-Feng Chi, Siddeshwar Raghavan, Fengqing Zhu

Organizations: Purdue University

Abstract

3D perception plays a crucial role in real-world applications such as autonomous driving, robotics, and AR/VR. In practical scenarios, 3D perception models need to continually adapt to newly emerging 3D object categories, making class-incremental learning (CIL) particularly important. However, unlike 2D images, 3D point clouds are inherently heterogeneous: objects from the same class may not only come from the clean CAD domain, but also from RGB-D camera scans of varying quality, video reconstructions, or even corrupted observations. We discover that such heterogeneity introduces a new challenge beyond catastrophic forgetting: the degree of performance degradation can vary substantially across domains, a phenomenon we term performance discrepancy. To investigate this problem, we establish the Domain3D-CIL training and evaluation protocol, which contains point cloud categories from heterogeneous domains. We further adapt a wide range of mainstream CIL methods to the 3D modality. The results demonstrate that this performance discrepancy consistently appears across these baselines. To mitigate this issue, we introduce PolyMem, an exemplar-free approach that implicitly models rich high-order statistics of the feature distribution to enhance cross-domain robustness. Experiments demonstrate that our method effectively alleviates the performance discrepancy while improving the model's performance across domains. Code will be made publicly available upon acceptance.

Figures & tables

Panel of Figure 1 — Figure 1: Visualization of performance discrepancy in 3D CIL. (a) Illustrates the performance discrepancy of APER [ 64 ] between CAD and real -scanned point clouds. As tasks progress, the performance gap between the CAD and the real-world domain gradually widens. In contrast, on Task 1 or under the joint-training setting, the performance across both domains is quite comparable. This demonstrates that the primary cause of the performance discrepancy is not static disparities in learning difficulty or in sample counts across domains. (b) and (c) demonstrate the prevalence of performance discrepancy across various baselines and different mixed datasets.

Panel of Figure 2 — Figure 2: Visualization of Domain3D-CIL. (a) shows representative categories containing cross-domain samples from the mixed datasets. (b) shows the corresponding class distributions.

Panel of Figure 3 — Figure 3: Histograms of 1st to 4th-order statistics on ShapeNet( CAD )+OmniObject3D( Real ).

Method	ModelNet ( $D_{1}$ ) + ScanObjectNN ( $D_{2}$ )						ModelNet ( $D_{1}$ ) + CO3D ( $D_{2}$ )
	Last Acc.			Cum. Acc.			Last Acc.			Cum. Acc.
	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$
Joint-Training	$\overline{93.68}$	$\overline{91.48}$	$\overline{2.20}$	-	-	-	$\overline{92.88}$	$\overline{89.21}$	$\overline{3.67}$	-	-	-
L2P [ 45 ]	62.44	52.49	9.95	78.49	72.54	5.95	52.03	24.63	27.40	68.21	53.16	15.04
DualPrompt [ 44 ]	58.59	45.92	12.67	78.88	74.65	4.23	52.07	23.13	28.94	72.07	54.93	17.14
CODA-Prompt [ 36 ]	61.14	51.69	9.45	77.51	69.49	8.02	58.43	36.51	21.92	72.63	55.20	17.44

Table 1: Quantitative evaluation of PTM-based CIL baselines and PolyMem on Domain3D-CIL.

Panel of Figure 4 — Figure 4: Task-wise forgetting curves showing performance discrepancy between $D_{1}$ and $D_{2}$ .

Q\leftarrow Q+\sum_{(x_{i},y_{i})\in\mathcal{D}_{t}}F_{i}^{\prime}e_{y_{i}}^{\top},\qquad G\leftarrow G+\sum_{(x_{i},y_{i})\in\mathcal{D}_{t}}F_{i}^{\prime}{F_{i}^{\prime}}^{\top}.

Algorithm 1 PolyMem for PTM-based Class-Incremental Learning

ScanObjectNN	ModelNet40
bed	bed
box	glass_box
cabinet	wardrobe
chair	chair
desk	desk
display	monitor

Table 2: Semantic alignments to ModelNet40 for ModelNet40 + ScanObjectNN and ModelNet40 + CO3D.

OmniObject3D	ShapeNet55	OmniObject3D	ShapeNet55
bed	bed	bottle	bottle
bowl	bowl	cabinet	cabinet
chair	chair	clock	clock
cup	mug	guitar	guitar
hat	cap	helmet	helmet
keyboard	keyboard	knife	knife

Table 3: Full OmniObject3D

\rightarrow

ShapeNet55 name mapping used in ShapeNet + OmniObject3D.

Method	Last Acc.			Cum. Acc.
Method	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}^{\dagger}_{\mathrm{LA}}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}_{\mathrm{CA}}\!\downarrow$
Joint-Training	$\overline{93.68}$	$\overline{91.48}$	$\overline{0.00}$	-	-	-
L2P [ 45 ]	$62.44_{\pm 0.69}$	$52.49_{\pm 1.06}$	$7.75_{\pm 0.58}$	$78.49_{\pm 0.33}$	$72.54_{\pm 0.82}$	$5.95_{\pm 0.44}$
DualPrompt [ 44 ]	$58.59_{\pm 0.42}$	$45.92_{\pm 1.24}$	$10.47_{\pm 1.05}$	$78.88_{\pm 0.61}$	$74.65_{\pm 0.70}$	$4.23_{\pm 0.51}$
CODA-Prompt [ 36 ]	$61.14_{\pm 0.83}$	$51.69_{\pm 0.57}$	$7.25_{\pm 0.91}$	$77.51_{\pm 0.29}$	$69.49_{\pm 0.96}$	$8.02_{\pm 1.18}$
SLCA [ 59 ]	$85.05_{\pm 0.44}$	$72.96_{\pm 0.72}$	$9.89_{\pm 0.53}$	$92.13_{\pm 0.18}$	$84.41_{\pm 0.69}$	$7.72_{\pm 0.84}$

Table 4: Mean and standard deviation on ModelNet + ScanObjectNN.

Method	Last Acc.			Cum. Acc.
Method	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}^{\dagger}_{\mathrm{LA}}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}_{\mathrm{CA}}\!\downarrow$
Joint-Training	$\overline{92.88}$	$\overline{89.21}$	$\overline{0.00}$	-	-	-
L2P [ 45 ]	$52.03_{\pm 1.18}$	$24.63_{\pm 2.04}$	$23.73_{\pm 1.47}$	$68.21_{\pm 0.77}$	$53.16_{\pm 1.41}$	$15.04_{\pm 1.09}$
DualPrompt [ 44 ]	$52.07_{\pm 0.96}$	$23.13_{\pm 1.83}$	$25.27_{\pm 2.22}$	$72.07_{\pm 0.49}$	$54.93_{\pm 1.28}$	$17.14_{\pm 1.53}$
CODA-Prompt [ 36 ]	$58.43_{\pm 0.74}$	$36.51_{\pm 1.62}$	$18.25_{\pm 1.31}$	$72.63_{\pm 0.67}$	$55.20_{\pm 1.05}$	$17.44_{\pm 1.43}$
SLCA [ 59 ]	$80.88_{\pm 0.56}$	$58.41_{\pm 1.14}$	$18.80_{\pm 0.92}$	$89.81_{\pm 0.36}$	$77.23_{\pm 0.93}$	$12.59_{\pm 1.18}$

Table 5: Mean and standard deviation on ModelNet + CO3D.

Method	Last Acc.			Cum. Acc.
Method	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}^{\dagger}_{\mathrm{LA}}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}_{\mathrm{CA}}\!\downarrow$
Joint-Training	$\overline{88.47}$	$\overline{86.11}$	$\overline{0.00}$	-	-	-
L2P [ 45 ]	$57.08_{\pm 1.31}$	$34.03_{\pm 2.24}$	$20.69_{\pm 1.73}$	$75.58_{\pm 0.86}$	$50.31_{\pm 1.87}$	$25.27_{\pm 2.10}$
DualPrompt [ 44 ]	$59.61_{\pm 1.05}$	$35.42_{\pm 2.03}$	$21.83_{\pm 2.37}$	$79.54_{\pm 0.79}$	$51.67_{\pm 1.61}$	$27.87_{\pm 1.92}$
CODA-Prompt [ 36 ]	$65.96_{\pm 0.91}$	$44.44_{\pm 1.74}$	$19.16_{\pm 1.28}$	$82.71_{\pm 0.69}$	$61.87_{\pm 1.30}$	$20.84_{\pm 1.56}$
SLCA [ 59 ]	$79.03_{\pm 0.64}$	$59.03_{\pm 1.47}$	$17.64_{\pm 1.03}$	$91.19_{\pm 0.44}$	$71.64_{\pm 1.12}$	$19.55_{\pm 1.35}$

Table 6: Mean and standard deviation on ShapeNet + OmniObject3D.

Method	Last Acc.			Cum. Acc.
Method	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}^{\dagger}_{\mathrm{LA}}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}_{\mathrm{CA}}\!\downarrow$
Joint-Training	$\overline{87.42}$	$\overline{85.01}$	$\overline{0.00}$	-	-	-
L2P [ 45 ]	$61.96_{\pm 0.88}$	$50.93_{\pm 1.23}$	$8.62_{\pm 0.74}$	$80.22_{\pm 0.61}$	$72.06_{\pm 0.92}$	$8.16_{\pm 1.05}$
DualPrompt [ 44 ]	$62.69_{\pm 0.73}$	$52.37_{\pm 1.06}$	$7.91_{\pm 1.18}$	$77.15_{\pm 0.66}$	$68.33_{\pm 0.83}$	$8.82_{\pm 0.69}$
CODA-Prompt [ 36 ]	$69.28_{\pm 0.64}$	$56.98_{\pm 0.94}$	$9.89_{\pm 0.81}$	$84.06_{\pm 0.47}$	$74.72_{\pm 0.71}$	$9.34_{\pm 0.96}$
SLCA [ 59 ]	$78.20_{\pm 0.52}$	$63.35_{\pm 0.81}$	$12.44_{\pm 1.04}$	$91.18_{\pm 0.36}$	$82.00_{\pm 0.58}$	$9.18_{\pm 0.62}$

Table 7: Mean and standard deviation on ShapeNet + ShapeNet-C.

Panel of Figure 8 — Figure 8: Task-wise forgetting curves showing performance discrepancy between $D_{1}$ and $D_{2}$ .

Panel of Figure 9 — Figure 9: Overall comparison of PTM-based CIL methods on Domain3D-CIL. Left: average Last Accuracy over both domains and all four mixed datasets. Right: average Last Accuracy Gap over the four mixed datasets. Higher average Last Accuracy and lower Last Accuracy Gap indicate better overall performance and stronger cross-domain balance.

Method	Last Acc.			Cum. Acc.
Method	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$
Joint-Training	$\overline{92.88}$	$\overline{89.21}$	$\overline{3.67}$	-	-	-
L2P [ 45 ]	$64.29_{\pm 0.72}$	$38.15_{\pm 1.16}$	$26.14_{\pm 0.93}$	$74.44_{\pm 0.48}$	$54.53_{\pm 0.84}$	$19.91_{\pm 0.71}$
DualPrompt [ 44 ]	$70.06_{\pm 0.61}$	$43.06_{\pm 1.02}$	$27.00_{\pm 1.28}$	$79.49_{\pm 0.44}$	$58.61_{\pm 0.76}$	$20.88_{\pm 0.65}$
CODA-Prompt [ 36 ]	$62.15_{\pm 0.58}$	$54.82_{\pm 0.95}$	$7.33_{\pm 0.62}$	$70.77_{\pm 0.39}$	$65.23_{\pm 0.68}$	$5.54_{\pm 0.49}$
SLCA [ 59 ]	$82.54_{\pm 0.41}$	$69.89_{\pm 0.86}$	$12.65_{\pm 0.77}$	$87.47_{\pm 0.25}$	$74.25_{\pm 0.58}$	$13.23_{\pm 0.67}$

Table 8: Quantitative results on ModelNet + CO3D under the B20Inc4 task split.

Method	Last Acc.			Cum. Acc.
Method	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$
Joint-Training	$\overline{87.42}$	$\overline{85.01}$	$\overline{2.41}$	-	-	-
L2P [ 45 ]	$42.35_{\pm 1.42}$	$34.72_{\pm 1.85}$	$7.63_{\pm 1.22}$	$62.33_{\pm 0.94}$	$54.84_{\pm 1.31}$	$7.48_{\pm 0.88}$
DualPrompt [ 44 ]	$45.33_{\pm 1.26}$	$35.11_{\pm 1.72}$	$10.22_{\pm 1.85}$	$65.83_{\pm 0.82}$	$57.59_{\pm 1.18}$	$8.24_{\pm 1.27}$
CODA-Prompt [ 36 ]	$42.34_{\pm 1.18}$	$33.56_{\pm 1.58}$	$8.78_{\pm 1.06}$	$58.21_{\pm 0.91}$	$49.03_{\pm 1.09}$	$9.18_{\pm 1.21}$
SLCA [ 59 ]	$75.29_{\pm 0.74}$	$57.00_{\pm 1.28}$	$18.29_{\pm 1.52}$	$83.40_{\pm 0.55}$	$70.18_{\pm 0.96}$	$13.21_{\pm 0.83}$

Table 9: Quantitative results on ShapeNet + ShapeNet-C under the B7Inc2 task split.

Panel of Figure 10 — Figure 10: Cross-domain average accuracy under additional task splits. Each curve reports the average accuracy over the CAD domain and the corresponding reconstructed or corrupted domain at each incremental task. Left: ModelNet + CO3D with B20Inc4 and 6 tasks. Right: ShapeNet + ShapeNet-C with B7Inc2 and 25 tasks. PolyMem maintains stronger average accuracy across the incremental process, confirming its robustness to different task-split designs.

Method	Learnable Params (M)	Auxiliary Params (M)
L2P [ 45 ]	1.88	88.96
DualPrompt [ 44 ]	0.97	88.96
CODA-Prompt [ 36 ]	3.10	–
SLCA [ 59 ]	0.05	32.44
SimpleCIL [ 64 ]	0.05	–
MOS [ 37 ]	7.34	32.44

Table 10: Learnable parameters and auxiliary params of PTM-based CIL methods. The frozen pre-trained backbone is excluded. All values are reported in

M

, where

1M=10^{6}

scalar entries.

Method	ModelNet ( $D_{1}$ ) + ScanObjectNN ( $D_{2}$ )						ModelNet ( $D_{1}$ ) + CO3D ( $D_{2}$ )
	Last Acc.			Cum. Acc.			Last Acc.			Cum. Acc.
	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$
Point-BERT Backbone
Joint-Training	$\overline{91.45}$	$\overline{82.52}$	$\overline{8.93}$	-	-	-	$\overline{91.53}$	$\overline{84.76}$	$\overline{6.77}$	-	-	-
SimpleCIL [ 64 ]	90.64	37.97	52.67	94.82	44.33	50.49	89.26	25.94	63.32	93.88	33.31	60.57
RanPAC [ 21 ]	90.93	66.38	24.55	94.79	79.14	15.65	90.24	65.07	25.17	93.04	71.26	21.78

Table 11: Backbone generalization analysis on Domain3D-CIL. We evaluate Point-BERT and Point-MAE on ModelNet + ScanObjectNN and ModelNet + CO3D.

{\color[rgb]{0.1719,0.6289,0.1719}D_{1}}

denotes the CAD domain and

{\color[rgb]{0.8398,0.1523,0.1563}D_{2}}

denotes the real-scanned or reconstructed domain.

Method	ModelNet ( $D_{1}$ ) + ScanObjectNN ( $D_{2}$ )						ModelNet ( $D_{1}$ ) + CO3D ( $D_{2}$ )
	Last Acc.			Cum. Acc.			Last Acc.			Cum. Acc.
	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$
Joint-Training	$\overline{90.72}$	$\overline{86.51}$	$\overline{4.21}$	-	-	-	$\overline{91.21}$	$\overline{86.62}$	$\overline{4.59}$	-	-	-
ER	69.00	40.95	28.05	74.17	46.38	27.79	66.17	35.28	30.89	67.53	38.53	29.00
iCaRL [ 32 ]	72.20	39.88	32.32	79.29	53.63	25.66	65.15	39.41	25.74	72.66	48.99	23.67
BiC [ 50 ]	60.94	38.57	22.37	63.60	40.67	22.93	61.39	38.46	22.93	61.39	42.92	18.47

Table 12: Quantitative evaluation of classic CIL baselines on Domain3D-CIL. All methods use PointNet++ trained from scratch with a replay budget of 10 exemplars per class.

Class-Incremental Learning

Class-incremental learning studies how models can add new categories over time while retaining the ability to recognize previously learned classes.

See all 162 papers

Feb 23, 2026

CLIPoint3D: Language-Grounded Few-Shot Unsupervised 3D Point Cloud Domain Adaptation

Recent vision-language models (VLMs) such as CLIP demonstrate impressive cross-modal reasoning, extending beyond images to 3D perception. Yet, these models remain fragile under domain shifts, especially when adapting from synthetic to real-world point clouds. Conventional 3D domain adaptation approaches rely on heavy trainable encoders, yielding strong accuracy but at the cost of efficiency. We introduce CLIPoint3D, the first framework for few-shot unsupervised 3D point cloud domain adaptation built upon CLIP. Our approach projects 3D samples into multiple depth maps and exploits the frozen CLIP backbone, refined through a knowledge-driven prompt tuning scheme that integrates high-level language priors with geometric cues from a lightweight 3D encoder. To adapt task-specific features effectively, we apply parameter-efficient fine-tuning to CLIP's encoders and design an entropy-guided view sampling strategy for selecting confident projections. Furthermore, an optimal transport-based alignment loss and an uncertainty-aware prototype alignment loss collaboratively bridge source-target distribution gaps while maintaining class separability. Extensive experiments on PointDA-10 and GraspNetPC-10 benchmarks show that CLIPoint3D achieves consistent 3-16% accuracy gains over both CLIP-based and conventional encoder-based baselines. Project page: https://sarthakm320.github.io/CLIPoint3D.

Mainak Singha, Sarthak Mehrotra, Paolo Casari +3

University of Trento, Italy · MDSR Labs Adobe, India · IIT Bombay, India +1

Unsupervised Domain Adaptation Few-Shot Domain Adaptation VLM Adaptation 3D VLMs Prompt Tuning

Ask PDF

Jun 16, 2026

Domain Generalizable Adaptation of 3D Vision-Language Models via Regularized Fine-Tuning

Domain adaptation remains a central challenge in 3D vision, especially for multimodal foundation models that align 3D point clouds with visual and textual data. While these models demonstrate strong general capabilities, adapting them to downstream domains with limited data often leads to overfitting and catastrophic forgetting. To address this, we introduce ReFine3D, a regularized fine-tuning framework designed for domain-generalizable tuning of 3D large multimodal models (LMMs). ReFine3D combines selective layer tuning with two targeted regularization strategies: multi-view consistency across augmented point clouds and text diversity through synonym-based prompts generated by large language models. Additionally, we incorporate point-rendered vision supervision and a test-time augmentation mechanism with confidence-based aggregation to further enhance robustness. Extensive experiments across different 3D domain generalization benchmarks show that ReFine3D improves base-to-novel class generalization by 1.36%, cross-dataset transfer by 2.43%, robustness to corruption by 1.80%, and few-shot accuracy by up to 3.11%, outperforming prior state-of-the-art methods with minimal added computational overhead.

Sneha Paul, Zachary Patterson, Nizar Bouguila

Concordia University, Canada

Multimodal Robustness VLM Adaptation Domain Generalization 3D VLMs

Ask PDF

Mar 6, 2026

SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation

Incremental Few-Shot (IFS) segmentation aims to learn new categories over time from only a few annotations. Although widely studied in 2D, it remains underexplored for 3D point clouds. Existing methods suffer from catastrophic forgetting or fail to learn discriminative prototypes under sparse supervision, and often overlook a key cue: novel categories frequently appear as unlabelled background in base-training scenes. We introduce SCOPE (Scene-COntextualised Prototype Enrichment), a plug-and-play background-guided prototype enrichment framework that integrates with any prototype-based 3D segmentation method. After base training, a class-agnostic segmentation model extracts high-confidence pseudo-instances from background regions to build a prototype pool. When novel classes arrive with few labelled samples, relevant background prototypes are retrieved and fused with few-shot prototypes to form enriched representations without retraining the backbone or adding parameters. Experiments on ScanNet and S3DIS show that SCOPE achieves SOTA performance, improving novel-class IoU by up to 6.98% and 3.61%, and mean IoU by 2.25% and 1.70%, respectively, while maintaining low forgetting. Code is available https://github.com/Surrey-UP-Lab/SCOPE.

Vishal Thengane, Zhaochong An, Tianjin Huang +5

University of Surrey, UK · University of Wollongong, Australia · University of Copenhagen, Denmark +2

Few-Shot Semantic Segmentation 3D Semantic Segmentation Class-Incremental Learning Prototype Learning

Ask PDF

Abstract

Class-Incremental Learning

Explore similar work

CLIPoint3D: Language-Grounded Few-Shot Unsupervised 3D Point Cloud Domain Adaptation

Domain Generalizable Adaptation of 3D Vision-Language Models via Regularized Fine-Tuning

SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation