Mitigating Performance Discrepancy in Cross-Domain 3D Class-Incremental Learning

Jinge Ma; Gautham Vinod; Bruce Coburn; Jui-Feng Chi; Siddeshwar Raghavan; Fengqing Zhu

cs.CVSep 4, 2026

Mitigating Performance Discrepancy in Cross-Domain 3D Class-Incremental Learning

Authors: Jinge Ma, Gautham Vinod, Bruce Coburn, Jui-Feng Chi, Siddeshwar Raghavan, Fengqing Zhu

Organizations: Purdue University

Abstract

3D perception plays a crucial role in real-world applications such as autonomous driving, robotics, and AR/VR. In practical scenarios, 3D perception models need to continually adapt to newly emerging 3D object categories, making class-incremental learning (CIL) particularly important. However, unlike 2D images, 3D point clouds are inherently heterogeneous: objects from the same class may not only come from the clean CAD domain, but also from RGB-D camera scans of varying quality, video reconstructions, or even corrupted observations. We discover that such heterogeneity introduces a new challenge beyond catastrophic forgetting: the degree of performance degradation can vary substantially across domains, a phenomenon we term performance discrepancy. To investigate this problem, we establish the Domain3D-CIL training and evaluation protocol, which contains point cloud categories from heterogeneous domains. We further adapt a wide range of mainstream CIL methods to the 3D modality. The results demonstrate that this performance discrepancy consistently appears across these baselines. To mitigate this issue, we introduce PolyMem, an exemplar-free approach that implicitly models rich high-order statistics of the feature distribution to enhance cross-domain robustness. Experiments demonstrate that our method effectively alleviates the performance discrepancy while improving the model's performance across domains. Code will be made publicly available upon acceptance.

Figures & tables

Panel of Figure 1 — Figure 1: Visualization of performance discrepancy in 3D CIL. (a) Illustrates the performance discrepancy of APER [ 64 ] between CAD and real -scanned point clouds. As tasks progress, the performance gap between the CAD and the real-world domain gradually widens. In contrast, on Task 1 or under the joint-training setting, the performance across both domains is quite comparable. This demonstrates that the primary cause of the performance discrepancy is not static disparities in learning difficulty or in sample counts across domains. (b) and (c) demonstrate the prevalence of performance discrepancy across various baselines and different mixed datasets.

Panel of Figure 2 — Figure 2: Visualization of Domain3D-CIL. (a) shows representative categories containing cross-domain samples from the mixed datasets. (b) shows the corresponding class distributions.

Panel of Figure 3 — Figure 3: Histograms of 1st to 4th-order statistics on ShapeNet( CAD )+OmniObject3D( Real ).

Method	ModelNet ( $D_{1}$ ) + ScanObjectNN ( $D_{2}$ )						ModelNet ( $D_{1}$ ) + CO3D ( $D_{2}$ )
	Last Acc.			Cum. Acc.			Last Acc.			Cum. Acc.
	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$
Joint-Training	$\overline{93.68}$	$\overline{91.48}$	$\overline{2.20}$	-	-	-	$\overline{92.88}$	$\overline{89.21}$	$\overline{3.67}$	-	-	-
L2P [ 45 ]	62.44	52.49	9.95	78.49	72.54	5.95	52.03	24.63	27.40	68.21	53.16	15.04
DualPrompt [ 44 ]	58.59	45.92	12.67	78.88	74.65	4.23	52.07	23.13	28.94	72.07	54.93	17.14
CODA-Prompt [ 36 ]	61.14	51.69	9.45	77.51	69.49	8.02	58.43	36.51	21.92	72.63	55.20	17.44

Table 1: Quantitative evaluation of PTM-based CIL baselines and PolyMem on Domain3D-CIL.

Panel of Figure 4 — Figure 4: Task-wise forgetting curves showing performance discrepancy between $D_{1}$ and $D_{2}$ .

Q\leftarrow Q+\sum_{(x_{i},y_{i})\in\mathcal{D}_{t}}F_{i}^{\prime}e_{y_{i}}^{\top},\qquad G\leftarrow G+\sum_{(x_{i},y_{i})\in\mathcal{D}_{t}}F_{i}^{\prime}{F_{i}^{\prime}}^{\top}.

Algorithm 1 PolyMem for PTM-based Class-Incremental Learning

ScanObjectNN	ModelNet40
bed	bed
box	glass_box
cabinet	wardrobe
chair	chair
desk	desk
display	monitor

Table 2: Semantic alignments to ModelNet40 for ModelNet40 + ScanObjectNN and ModelNet40 + CO3D.

OmniObject3D	ShapeNet55	OmniObject3D	ShapeNet55
bed	bed	bottle	bottle
bowl	bowl	cabinet	cabinet
chair	chair	clock	clock
cup	mug	guitar	guitar
hat	cap	helmet	helmet
keyboard	keyboard	knife	knife

Table 3: Full OmniObject3D

\rightarrow

ShapeNet55 name mapping used in ShapeNet + OmniObject3D.

Method	Last Acc.			Cum. Acc.
Method	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}^{\dagger}_{\mathrm{LA}}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}_{\mathrm{CA}}\!\downarrow$
Joint-Training	$\overline{93.68}$	$\overline{91.48}$	$\overline{0.00}$	-	-	-
L2P [ 45 ]	$62.44_{\pm 0.69}$	$52.49_{\pm 1.06}$	$7.75_{\pm 0.58}$	$78.49_{\pm 0.33}$	$72.54_{\pm 0.82}$	$5.95_{\pm 0.44}$
DualPrompt [ 44 ]	$58.59_{\pm 0.42}$	$45.92_{\pm 1.24}$	$10.47_{\pm 1.05}$	$78.88_{\pm 0.61}$	$74.65_{\pm 0.70}$	$4.23_{\pm 0.51}$
CODA-Prompt [ 36 ]	$61.14_{\pm 0.83}$	$51.69_{\pm 0.57}$	$7.25_{\pm 0.91}$	$77.51_{\pm 0.29}$	$69.49_{\pm 0.96}$	$8.02_{\pm 1.18}$
SLCA [ 59 ]	$85.05_{\pm 0.44}$	$72.96_{\pm 0.72}$	$9.89_{\pm 0.53}$	$92.13_{\pm 0.18}$	$84.41_{\pm 0.69}$	$7.72_{\pm 0.84}$

Table 4: Mean and standard deviation on ModelNet + ScanObjectNN.

Method	Last Acc.			Cum. Acc.
Method	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}^{\dagger}_{\mathrm{LA}}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}_{\mathrm{CA}}\!\downarrow$
Joint-Training	$\overline{92.88}$	$\overline{89.21}$	$\overline{0.00}$	-	-	-
L2P [ 45 ]	$52.03_{\pm 1.18}$	$24.63_{\pm 2.04}$	$23.73_{\pm 1.47}$	$68.21_{\pm 0.77}$	$53.16_{\pm 1.41}$	$15.04_{\pm 1.09}$
DualPrompt [ 44 ]	$52.07_{\pm 0.96}$	$23.13_{\pm 1.83}$	$25.27_{\pm 2.22}$	$72.07_{\pm 0.49}$	$54.93_{\pm 1.28}$	$17.14_{\pm 1.53}$
CODA-Prompt [ 36 ]	$58.43_{\pm 0.74}$	$36.51_{\pm 1.62}$	$18.25_{\pm 1.31}$	$72.63_{\pm 0.67}$	$55.20_{\pm 1.05}$	$17.44_{\pm 1.43}$
SLCA [ 59 ]	$80.88_{\pm 0.56}$	$58.41_{\pm 1.14}$	$18.80_{\pm 0.92}$	$89.81_{\pm 0.36}$	$77.23_{\pm 0.93}$	$12.59_{\pm 1.18}$

Table 5: Mean and standard deviation on ModelNet + CO3D.

Method	Last Acc.			Cum. Acc.
Method	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}^{\dagger}_{\mathrm{LA}}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}_{\mathrm{CA}}\!\downarrow$
Joint-Training	$\overline{88.47}$	$\overline{86.11}$	$\overline{0.00}$	-	-	-
L2P [ 45 ]	$57.08_{\pm 1.31}$	$34.03_{\pm 2.24}$	$20.69_{\pm 1.73}$	$75.58_{\pm 0.86}$	$50.31_{\pm 1.87}$	$25.27_{\pm 2.10}$
DualPrompt [ 44 ]	$59.61_{\pm 1.05}$	$35.42_{\pm 2.03}$	$21.83_{\pm 2.37}$	$79.54_{\pm 0.79}$	$51.67_{\pm 1.61}$	$27.87_{\pm 1.92}$
CODA-Prompt [ 36 ]	$65.96_{\pm 0.91}$	$44.44_{\pm 1.74}$	$19.16_{\pm 1.28}$	$82.71_{\pm 0.69}$	$61.87_{\pm 1.30}$	$20.84_{\pm 1.56}$
SLCA [ 59 ]	$79.03_{\pm 0.64}$	$59.03_{\pm 1.47}$	$17.64_{\pm 1.03}$	$91.19_{\pm 0.44}$	$71.64_{\pm 1.12}$	$19.55_{\pm 1.35}$

Table 6: Mean and standard deviation on ShapeNet + OmniObject3D.

Method	Last Acc.			Cum. Acc.
Method	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}^{\dagger}_{\mathrm{LA}}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}_{\mathrm{CA}}\!\downarrow$
Joint-Training	$\overline{87.42}$	$\overline{85.01}$	$\overline{0.00}$	-	-	-
L2P [ 45 ]	$61.96_{\pm 0.88}$	$50.93_{\pm 1.23}$	$8.62_{\pm 0.74}$	$80.22_{\pm 0.61}$	$72.06_{\pm 0.92}$	$8.16_{\pm 1.05}$
DualPrompt [ 44 ]	$62.69_{\pm 0.73}$	$52.37_{\pm 1.06}$	$7.91_{\pm 1.18}$	$77.15_{\pm 0.66}$	$68.33_{\pm 0.83}$	$8.82_{\pm 0.69}$
CODA-Prompt [ 36 ]	$69.28_{\pm 0.64}$	$56.98_{\pm 0.94}$	$9.89_{\pm 0.81}$	$84.06_{\pm 0.47}$	$74.72_{\pm 0.71}$	$9.34_{\pm 0.96}$
SLCA [ 59 ]	$78.20_{\pm 0.52}$	$63.35_{\pm 0.81}$	$12.44_{\pm 1.04}$	$91.18_{\pm 0.36}$	$82.00_{\pm 0.58}$	$9.18_{\pm 0.62}$

Table 7: Mean and standard deviation on ShapeNet + ShapeNet-C.

Panel of Figure 8 — Figure 8: Task-wise forgetting curves showing performance discrepancy between $D_{1}$ and $D_{2}$ .

Panel of Figure 9 — Figure 9: Overall comparison of PTM-based CIL methods on Domain3D-CIL. Left: average Last Accuracy over both domains and all four mixed datasets. Right: average Last Accuracy Gap over the four mixed datasets. Higher average Last Accuracy and lower Last Accuracy Gap indicate better overall performance and stronger cross-domain balance.

Method	Last Acc.			Cum. Acc.
Method	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$
Joint-Training	$\overline{92.88}$	$\overline{89.21}$	$\overline{3.67}$	-	-	-
L2P [ 45 ]	$64.29_{\pm 0.72}$	$38.15_{\pm 1.16}$	$26.14_{\pm 0.93}$	$74.44_{\pm 0.48}$	$54.53_{\pm 0.84}$	$19.91_{\pm 0.71}$
DualPrompt [ 44 ]	$70.06_{\pm 0.61}$	$43.06_{\pm 1.02}$	$27.00_{\pm 1.28}$	$79.49_{\pm 0.44}$	$58.61_{\pm 0.76}$	$20.88_{\pm 0.65}$
CODA-Prompt [ 36 ]	$62.15_{\pm 0.58}$	$54.82_{\pm 0.95}$	$7.33_{\pm 0.62}$	$70.77_{\pm 0.39}$	$65.23_{\pm 0.68}$	$5.54_{\pm 0.49}$
SLCA [ 59 ]	$82.54_{\pm 0.41}$	$69.89_{\pm 0.86}$	$12.65_{\pm 0.77}$	$87.47_{\pm 0.25}$	$74.25_{\pm 0.58}$	$13.23_{\pm 0.67}$

Table 8: Quantitative results on ModelNet + CO3D under the B20Inc4 task split.

Method	Last Acc.			Cum. Acc.
Method	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$
Joint-Training	$\overline{87.42}$	$\overline{85.01}$	$\overline{2.41}$	-	-	-
L2P [ 45 ]	$42.35_{\pm 1.42}$	$34.72_{\pm 1.85}$	$7.63_{\pm 1.22}$	$62.33_{\pm 0.94}$	$54.84_{\pm 1.31}$	$7.48_{\pm 0.88}$
DualPrompt [ 44 ]	$45.33_{\pm 1.26}$	$35.11_{\pm 1.72}$	$10.22_{\pm 1.85}$	$65.83_{\pm 0.82}$	$57.59_{\pm 1.18}$	$8.24_{\pm 1.27}$
CODA-Prompt [ 36 ]	$42.34_{\pm 1.18}$	$33.56_{\pm 1.58}$	$8.78_{\pm 1.06}$	$58.21_{\pm 0.91}$	$49.03_{\pm 1.09}$	$9.18_{\pm 1.21}$
SLCA [ 59 ]	$75.29_{\pm 0.74}$	$57.00_{\pm 1.28}$	$18.29_{\pm 1.52}$	$83.40_{\pm 0.55}$	$70.18_{\pm 0.96}$	$13.21_{\pm 0.83}$

Table 9: Quantitative results on ShapeNet + ShapeNet-C under the B7Inc2 task split.

Panel of Figure 10 — Figure 10: Cross-domain average accuracy under additional task splits. Each curve reports the average accuracy over the CAD domain and the corresponding reconstructed or corrupted domain at each incremental task. Left: ModelNet + CO3D with B20Inc4 and 6 tasks. Right: ShapeNet + ShapeNet-C with B7Inc2 and 25 tasks. PolyMem maintains stronger average accuracy across the incremental process, confirming its robustness to different task-split designs.

Method	Learnable Params (M)	Auxiliary Params (M)
L2P [ 45 ]	1.88	88.96
DualPrompt [ 44 ]	0.97	88.96
CODA-Prompt [ 36 ]	3.10	–
SLCA [ 59 ]	0.05	32.44
SimpleCIL [ 64 ]	0.05	–
MOS [ 37 ]	7.34	32.44

Table 10: Learnable parameters and auxiliary params of PTM-based CIL methods. The frozen pre-trained backbone is excluded. All values are reported in

M

, where

1M=10^{6}

scalar entries.

Method	ModelNet ( $D_{1}$ ) + ScanObjectNN ( $D_{2}$ )						ModelNet ( $D_{1}$ ) + CO3D ( $D_{2}$ )
	Last Acc.			Cum. Acc.			Last Acc.			Cum. Acc.
	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$
Point-BERT Backbone
Joint-Training	$\overline{91.45}$	$\overline{82.52}$	$\overline{8.93}$	-	-	-	$\overline{91.53}$	$\overline{84.76}$	$\overline{6.77}$	-	-	-
SimpleCIL [ 64 ]	90.64	37.97	52.67	94.82	44.33	50.49	89.26	25.94	63.32	93.88	33.31	60.57
RanPAC [ 21 ]	90.93	66.38	24.55	94.79	79.14	15.65	90.24	65.07	25.17	93.04	71.26	21.78

Table 11: Backbone generalization analysis on Domain3D-CIL. We evaluate Point-BERT and Point-MAE on ModelNet + ScanObjectNN and ModelNet + CO3D.

{\color[rgb]{0.1719,0.6289,0.1719}D_{1}}

denotes the CAD domain and

{\color[rgb]{0.8398,0.1523,0.1563}D_{2}}

denotes the real-scanned or reconstructed domain.

Method	ModelNet ( $D_{1}$ ) + ScanObjectNN ( $D_{2}$ )						ModelNet ( $D_{1}$ ) + CO3D ( $D_{2}$ )
	Last Acc.			Cum. Acc.			Last Acc.			Cum. Acc.
	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$	$D_{1}$ $\!\uparrow$	$D_{2}$ $\!\uparrow$	$\mathrm{Gap}\!\downarrow$
Joint-Training	$\overline{90.72}$	$\overline{86.51}$	$\overline{4.21}$	-	-	-	$\overline{91.21}$	$\overline{86.62}$	$\overline{4.59}$	-	-	-
ER	69.00	40.95	28.05	74.17	46.38	27.79	66.17	35.28	30.89	67.53	38.53	29.00
iCaRL [ 32 ]	72.20	39.88	32.32	79.29	53.63	25.66	65.15	39.41	25.74	72.66	48.99	23.67
BiC [ 50 ]	60.94	38.57	22.37	63.60	40.67	22.93	61.39	38.46	22.93	61.39	42.92	18.47

Table 12: Quantitative evaluation of classic CIL baselines on Domain3D-CIL. All methods use PointNet++ trained from scratch with a replay budget of 10 exemplars per class.

Class-Incremental Learning

Class-incremental learning studies how models can add new categories over time while retaining the ability to recognize previously learned classes.

See all 162 papers

CLIPoint3D: Language-Grounded Few-Shot Unsupervised 3D Point Cloud Domain Adaptation
Feb 23, 2026Mainak Singha, Sarthak Mehrotra, Paolo Casari +3Unsupervised Domain Adaptation Few-Shot Domain Adaptation
University of Trento, Italy · MDSR Labs Adobe, India · IIT Bombay, India +1
Domain Generalizable Adaptation of 3D Vision-Language Models via Regularized Fine-Tuning
Jun 16, 2026Sneha Paul, Zachary Patterson, Nizar BouguilaMultimodal Robustness VLM Adaptation
Concordia University, Canada
SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation
Mar 6, 2026Vishal Thengane, Zhaochong An, Tianjin Huang +5Few-Shot Semantic Segmentation 3D Semantic Segmentation
University of Surrey, UK · University of Wollongong, Australia · University of Copenhagen, Denmark +2

Feb 23, 2026J/K move · Enter open · S save

CLIPoint3D: Language-Grounded Few-Shot Unsupervised 3D Point Cloud Domain Adaptation

Mainak Singha, Sarthak Mehrotra, Paolo Casari, Subhasis Chaudhuri+2

University of Trento, Italy · MDSR Labs Adobe, India · IIT Bombay, India+1

Unsupervised Domain Adaptation Few-Shot Domain Adaptation VLM Adaptation 3D VLMs Prompt Tuning

Open paper AskPDF arXiv

Recent vision-language models (VLMs) such as CLIP demonstrate impressive cross-modal reasoning, extending beyond images to 3D perception. Yet, these models remain fragile under domain shifts, especially when adapting from synthetic to real-world point clouds. Conventional 3D domain adaptation approaches rely on heavy trainable encoders, yielding strong accuracy but at the cost of efficiency. We introduce CLIPoint3D, the first framework for few-shot unsupervised 3D point cloud domain adaptation built upon CLIP. Our approach projects 3D samples into multiple depth maps and exploits the frozen CLIP backbone, refined through a knowledge-driven prompt tuning scheme that integrates high-level language priors with geometric cues from a lightweight 3D encoder. To adapt task-specific features effectively, we apply parameter-efficient fine-tuning to CLIP's encoders and design an entropy-guided view sampling strategy for selecting confident projections. Furthermore, an optimal transport-based alignment loss and an uncertainty-aware prototype alignment loss collaboratively bridge source-target distribution gaps while maintaining class separability. Extensive experiments on PointDA-10 and GraspNetPC-10 benchmarks show that CLIPoint3D achieves consistent 3-16% accuracy gains over both CLIP-based and conventional encoder-based baselines. Project page: https://sarthakm320.github.io/CLIPoint3D.

Abstract

Class-Incremental Learning

Explore similar work

CLIPoint3D: Language-Grounded Few-Shot Unsupervised 3D Point Cloud Domain Adaptation

Domain Generalizable Adaptation of 3D Vision-Language Models via Regularized Fine-Tuning

SCOPE: Scene-Contextualized Incremental Few-Shot 3D Segmentation