CLIPoint3D: Language-Grounded Few-Shot Unsupervised 3D Point Cloud Domain Adaptation

Mainak Singha; Sarthak Mehrotra; Paolo Casari; Subhasis Chaudhuri; Elisa Ricci; Biplab Banerjee

cs.CVFeb 23, 2026

CLIPoint3D: Language-Grounded Few-Shot Unsupervised 3D Point Cloud Domain Adaptation

Authors: Mainak Singha, Sarthak Mehrotra, Paolo Casari, Subhasis Chaudhuri, Elisa Ricci, Biplab Banerjee

Organizations: University of Trento, Italy · MDSR Labs Adobe, India · IIT Bombay, India · Fondazione Bruno Kessler, Italy

Abstract

Recent vision-language models (VLMs) such as CLIP demonstrate impressive cross-modal reasoning, extending beyond images to 3D perception. Yet, these models remain fragile under domain shifts, especially when adapting from synthetic to real-world point clouds. Conventional 3D domain adaptation approaches rely on heavy trainable encoders, yielding strong accuracy but at the cost of efficiency. We introduce CLIPoint3D, the first framework for few-shot unsupervised 3D point cloud domain adaptation built upon CLIP. Our approach projects 3D samples into multiple depth maps and exploits the frozen CLIP backbone, refined through a knowledge-driven prompt tuning scheme that integrates high-level language priors with geometric cues from a lightweight 3D encoder. To adapt task-specific features effectively, we apply parameter-efficient fine-tuning to CLIP's encoders and design an entropy-guided view sampling strategy for selecting confident projections. Furthermore, an optimal transport-based alignment loss and an uncertainty-aware prototype alignment loss collaboratively bridge source-target distribution gaps while maintaining class separability. Extensive experiments on PointDA-10 and GraspNetPC-10 benchmarks show that CLIPoint3D achieves consistent 3-16% accuracy gains over both CLIP-based and conventional encoder-based baselines. Project page: https://sarthakm320.github.io/CLIPoint3D.

Figures & tables

Figure 1: Comparison of CLIPoint3D with SOTA methods on GraspNetPC-10. Encoder-based 3D UDA methods (e.g., PointDAN [ 45 ] , GAST [ 75 ] , MLSP [ 36 ] ) are accurate but computationally expensive, while CLIP-based extensions fail to bridge the synthetic-real gap. CLIPoint3D achieves +16.4% improvement with minimal overhead.

Figure 2: Overview of CLIPoint3D , the first CLIP-based unsupervised 3D point cloud domain adaptation framework, comprises four key modules: (1) Knowledge-driven prompt tuning generates LLM-guided textual and 3D-aware visual prompts; (2) Parameter-efficient fine-tuning (PEFT) jointly optimizes these prompts and the encoder while (3) entropy-based view selection filters unreliable projections; (4) Dual objectives, uncertainty-aware prototype loss $\mathbf{L}_{\mathrm{proto}}$ and optimal transport loss $\mathbf{L}_{\mathrm{OT}}$ , achieve joint semantic and statistical alignment. Additional regularizers include $\mathbf{L}_{\mathrm{conf}}=\mathbf{L}_{\mathrm{conf(S)}}+\mathbf{L}_{\mathrm{conf(T)}}$ , and $\mathbf{L}_{\mathrm{ortho}}=\mathbf{L}_{\mathrm{ortho(S)}}+\mathbf{L}_{\mathrm{ortho(T)}}$ , to ensure stable learning across source and target domains.

Methods	M $\rightarrow$ S	M $\rightarrow$ S ∗	S $\rightarrow$ M	S $\rightarrow$ S ∗	S ${}^{*}\rightarrow$ M	S ${}^{*}\rightarrow$ S	Avg
DANN [ 14 ]	74.8	42.1	57.5	50.9	43.7	71.6	56.8
PointDAN [ 45 ]	83.9	44.8	63.3	45.7	43.6	56.4	56.3
RS [ 51 ]	79.9	46.7	75.2	51.4	71.8	71.2	66.0
DAE-Global [ 20 ]	83.5	42.6	74.8	45.5	64.9	67.3	63.1
DefRec [ 2 ]	82.7	43.9	79.8	48.0	66.0	67.4	64.6
DefRec + PCM [ 2 ]	83.3	53.5	78.5	53.2	73.7	75.5	69.6

Table 1: Domain adaptation performance on the PointDA-10 benchmark. M: ModelNet, S: ShapeNet, S ∗ : ScanNet;

\rightarrow

indicates the adaptation direction. Best results and second-best results are reported in bold and underlined, respectively.

Methods	Syn. $\rightarrow$ Kin.	Syn $\rightarrow$ RS.	Kin. $\rightarrow$ RS.	RS. $\rightarrow$ Kin.	Avg
DANN [ 14 ]	78.6	70.3	46.1	67.9	65.7
PointDAN [ 45 ]	77.0	72.5	65.9	82.3	74.4
RS [ 51 ]	67.3	58.6	55.7	69.6	62.8
DefRec + PCM [ 2 ]	80.7	70.5	65.1	77.7	73.5
GAST [ 75 ]	69.8	61.3	58.7	70.6	65.1
GAI [ 53 ]	81.2	73.1	66.4	82.6	75.8

Table 2: Domain adaptation performance on the GraspNetPC-10 benchmark. Syn.: Synthetic domain, Kin.: Kinect domain, RS.: Realsense domain;

\rightarrow

indicates the adaptation direction.

	PEFT			PT	PEFT + PT
Method	Text	Vision	Both	-	Text	Vision	Both
LoRA [ 24 ]	80.3	89.2	90.5		78.9	90.7	92.2
LayerNorm [ 28 ]	75.4	84.0	76.6	73.7	75.0	79.5	78.8
BitFit [ 67 ]	80.1	87.8	81.1		79.7	88.5	81.5

Table 3: Ablation study of PEFT methods. Here, ‘PT’ refers to our proposed knowledge-driven prompt tuning strategy. The results reported are the average performances on GraspNetPC-10.

$\mathbf{L}_{\mathrm{ce}}$	$\mathbf{L}_{\mathrm{ortho}}$	$\mathbf{L}_{\mathrm{proto}}$	$\mathbf{L}_{\mathrm{OT}}$	$\mathbf{L}_{\mathrm{conf}}$	PointDA-10	GraspNetPC-10
✓	✗	✗	✗	✗	49.8	64.3
✓	✓	✗	✗	✗	58.6	74.9
✓	✓	✓	✗	✗	64.5	80.3
✓	✓	✗	✓	✗	70.4	85.0
✓	✓	✗	✗	✓	71.3	81.9
✓	✓	✓	✓	✗	71.0	85.8

Table 4: Ablation of different loss components. The results reported are the average performance for benchmarks.

Strategy	M $\rightarrow$ S	M $\rightarrow$ S ∗	S $\rightarrow$ M	S $\rightarrow$ S ∗	S ${}^{*}\rightarrow$ M	S ${}^{*}\rightarrow$ S	Avg
$\mathbf{P}_{t}(\mathbf{q})$ only	80.1	45.6	84.4	44.3	81.5	72.3	68.0
$\mathbf{P}_{v}(\mathbf{q})$ only	82.1	48.9	85.7	48.1	82.1	71.7	69.8
$\mathbf{P}_{t}(\mathbf{q})$ + $\mathbf{P}_{v}(\mathbf{q})$	85.4	50.8	90.8	48.4	87.5	71.3	72.4
$\mathbf{P}_{t}(\mathbf{q})$ + $\mathbf{P}_{v}(\mathbf{I}_{3D},\mathbf{q})$	83.2	49.5	91.2	48.9	83.9	71.1	71.3
$\mathbf{P}_{t}(\mathbf{T}^{\text{llm}},\mathbf{q})$ + $\mathbf{P}_{v}(\mathbf{q})$	81.4	52.3	91.2	49.2	84.5	89.2	74.6
$\mathbf{P}_{t}(\mathbf{T}^{\text{llm}},\mathbf{q})$ + $\mathbf{P}_{v}(\mathbf{I}_{3D},\mathbf{q})$	84.6	53.5	91.6	55.3	87.9	81.3	75.7

Table 5: Analysis of our prompting strategy in PointDA-10 benchmark. Explanations of the notations below are given in Section 3 . M: ModelNet, S: ShapeNet, S ∗ : ScanNet.

Panel of Figure 3 — Figure 3: (a) Effect of the number of labeled samples in $\mathcal{D}_{s}$ during training. (b) Effect of projected views. Accuracy variation with projection count $M$ .

Method	Avg.	W. Avg	Random	Max Sim.	Entropy-guided
CLIPoint3D-T	82.5	77.7	51.4	80.1	78.9
CLIPoint3D-V	91.5	86.5	71.5	86.3	90.7
CLIPoint3D-B	92.0	91.1	70.9	85.4	92.2

Table 6: Ablation of view selection strategies on GraspNetPC-10. ‘Avg.’, ‘W. avg.’, and ‘Max Sim’ denote uniform averaging, weighted averaging, and maximum-similarity-driven logit selection. ‘Random’ selects a random single view, while ‘Entropy-guided’ is our proposed uncertainty-based selection scheme.

Methods	DANN	PointDAN	DAE-Global	DefRec+PCM	GAST
Train params. (M)	2.50	11.26	12.60	2.50	161.09
PointDA-10 (%)	56.8	56.3	63.1	69.6	69.5
GraspNetPC-10 (%)	65.7	74.4	-	73.5	65.1
Methods	GAI	MLSP	Ours-T	Ours-V	Ours-B
Train params. (M)	22.02	28.0	9.24	9.83	11.00
PointDA-10 (%)	70.0	71.0	50.7	75.7	73.6

Table 7: Trade-off between computational complexity and model performances. Trainable parameters and accuracy are reported in millions (M) and %, respectively.

Figure 4: t-SNE visualization of CLIPoint3D’s performance. Alignment between synthetic and real domains post-adaptation. FD and MMD quantify domain gap reduction.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Domains	Common Classes	# Samples	# Training / Test
PointDA-10 [ 45 ]	ModelNet [ 63 ]	Bathtub, Bed, Bookshelf,	5039	4183 / 856
	ShapeNet [ 6 ]	Cabinet, Chair, Lamp,	19870	17378 / 2492
	ScanNet [ 8 ]	Monitor, Plant, Sofa, Table	7879	6110 / 1769
GraspNetPC-10 [ 11 , 53 ]	Synthetic	Banana, Box, Can,	12,000	12,000 / -
	Kinect	Camel, Dish, Drill, Mouse,	13,533	10,973 / 2560
	Realsense	Pear, Scissors, Shampoo	13,258	10,698 / 2560

Appendix

Table A1 : Domain Generation dataset statistical details on class, training and test splits, prefix template.

Figure A1 : LLM attributes generation. To derive high-level 3D knowledge representations, we follow a three-stage pipeline. First (top box), we provide an instructional query prompt to a LLM (e.g. GPT-5 [ 42 ] ). In response, the LLM produces detailed, geometry-aware visual descriptions (middle box). Finally (bottom box), we generate highly contextualized textual prompts (one caption per class) by combining a modality-specific prefix template with the LLM-generated attributes.

Panel of Figure 14 — Figure A2 : Domain Visualization. We show the diverse geometry variations across the domains of PointDA-10 and GraspNetPC-10 datasets.

Figure A3 : Effect of varying LoRA rank. We report the adaptation performances of CLIPoint3D-V and CLIPoint3D-B on PointDA-10 and GraspNetPC-10 datasets respectively.

Methods	M $\rightarrow$ S	M $\rightarrow$ S ∗	S $\rightarrow$ M	S $\rightarrow$ S ∗	S ${}^{*}\rightarrow$ M	S ${}^{*}\rightarrow$ S	Avg
ZS-CLIP [ 46 ]	46.1	17.0	52.0	17.0	52.0	46.1	38.4
CLIP [ 46 ] + DANN [ 14 ]	62.0	8.6	77.3	10.5	56.7	50.4	44.3
CLIP + CDAN [ 38 ]	60.9	7.0	76.5	11.0	56.7	50.0	43.7
CLIP + SCDA [ 32 ]	46.5	16.2	51.2	17.0	51.8	46.2	38.2
PointCLIP [ 71 ]	50.8	20.9	50.1	20.9	50.1	50.8	40.6
PointCLIP [ 71 ] + DANN	55.3	9.8	74.2	14.3	50.4	49.7	42.3

Appendix

Table A2 : Comparison of plug-in UDA methods in CLIP baselines with CLIPoint3D . We report the adaptation performances on the PointDA-10 benchmark. M: ModelNet, S: ShapeNet, S ∗ : ScanNet;

\rightarrow

indicates the adaptation direction. Best results and second-best results are reported in bold and underlined, respectively.

Figure A4 : Effect of varying

\alpha

hyperparameter. We report the adaptation performances of CLIPoint3D-V and CLIPoint3D-B on PointDA-10 and GraspNetPC-10 datasets respectively.

Methods	ViT-B/16	ViT-B/32	ViT-L/14
ModelNet $\rightarrow$ ShapeNet	84.6	82.4	85.7
ModelNet $\rightarrow$ ScanNet	53.5	42.7	54.4
ShapeNet $\rightarrow$ ModelNet	91.6	88.3	92.1
ShapeNet $\rightarrow$ ScanNet	55.3	36.9	59.5
ScanNet $\rightarrow$ ModelNet	87.9	88.2	88.5
ScanNet $\rightarrow$ ShapeNet	81.3	73.7	82.3

Appendix

Table A3 : Effect of using various CLIP variants on CLIPoint3D-V . We report the adaptation performances on the PointDA-10 benchmark.

Methods	Attributes Snippet	Syn. $\rightarrow$ Kin.	Syn $\rightarrow$ RS.	Kin. $\rightarrow$ RS.	RS. $\rightarrow$ Kin.	Avg
Handcrafted	“A point cloud object of”	93.5	81.6	77.9	94.8	87.0
Llama-3.2-3B [ 18 ]	“Compact, rectangular structure with a flat surface”	95.5	84.1	84.3	95.6	89.9
Qwen2.5-14B [ 66 ]	“Cylindrical main body with handle and trigger”	95.0	87.5	83.2	95.4	90.3
Phi-4 [ 1 ]	“Irregular mechanical shape”	95.8	83.2	79.6	92.3	87.7
GPT-5 [ 42 ]	“Elongated body with a cylindrical or pistol-like grip”	96.5	89.3	86.8	96.2	92.2

Appendix

Table A4 : Effect of using different LLMs on CLIPoint3D-B for attributes generation . We report the adaptation performances on the GraspNetPC-10 benchmark. and choose a snippet of attributes of class ‘drill machine’.

Figure A5 : Effect of varying length of

\mathbf{q}

. We report the adaptation performances of CLIPoint3D-V and CLIPoint3D-B on PointDA-10 and GraspNetPC-10 datasets respectively.

Unsupervised Domain Adaptation

Unsupervised domain adaptation transfers models from labeled source data to unlabeled target domains with different data distributions.

See all 179 papers

P3T: Prototypical Point-level Prompt Tuning with Enhanced Generalization for 3D Vision-Language Models
Apr 17, 2026Geunyoung Jung, Soohong Kim, Kyungwoo Song +1Point Cloud Classification VLM Adaptation
Department of Artificial Intelligence, University of Seoul, South Korea · Department of Applied Statistics, Yonsei University, South Korea
Domain Generalizable Adaptation of 3D Vision-Language Models via Regularized Fine-Tuning
Jun 16, 2026Sneha Paul, Zachary Patterson, Nizar BouguilaMultimodal Robustness VLM Adaptation
Concordia University, Canada
Refine Then Fusion: Training-Free 3D Point Cloud Adaptation with Priority Refinement and Multi-Modal Knowledge Fusion
Sep 18, 2026Hang Cheng, Yan Chen, Mingyu Fan +1Few-Shot Learning Point Cloud Learning
Tsinghua Shenzhen International Graduate School, Tsinghua University

Apr 17, 2026J/K move · Enter open · S save

P3T: Prototypical Point-level Prompt Tuning with Enhanced Generalization for 3D Vision-Language Models

Geunyoung Jung, Soohong Kim, Kyungwoo Song, Jiyoung Jung

Department of Artificial Intelligence, University of Seoul, South Korea · Department of Applied Statistics, Yonsei University, South Korea

Point Cloud Classification VLM Adaptation Domain Generalization Visual Prompt Learning 3D VLMs

Open paper AskPDF arXiv

With the rise of pre-trained models in the 3D point cloud domain for a wide range of real-world applications, adapting them to downstream tasks has become increasingly important. However, conventional full fine-tuning methods are computationally expensive and storage-intensive. Although prompt tuning has emerged as an efficient alternative, it often suffers from overfitting, thereby compromising generalization capability. To address this issue, we propose Prototypical Point-level Prompt Tuning (P

^3

T), a parameter-efficient prompt tuning method designed for pre-trained 3D vision-language models (VLMs). P

^3

T consists of two components: 1) Point Prompter, which generates instance-aware point-level prompts for the input point cloud, and 2) Text Prompter, which employs learnable prompts into the input text instead of hand-crafted ones. Since both prompters operate directly on input data, P

^3

T enables task-specific adaptation of 3D VLMs without sacrificing generalizability. Furthermore, to enhance embedding space alignment, which is key to fine-tuning 3D VLMs, we introduce a prototypical loss that reduces intra-category variance. Extensive experiments demonstrate that our method matches or outperforms full fine-tuning in classification and few-shot learning, and further exhibits robust generalization under data shift in the cross-dataset setting. The code is available at \textcolor{violet}{https://github.com/gyjung975/P3T}.

Abstract

Unsupervised Domain Adaptation

Explore similar work

P3T: Prototypical Point-level Prompt Tuning with Enhanced Generalization for 3D Vision-Language Models

Domain Generalizable Adaptation of 3D Vision-Language Models via Regularized Fine-Tuning

Refine Then Fusion: Training-Free 3D Point Cloud Adaptation with Priority Refinement and Multi-Modal Knowledge Fusion