SPACE: Semantic Projection and Alignment of CLIP Embeddings for Domain Adaptation

João Renato Ribeiro Manesco; Danilo Samuel Jodas; Douglas Rodrigues; Leandro Aparecido Passos; João Paulo Papa

cs.CVSep 19, 2026

SPACE: Semantic Projection and Alignment of CLIP Embeddings for Domain Adaptation

Authors: João Renato Ribeiro Manesco, Danilo Samuel Jodas, Douglas Rodrigues, Leandro Aparecido Passos, João Paulo Papa

Organizations: São Paulo State University (UNESP) School of Sciences, Bauru-SP, Brazil

Abstract

A fundamental challenge in deploying vision models is domain shift, which arises when training and test data follow different distributions, leading to degraded performance. This challenge is amplified when the same semantic concept appears under distinct visual forms, such as photographs and sketches, where visual similarity is weak despite semantic correspondence. Existing unsupervised domain-adaptation methods aim to align distributions across domains but often ignore semantic relationships among samples of the same class. To address this issue, this paper introduces SPACE, a method that exploits the semantic structure of CLIP's vision-language space for domain adaptation. The key idea is to use text descriptions as semantic anchors by applying Singular Value Decomposition to CLIP embeddings of class descriptions, yielding an orthogonal basis that captures semantic relationships among categories. Visual features from both domains are projected into this semantic subspace, aligning images based on meaning rather than appearance.

Figures & tables

Figure 1: Overview of the proposed CLIP-based domain alignment framework. Class prompts are encoded by the CLIP text encoder to construct an orthogonal semantic basis via SVD. Source and target images are encoded by the CLIP visual encoder and decomposed into semantic projections onto this basis and orthogonal residual components. Semantic projections are used to form class prototypes and to perform classification using source labels and target pseudo-labels, while residual features capturing domain-specific variation are aligned across domains using the MMD loss.

Method	Venue (Year)	CLIP-based	Ar $\to$ Cl	Ar $\to$ Pr	Ar $\to$ Rw	Cl $\to$ Ar	Cl $\to$ Pr	Cl $\to$ Rw	Pr $\to$ Ar	Pr $\to$ Cl	Pr $\to$ Rw	Rw $\to$ Ar	Rw $\to$ Cl	Rw $\to$ Pr	Avg.
ResNet-50 [ 3 ]	CVPR (2016)	✗	34.9	50.0	58.0	37.4	41.9	46.2	38.5	31.2	60.4	53.9	41.2	59.9	46.1
SHOT [ 7 ]	ICML (2020)	✗	57.1	78.1	81.5	68.0	78.2	78.1	67.4	54.9	82.2	73.3	58.8	84.3	71.8
FixBi [ 9 ]	CVPR (2021)	✗	58.1	77.3	80.4	67.7	79.5	78.1	65.8	57.9	81.7	76.4	62.9	86.7	72.7
VFR [ 4 ]	WACV (2024)	✓	58.1	85.0	84.5	77.4	85.0	84.7	76.5	58.8	85.7	75.9	60.4	86.4	76.5
PADCLIP [ 5 ]	ICCV (2023)	✓	57.5	84.0	83.8	77.8	85.5	84.7	76.3	59.2	85.4	78.1	60.2	86.7	76.6
SKD [ 17 ]	arXiv (2023)	✓	61.6	86.8	86.7	78.0	87.4	86.8	77.3	61.0	87.1	79.6	64.1	88.9	78.8

Table 1: Classification accuracy (%) on the Office-Home dataset on methods that use the ResNet-50 backbone. SPACE (Ours) is shown in the final block. The best result for each task among all ResNet-50 methods (top block and ours) is in bold.

Method	A $\to$ W	D $\to$ W	W $\to$ D	A $\to$ D	D $\to$ A	W $\to$ A	Avg.
ResNet-50 [ 3 ]	68.4	96.7	99.3	68.9	62.5	60.7	76.1
SHOT [ 7 ]	90.1	98.4	99.9	94.0	74.7	74.3	88.6
CDAN+TN [ 16 ]	95.7	98.7	100	94.0	73.4	74.2	89.3
MDD+SCDA [ 6 ]	95.3	99.0	100	95.4	77.2	75.9	90.5
FixBi [ 9 ]	96.1	99.3	100	95.0	78.7	79.4	91.4
MPA-LLM [ 18 ]	91.4	97.5	99.8	93.8	81.8	80.6	90.8

Table 2: Comparison of accuracies (%) on Office-31 between models using the ResNet-50 backbone.

Method	Venue (Year)	plane	bcycl	bus	car	horse	knife	mcycl	person	plant	sktbrd	train	truck	Avg.
ResNet-101 [ 3 ]	CVPR (2016)	55.1	53.3	61.9	59.1	80.6	17.9	79.7	31.2	81.0	26.5	73.5	8.5	52.4
FixBi [ 9 ]	CVPR (2021)	96.1	87.8	90.5	90.3	96.8	95.3	92.8	88.7	97.2	94.2	90.9	25.7	89.7
AaD [ 19 ]	NeurIPS (2022)	97.4	90.5	80.8	76.2	97.3	96.1	89.8	82.9	95.5	93.0	92.0	64.7	88.0
PADCLIP [ 5 ]	ICCV (2023)	96.7	88.8	87.0	82.8	97.1	93.0	91.3	83.0	95.5	91.8	91.5	63.0	88.5
SKD [ 17 ]	arXiv (2023)	98.8	87.8	92.0	72.0	98.7	93.4	94.8	75.1	92.5	96.0	95.9	72.0	89.1
VFR [ 4 ]	WACV (2024)	97.2	89.3	87.6	83.1	98.4	95.4	92.2	82.5	94.9	93.2	91.3	64.7	89.1

Table 3: Accuracy (%) on the VisDA-2017 dataset using the ResNet-101 backbone. The best result in each class is marked in bold .

Figure 2: UMAP projection of the features obtained by SPACE in the VISDA-2017 dataset, after the proposed method’s training protocol. Source samples are shown as points, target samples as triangles, and text probes as crosses.

Domain Adaptation

Domain adaptation transfers machine-learning models across changing data distributions, often using unlabeled or limited target-domain examples.

See all 254 papers

VisCoP: Visual Probing for Video Domain Adaptation of Vision Language Models
Oct 15, 2025Dominick Reilly, Manish Kumar Govind, Le Xue +1Domain Adaptation VLM Adaptation
University of North Carolina at Charlotte · Elorian AI
Improving CLIP Adaptation by Breaking Tail Alignment for Source-Free Cross-Domain Few-Shot Learning
May 28, 2026Shuai Yi, Yixiong Zou, Yuhua Li +1Source-Free Cross-Domain Few-Shot Learning Source-Free Domain Adaptation
Institute of Artificial Intelligence, Huazhong University of Science and Technology, Wuhan, China · School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China.
BiCLIP: Domain Canonicalization via Structured Geometric Transformation
Mar 9, 2026Pranav Mantini, Shishir K. ShahVLM Adaptation Domain Adaptation
University of Houston, Department of Computer Science · The University of Oklahoma, School of Computer Science

Oct 15, 2025J/K move · Enter open · S save

VisCoP: Visual Probing for Video Domain Adaptation of Vision Language Models

Dominick Reilly, Manish Kumar Govind, Le Xue, Srijan Das

University of North Carolina at Charlotte · Elorian AI

Domain Adaptation VLM Adaptation VLMs for Robotics Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Large Vision Language Models (VLMs) excel at general visual reasoning but experience significant performance degradation when deployed in novel domains that exhibit substantial distribution shifts from their pretraining data. Existing domain adaptation methods rely on finetuning standard VLM components; however, depending on which components are updated, these approaches either limit the model's ability to learn domain-specific representations or cause catastrophic forgetting of previously acquired capabilities. We introduce Vision Contextualized Probing (VisCoP), a parameter-efficient adaptation framework that augments the VLM vision encoder with a compact set of learnable visual probes. By learning domain-specific visual representations through these probes while requiring only minimal updates to pretrained model components, VisCoP effectively adapts to new domains without sacrificing existing knowledge. We evaluate VisCoP across three challenging adaptation settings: cross-view (exocentric to egocentric), cross-modal (RGB to depth), and cross-task (human understanding to robot control). Across all scenarios, VisCoP consistently outperforms existing domain adaptation strategies, achieving superior target-domain performance while preserving the pretrained VLM's capabilities on the source domain. These results demonstrate that lightweight visual probing provides an effective and robust solution for adapting VLMs under substantial distribution shifts. Code, models, and evaluation protocols are available at https://github.com/dominickrei/VisCoP.

Abstract

Domain Adaptation

Explore similar work

VisCoP: Visual Probing for Video Domain Adaptation of Vision Language Models

Improving CLIP Adaptation by Breaking Tail Alignment for Source-Free Cross-Domain Few-Shot Learning

BiCLIP: Domain Canonicalization via Structured Geometric Transformation