PISA: A Pseudo-Individual Source-Domain Feature Adaptation Framework for Test-Time Open-Vocabulary Object Detection

Ziyan He; Xiongtai Yang; Tao Wang

cs.CVAug 14, 2026

PISA: A Pseudo-Individual Source-Domain Feature Adaptation Framework for Test-Time Open-Vocabulary Object Detection

Authors: Ziyan He, Xiongtai Yang, Tao Wang

Organizations: Sichuan University

Abstract

Open-vocabulary object detection test-time adaptation (OVOD-TTA) aims to address the performance degradation that pre-trained base models suffer when encountering image-domain shifts. Existing source-free OVOD-TTA methods rely either on refined test-time information for re-scoring or on pseudo-labels for self-training, leading to significant accuracy degradation when initial predictions are poor. Meanwhile, most conventional source-domain estimation methods recover abstract, sparse representations suitable for the classification task, but fail to capture the dense, concrete features required for detection. To address these issues, we propose PISA, a novel source-free OVOD-TTA method that can be seamlessly integrated into open-vocabulary visual backbones. The core components of our method are the Corruption-Invariant Feature Extractor (CIFE), the Feature Alignment Module (FAM), and a multi-scale alignment framework (BAA). To capture detection-suitable features, we develop CIFE to exploit the invariance of CLIP's visual features across corrupted images, ensuring robustness against various corruptions. We further develop FAM and BAA for the pre-training and adaptation to transform the corruption-invariant features into pseudo-individual source-domain features that are close to the original source-domain features. In this way, dense and concrete pseudo-individual source-domain features are used for supervision instead of unreliable pseudo-label signals. Experiments on the corrupted VOC-C, COCO-C, and LVIS-C benchmarks across three base models demonstrate that PISA substantially improves both the localization precision and the category recognition accuracy of the original models. Notably, PISA achieves state-of-the-art performance without requiring access to source-domain data, surpassing existing methods by 3.92% in AP@50% on COCO-C.

Figures & tables

Figure 1: Comparison of PISA with other source free OVOD-TTA methods on COCO-C(AP@50%) with 15 different kinds of corruptions. PISA achieves state-of-the-art performance across most types of corruptions, demonstrating exceptional robustness.

Figure 2: Comparison of different OVOD-TTA methods. (a) illustrates part of OVOD-TTA method rely on source-domain distribution or pseudo label for self-training. (b) shows the paradigm of cache-based training-free OVOD-TTA mehod. (c) presents our OVOD-TTA paradigm, which uses pseudo-individual source-domain feature for adaptation.

Figure 3: Overview of PISA. The CIFE module extracts corruption-invariant features, while the FAM aligns them with patch embeddings via main and target branches. BAA integrates multi-scale adapters into the backbone to bridge the domain gap.

Figure 4: Visualization of the outputs from CLIP visual backbones: ResNet-50 (top) and ViT-B/16 (bottom). Green bboxes highlight features that similar to original images’.

Dataset	Method	Base		Novel		All
Dataset	Method	AP@50%	mAP	AP@50%	mAP	AP@50%	mAP
VOC	GLIP ( Li et al. 2022 )	17.86	12.40	36.17	25.40	33.68	23.63
	PISA (Ours)	35.98 (+18.12)	24.25 (+11.85)	43.31 (+7.14)	30.24 (+4.84)	42.31 (+8.63)	29.42 (+5.79)
	GDINO ( Liu et al. 2024 )	31.00	22.60	44.77	33.86	42.90	32.33
	PISA (Ours)	48.33 (+17.33)	35.17 (+12.57)	48.61 (+3.84)	36.63 (+2.77)	48.57 (+5.67)	36.42 (+4.09)
	LLMDet ( Fu et al. 2025 )	31.77	21.30	50.44	36.46	47.90	34.39

Table 1: Object detection performance compared with baseline models on VOC-C, COCO-C, and LVIS-C.

Figure 5: The architecture of FAM and the inserted position of

\text{BAA}_{t}

(last block) and

\text{BAA}_{a}

(every block) modules.

Method	Source	Imp	Sht	Gau	Def	Gls	Mtn	Zm	Bgt	Fog	Fro	Snw	Cnt	Els	Pix	Jpeg	AVG
TDA	CVPR2024	16.7	17.1	15.8	20.3	9.9	16.9	9.6	45.5	45.9	32.3	28.2	24.4	27.1	9.0	26.3	23.00
HisTPT	NeuralPS2024	18.2	17.9	17.6	21.3	12.5	19.1	9.9	45.6	45.1	32.8	28.2	26.1	26.0	8.6	26.2	23.67
IOU-Filter	CVPR2024	18.4	18.7	18.7	22.3	11.6	18.8	11.7	50.1	53.4	36.6	32.6	31.2	32.0	8.5	26.5	26.07
BCA	CVPR2025	17.3	18.0	16.0	21.0	10.9	16.1	9.9	46.3	46.7	33.9	29.0	25.2	27.5	8.9	26.2	23.53
BCA+	ARXIV2025	18.7	19.9	17.9	22.2	11.5	19.1	10.1	49.6	49.8	35.7	30.1	26.6	28.7	7.8	28.4	25.07
ViTPrompt	CVPR2026	20.4	21.2	18.9	23.1	12.4	19.7	9.9	52.0	51.9	37.1	31.5	27.2	30.0	10.2	28.4	26.26

Table 2: Object Detection Performance Comparison with State-of-the-Art Methods on COCO-C. Metric: mean average precision at 50% (AP@50%). The best results are denoted in bold and the second best results are denoted in bold gray.

Figure 6: t-SNE visualization of clean (red) and corrupted (other colors) feature distributions at the first output layer of different PISA modules. The distinct cluster-like distribution in the right plot indicates that pseudo-individual source-domain features closely resemble real source-domain features.

Figure 7: Qualitative comparison against the baseline GLIP. Bounding boxes are overlaid on the images. Red and green boxes indicate negative and positive results, respectively, with an IoU threshold of 0.5.

Base	FAM		$\text{BAA}_{t}$	$\text{BAA}_{a}$	AP@50%	AP@75%	mAP
Base	Main	Target	$\text{BAA}_{t}$	$\text{BAA}_{a}$	AP@50%	AP@75%	mAP
$\checkmark$					29.32	21.99	20.45
$\checkmark$	$\checkmark$			$\checkmark$	31.46	23.66	22.00
$\checkmark$	$\checkmark$	$\checkmark$			31.54	23.25	21.74
$\checkmark$	$\checkmark$	$\checkmark$		$\checkmark$	32.84	24.62	22.92
$\checkmark$	$\checkmark$		$\checkmark$	$\checkmark$	32.81	24.45	22.77

Table 3: Ablation studies on COCO-C dataset with GDINO Swin-T backbone.

Figure 8: Average AP@50% on COCO-C under different initialization intervals $T$ for GDINO, GLIP, and LLMDet.

Open-Vocabulary Object Detection

Open-vocabulary object detection lets models locate and label objects using text categories beyond their fixed training labels.

See all 87 papers

Mar 12, 2026

ABRA: Teleporting Fine-Tuned Knowledge Across Domains for Open-Vocabulary Object Detection

Although recent Open-Vocabulary Object Detection architectures, such as Grounding DINO, demonstrate strong zero-shot capabilities, their performance degrades significantly under domain shifts. Moreover, many domains of practical interest, such as nighttime or foggy scenes, lack large annotated datasets, preventing direct fine-tuning. In this paper, we introduce Aligned Basis Relocation for Adaptation(ABRA), a method that transfers class-specific detection knowledge from a labeled source domain to a target domain where no training images containing these classes are accessible. ABRA formulates this adaptation as a geometric transport problem in the weight space of a pretrained detector, aligning source and target domain experts to transport class-specific knowledge. Extensive experiments across challenging domain shifts demonstrate that ABRA successfully teleports class-level specialization under multiple adverse conditions. Our code will be made public upon acceptance.

Mattia Bernardi, Chiara Cappellino, Matteo Mosconi +3

University of Modena and Reggio Emilia

Open-Vocabulary Object Detection Vision Foundation Model Adaptation

Ask PDF

May 5, 2026

FACTOR: Counterfactual Training-Free Test-Time Adaptation for Open-Vocabulary Object Detection

Open-vocabulary object detection often fails under distribution shifts, as it can be misled by spurious correlations between non-causal visual attributes (e.g., brightness, texture) and object categories. Existing test-time adaptation (TTA) methods either depend on costly online optimization or perform global calibration, overlooking the attribute-specific nature of these failures. To address this, we propose FACTOR (counterFACtual training-free Test-time adaptation for Open-vocabulaRy object detection), a lightweight framework grounded in counterfactual reasoning. By perturbing test images along non-causal attributes and comparing region-level predictions between original and counterfactual views, FACTOR quantifies attribute sensitivity, semantic relevance, and prediction variation to selectively suppress attribute-dependent predictions-without parameter updates. Experiments on PASCAL-C, COCO-C, and FoggyCityscapes show that FACTOR consistently outperforms prior TTA methods, demonstrating that explicit counterfactual reasoning effectively improves robustness under distribution shifts.

Kaixiang Zhao, Mao Ye, Lihua Zhou +4

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China · Institute of Machine Intelligence (IMI), University of Shanghai for Science and Technology, Shanghai, China · Surrey Institute for People-Centred Artificial Intelligence, CVSSP, University of Surrey, Guildford, UK.

Open-Vocabulary Object Detection Distribution Shift Robustness Test-Time Adaptation

Ask PDF

Apr 25, 2026

Exploring Hierarchical Consistency and Unbiased Objectness for Open-Vocabulary Object Detection

Conventional object detectors typically operate under a closed-set assumption, limiting recognition to a predefined set of base classes seen during training. Open-vocabulary object detection (OVD) addresses this limitation by leveraging vision-language models (VLMs) to generate pseudo labels for novel object classes. However, existing OVD methods suffer from two critical drawbacks: (1) inaccurate class label assignments, as VLMs are optimized for image-level predictions rather than the region-level predictions required for pseudo labeling, and (2) unreliable objectness scores from region proposal networks (RPNs) trained exclusively on base object classes. To address these issues, we propose a novel pseudo labeling framework for OVD. Our approach introduces a hierarchical confidence calibration (HCC) technique, which ensures reliable class label estimation by assessing consistency across hierarchical semantic levels (class, super- and sub-category). We also present LoCLIP, a parameter-efficient adaptation of CLIP that incorporates an objectness token to mitigate base class bias problem of RPNs and provide reliable objectness estimations for novel object classes. Extensive experiments on standard OVD benchmarks, including COCO and LVIS, demonstrate that our approach clearly sets a new state of the art, validating the effectiveness of our approach. Project site: https://cvlab.yonsei.ac.kr/projects/HCC

Sanghoon Lee, Geon Lee, Hyekang Park +1

Yonsei University · Korea Institute of Science and Technology (KIST)

Open-Vocabulary Object Detection Probability Calibration VLM Adaptation Pseudo-Labeling Object Detection

Ask PDF

Abstract

Open-Vocabulary Object Detection

Explore similar work

ABRA: Teleporting Fine-Tuned Knowledge Across Domains for Open-Vocabulary Object Detection

FACTOR: Counterfactual Training-Free Test-Time Adaptation for Open-Vocabulary Object Detection

Exploring Hierarchical Consistency and Unbiased Objectness for Open-Vocabulary Object Detection