Source-Free Domain Adaptive Object Detection with Semantics Compensation
Authors: Song Tang, Jiuzheng Yang, Mao Ye, Boyu Wang, Yan Gan, Xiatian Zhu
Organizations: University of Shanghai for Science and Technology · Universität Hamburg · University of Electronic Science and Technology of China · Western University · Vector Institute · Chongqing University · University of Surrey
Strong data augmentation is a fundamental component of state-of-the-art mean teacher-based Source-Free domain adaptive Object Detection (SFOD) methods, enabling consistency-based self-supervised optimization along weak augmentation. However, our theoretical analysis and empirical observations reveal a critical limitation: strong augmentation can inadvertently erase class-relevant components, leading to artificial inter-category confusion. To address this issue, we introduce Weak-to-strong Semantics Compensation (WSCo), a novel remedy that leverages weakly augmented images, which preserve full semantics, as anchors to enrich the feature space of their strongly augmented counterparts. Essentially, this compensates for the class-relevant semantics that may be lost during strong augmentation on the fly. Notably, WSCo can be implemented as a generic plug-in, easily integrable with any existing SFOD pipelines. Extensive experiments validate the negative impact of strong augmentation on detection performance, and the effectiveness of WSCo in enhancing the performance of previous detection models on standard benchmarks.
Figures & tables
Figure 1: Illustration of artificial inter-category confusion caused by strong data augmentation. (a) The head is mosaicked during augmentation. (b) As a result, the model confuses the classes “person” and “umbrella”.
Figure 2: Overview of WSCo . The standard MT framework regularized by objective Lmt has generated weak ( Xˉ ) and strong ( X^ ) instance features. In WSCo (blocked by green box), a mapping network (termed MNet), which is optimized by I: Adversarial semantics calibration , first projects Xˉ , X^ into a latent space, obtaining semantics bias-less weak and strong instance embedding sets (i.e., WIE, SIE) Zˉ , Z^ . After that, II: Adaptation-ware prototype-guided labeling refines semantics in Zˉ , while III: Uncertainty-ware supervised contrastive learning enhances the representation of Z^ by encoding the mined semantics.
Figure 3: TP, FP-based empirical evidence for that strong augmentation causes artificial inter-category confusion. (a) displays results of SMT w/ weak-only (red) and SMT (blue). (b) shows SMT cases with cumulative augmentations. By incremental intensity, the augmentations are five levels (1 ∼ 5): Horizontal Flip, ColorJitter, RandomGrayscale, GaussianBlur and RandomErasing.
Table 4
Pascal → Watercolor
P → C
Methods
SF
bike
bird
car
cat
dog
prsn
mAP
mAP
ResNet-101
Source
-
70.8
46.4
47.7
30.4
29.8
53.8
48.5
30.9
SWDA [ 33 ]
✗
82.3
55.9
46.5
32.7
35.5
66.7
53.3
38.1
SAPNet [ 22 ]
✗
81.1
51.1
53.6
34.3
39.8
71.3
55.2
42.2
PD [ 41 ]
✗
95.8
54.3
48.3
42.4
35.1
65.8
56.9
42.1
UMT [ 8 ]
✗
88.2
55.3
51.7
39.8
43.6
69.9
58.1
44.1
Table 3: Results on Pascal → Watercolor, Pascal → Clipart (P → C); the full results on Clipart are in Appendix G .
Figure 4: FP-Gain analysis on Cityscapes → FoggyCityscapes. Left to Right display results of SMT, LODS, IRG, and LPLD groups.
#
Methods
Lmt
Lsc
Luscl
C → F
P → C
1
Source
✗
✗
✗
26.3
30.9
2
SMT [ 38 ]
✓
✗
✗
36.3
35.8
3
✓
✓
✗
39.6
38.7
4
✓
✗
✓
37.8
38.1
5
SMT + WSCo
✓
✓
✓
40.6
41.3
6
SMT+WSCo w/o MNet
39.2
38.4
Table 4: Ablation study results of mAP on adaptation tasks Cityscapes → FoggyCityscapes (C → F) and Pascal → Clipart (P → C).
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8
Figure 7: Statistical analysis of size imbalance between hard and easy positives based on the proposed metric of Hard-Easy (HE) ratio-score. (a) and (b) present the results at the image-level, whilst the dataset-level results are given in (c) and (d).
Structure of Mapping Network
Conv 2048 × 3 × 3, stride 2 → BatchNorm → ReLU
Conv 1024 × 3 × 3, stride 2 → BatchNorm → ReLU
Conv 1024 × 3 × 3, stride 2 → BatchNorm → ReLU
FC (1024 , 2048) → BatchNorm → ReLU
FC (2048 , 2048)
FC (2048 , 512)
Appendix
Table 5: Architectures of the MNet.
Figure 8: Feature distribution visualization on task Cityscapes → FoggyCityscapes by the t-SNE tool. Categories are presented in different colors.
Figure 9: Qualitative results on all five tasks: P → C (Pascal → Clipart), P → W (Pascal → Watercolor), C → F (Cityscapes → FoggyCityscapes), K → C (KITTI → Cityscapes), and S → C (SIM10k → Cityscapes). Top five rows: Results of MT and IRG groups. Bottom five rows: Results of LODS and LPLD groups. The green , red , and blue bounding boxes represent true positives (TP), false negatives (FN), and false positives (FP), respectively. Zoom in for best view.
Method
SF
aero
bcle
bird
boat
bott
bus
car
cat
chri
cow
tabl
dog
hors
bike
prsn
plnt
shep
sofa
trin
tv
mAP
ResNet-101
Source [ 29 ]
–
26.8
28.8
23.4
24.1
41.9
31.4
28.5
4.9
32.0
11.0
29.8
4.3
41.2
53.3
43.7
42.0
13.1
19.9
37.1
34.4
28.6
SWDA [ 33 ]
✗
26.2
48.5
32.6
33.7
38.5
54.3
37.1
18.6
34.8
58.3
17.0
12.5
33.8
65.5
61.6
52.0
9.3
24.9
54.1
49.1
38.1
SAPNet [ 22 ]
✗
27.4
70.8
32.0
27.9
42.4
63.5
47.5
14.3
48.2
46.1
31.8
17.9
43.8
68.0
68.1
49.0
18.7
20.4
55.8
51.3
42.2
PD [ 41 ]
✗
41.5
52.7
34.5
28.1
43.7
58.5
41.8
15.3
40.1
54.4
26.7
28.5
37.7
75.4
63.7
48.7
16.5
30.8
54.5
48.7
42.1
UMT [ 8 ]
✗
39.6
59.1
32.4
35.0
45.1
61.9
48.4
7.5
46.0
67.6
21.4
29.5
48.2
75.9
70.5
56.7
25.9
28.9
39.4
43.6
44.1
SMT [ 38 ]
✓
33.3
43.4
23.9
35.7
48.9
65.3
36.4
6.1
41.0
18.9
26.7
13.1
37.4
60.4
44.2
41.2
24.5
15.2
57.0
43.3
35.8
Appendix
Table 6: Full of results on task Pascal → Clipart.
Figure 10: Visualization of proposal-based image uncertainty estimation ( σ ) on task Cityscapes → FoggyCityscapes. Top and Bottom provide the cases of σ=9.8 and σ=563.9 , respectively.
Figure 11: The mAP varying curve of three main hyper-parameters in WSCo based on the task Pascal → Clipart. Left , Middle and Right are results of the parameter α , β , and λ , respectively.
Figure 12: Classification accuracy of foreground instances on transfer task Cityscapes → FoggyCityscapes ( Left ), Pascal → Watercolor ( Right ).
Source-Free Domain Adaptive Object Detection (SF-DAOD) aims to adapt a detector trained on a labeled source domain to an unlabeled target domain without retaining any source data. Despite recent progress, most popular approaches focus on tuning pseudo-label thresholds or refining the teacher-student framework, while overlooking object-level structural cues within cross-domain data. In this work, we present CGSA, the first framework that brings Object-Centric Learning (OCL) into SF-DAOD by integrating slot-aware adaptation into the DETR-based detector. Specifically, our approach integrates a Hierarchical Slot Awareness (HSA) module into the detector to progressively disentangle images into slot representations that act as visual priors. These slots are then guided toward class semantics via a Class-Guided Slot Contrast (CGSC) module, maintaining semantic consistency and prompting domain-invariant adaptation. Extensive experiments on multiple cross-domain datasets demonstrate that our approach outperforms previous SF-DAOD methods, with theoretical derivations and experimental analysis further demonstrating the effectiveness of the proposed components and the framework, thereby indicating the promise of object-centric design in privacy-sensitive adaptation scenarios. Code is released at https://github.com/Michael-McQueen/CGSA.
Boyang Dai, Zeng Fan, Zihao Qi +2
School of Computing and Data Science, The University of Hong Kong · Department of Electrical & Computer Engineering, National University of Singapore · School of Information and Electronics, Beijing Institute of Technology
Open-vocabulary object detection (OVOD) enables models to detect any object category, including unseen ones. Benefiting from large-scale pre-training, existing OVOD methods achieve strong detection performance on general scenarios (e.g., OV-COCO) but suffer severe performance drops when transferred to downstream tasks with substantial domain shifts. This degradation stems from the scarcity and weak semantics of category labels in domain-specific task, as well as the inability of existing models to capture auxiliary semantics beyond coarse-grained category label. To address these issues, we propose HSA-DINO, a parameter-efficient semantic augmentation framework for enhancing open-vocabulary object detection. Specifically, we propose a multi-scale prompt bank that leverages image feature pyramids to capture hierarchical semantics and select domain-specific local semantic prompts, progressively enriching textual representations from coarse to fine-grained levels. Furthermore, we introduce a semantic-aware router that dynamically selects the appropriate semantic augmentation strategy during inference, thereby preventing parameter updates from degrading the generalization ability of the pre-trained OVOD model. We evaluate HSA-DINO on OV-COCO, several vertical domain datasets, and modified benchmark settings. The results show that HSA-DINO performs favorably against previous state-of-the-art methods, achieving a superior trade-off between domain adaptability and open-vocabulary generalization.
Weihao Cao, Runqi Wang, Xiaoyue Duan +3
State Key Laboratory of Advanced Rail Autonomous Operation, Beijing Jiaotong University · Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University · School of Computer Science & Technology, Beijing Jiaotong University +1
Adverse weather conditions such as fog, rain, snow, and low illumination introduce structured visibility degradations that severely impair object detectors trained on clear-weather data. When source data are inaccessible due to privacy or trans- mission constraints, source-free domain adaptation (SFDA) becomes necessary. However, existing SFDA methods based on pairwise contrastive learning struggle to capture high-order semantics and often suffer from agreement collapse under severe corruption. We propose REUNION, a hypergraph-guided SFDA framework for object detection in adverse weather conditions. REUNION constructs a contrastive hypergraph over target-domain object proposals, encoding high-order relations through intra-image context, weather-aware grouping, prototype-based semantic anchors, and uncertainty-aware connections between reliable and low-confidence instances. To effectively exploit these structured relations, we introduce a group-wise Hyper-InfoNCE objective that optimizes representations at the hyperedge level, enabling semantic information to propagate from confident proposals to corrupted or low-contrast instances. Experiments with diverse benchmarks demonstrate that REUNION effectively mitigates domain shifts and achieves state-of-the-art performance in SFDA settings.
Jicheng Yuan, Tung Kieu, Manh Nguyen-Duc +2
Open Distributed Systems, Technical University of Berlin · Department of Computer Science, Aalborg University, Denmark · Fraunhofer Institute for Open Communication Systems, Berlin, Germany