GIFT: Representation Geometry Matters for Single-Domain Generalized Object Detection

Yin Zhang; Yaoyue Zheng; Yongqiang Zhang; Bogdan Raducanu; Dan Li

cs.CVOct 26, 2026

GIFT: Representation Geometry Matters for Single-Domain Generalized Object Detection

Authors: Yin Zhang, Yaoyue Zheng, Yongqiang Zhang, Bogdan Raducanu, Dan Li

Organizations: School of Instrument Science and Engineering, Harbin Institute of Technology · Computer Vision Center, Universitat Autònoma de Barcelona · Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University · College of Computer Science, Inner Mongolia University

Abstract

Single-Domain Generalized Object Detection (S-DGOD) aims to generalize from a single source domain to unseen domains under distribution shifts. Existing efforts mainly focus on simulating unseen domains or suppressing domain-specific factors. However, they largely overlook the intrinsic generalization capability embedded in Vision Foundation Models (VFMs). In this work, we show that unconstrained fine-tuning disrupts the geometric structure of pre-trained representations, which encodes domain-invariant semantics. Motivated by this, we propose Geometric-Invariance Fine-Tuning (GIFT), a geometric-aware fine-tuning method for VFMs. Specifically, we introduce a series of learnable Householder reflections to the pre-trained weights to encourage geometry-consistent adaptation under domain shift. In addition, we propose a lightweight channel-wise scaling to facilitate effective task-specific adaptation. Extensive experiments demonstrate that GIFT consistently outperforms state-of-the-art (SOTA) approaches on two S-DGOD benchmarks. Notably, by introducing fewer than 1% additional trainable parameters into the frozen backbone, the proposed method achieves improvements of +7.9% and +2.9% in mPC over the previous SOTA on the Cityscapes-C and Diverse Weather Dataset, respectively. This finding highlights that preserving representation geometry is crucial for robust cross-domain generalization, establishing GIFT as a principled and efficient solution to domain shift.

Figures & tables

Figure 1: (a) The pre-trained VFM exhibits an intrinsically stable geometry, while (b) unconstrained fine-tuning Jia et al. (2022) often disrupts this structure, degrading generalization. In contrast, (c) by preserving geometric structure during fine-tuning, GIFT effectively improves generalization across unseen domains (DF., DR., NR., and NS.) in the DWD dataset Wu and Deng (2022) . More results are provided in Section 4.4 .

Figure 3: An overview of our proposed method. It is composed of two key components: (a) Fine-Tuning with Geometric Preservation (Section 3.2 ) and (b) Channel-Level Scaling (Section 3.3 ). Notably, we optimize only the multi-head attention parameters within each block. The updated weights, $\bm{W}_{\text{new}}$ , are obtained by adding the scaled residual and channel-wise adjustment to the original weights $\bm{W}$ .

		Noise			Blur				Weather				Digital
Methods	Norm.	Gau.	Sho.	Imp.	Def.	Gla.	Mot.	Zoo.	Sno.	Fro.	Fog	Bri.	Con.	Ela.	Pix.	JPE.	mPC
Traditional S-DGOD
FRCNN Ren et al. (2016)	42.2	0.5	1.1	1.1	17.2	16.5	18.3	2.1	2.2	12.3	29.8	32.0	24.1	40.1	18.7	15.1	15.4
AutoAug Zoph et al. (2020)	42.4	0.9	1.6	0.9	16.8	14.4	18.9	2.0	1.9	16.0	32.9	35.2	26.3	39.4	17.9	11.6	15.8
AugMix Hendrycks et al. (2019)	39.5	5.0	6.8	5.1	18.3	18.1	19.3	6.2	5.0	20.5	31.2	33.7	25.6	37.4	20.3	19.6	18.1
Stylized Geirhos et al. (2018)	36.3	4.8	6.8	4.3	19.5	18.7	18.5	2.7	3.5	17.0	30.5	31.9	22.7	33.9	22.6	20.8	17.2

Table 2: Comparison with SOTA methods on Cityscapes-C Michaelis et al. (2019) . mPC denotes the average performance across 15 different corruption types. For fairness, we adopt Faster R-CNN Ren et al. (2016) as the detection head. ‘Baseline’ denotes a fully frozen DINOv2 Oquab et al. (2023) backbone with only the Faster R-CNN detection head. ‘

\dagger

’ denotes unreported results re-implemented with the official code under the same settings.

Table 5

Figure 7: Feature correlation analysis on Cityscapes Cordts et al. (2016) (Clean domain) and Cityscapes-C Michaelis et al. (2019) (Corrupted domain) datasets. Darker colors indicate higher cosine similarity between class features.

Table 8

Figure 8: Visualization of attention maps for the DINOv2 Oquab et al. (2023) , conventional PEFT methods AdaptFormer Chen et al. (2022) , VPT Jia et al. (2022) , and our geometric-preserving (GP) fine-tuning strategy on the DWD Wu and Deng (2022) .

Figure 9: Visualization of detection results on the DWD Wu and Deng (2022) . (a) to (d) correspond to the four weather conditions of the DWD target domain: DF., DR., NR., and NS. GIFT is compared with the traditional S-DGOD method ClipGap Vidit et al. (2023) , Baseline (Frozen DINOv2 Oquab et al. (2023) with Faster R-CNN Ren et al. (2016) ) and the SOTA method Cauvis Li et al. (2025) . Best seen on computer, in color and zoomed in.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

$QKV$	$Output$	$FFN$	DS.	DF.	DR.	NR.	NS.	Avg.	mPC
-	-	-	71.2	53.5	60.8	42.6	59.5	57.5	54.1
✓	-	-	75.1	57.5	65.5	50.1	62.3	62.1	58.9
✓	✓	-	76.0	58.3	65.9	49.9	63.3	62.7	59.4
✓	✓	✓	75.0	57.5	65.6	49.4	62.4	62.0	58.7

Appendix

Table 8: Position of the HR matrices within the Transformer blocks. First row is Baseline.

QKV

Output

, and

FFN

denote QKV projection layer, output projection layer and feed-forward network (MLP layer), respectively.

Methods	C. $\rightarrow$ B.	C. $\rightarrow$ M.	G. $\rightarrow$ C.	G. $\rightarrow$ B.	G. $\rightarrow$ M.
Rein [ 50 ]	63.54	74.03	66.40	60.40	66.10
FADA [ 2 ]	65.12	75.86	68.23	61.94	68.09
SoMA [ 59 ]	67.02	76.45	71.82	61.31	71.67
Causal-Tune [ 61 ]	66.28	76.05	66.22	61.80	68.21
Ours	67.42	76.71	68.60	62.01	68.08

Appendix

Table 9: DGSS performance of our proposed GIFT. ( C. ), ( B. ), ( M. ), and ( G. ) denote Cityscapes, BDD100K, Mapillary and GTA5 datasets, respectively.

VFMs	Fine-tune Method	DS.	DF.	DR.	NR.	NS.	Avg.	mPC
EVA02-Large [ 15 , 14 ]	Freeze	63.0	45.2	48.6	27.3	38.8	44.6	40.0
	+ Rein [ 50 ]	68.3	49.2	54.8	32.2	48.1	50.5 (+5.9 $\uparrow$ )	46.1 (+6.1 $\uparrow$ )
	+ Cauvis [ 30 ]	69.7	50.2	57.6	34.2	48.1	52.0 (+7.4 $\uparrow$ )	47.5 (+7.5 $\uparrow$ )
	+ Ours	71.6	51.3	59.2	35.9	49.2	53.4 (+8.8 $\uparrow$ )	48.9 (+8.9 $\uparrow$ )
SAM-Huge [ 28 ]	Freeze	69.7	50.5	52.5	28.8	52.9	50.9	46.2
	+ Rein [ 50 ]	70.0	51.9	54.0	30.9	54.4	52.2 (+1.3 $\uparrow$ )	47.8 (+1.6 $\uparrow$ )

Appendix

Table 10: Performance of different VFMs with a DETR-based [ 60 ] detection head. “Avg.” indicates the mean mAP across the five conditions.

	Daytime-Foggy				Dusk-Rainy				Night-Rainy				Night-Sunny
Methods	hev.	mid.	non.	all	hev.	mid.	non.	all	hev.	mid.	non.	all	hev.	mid.	non.	all	mPC
IN [ 21 ]	25.7	33.4	27.0	28.5	34.3	25.0	13.7	22.8	19.5	11.5	8.8	12.6	38.3	27.4	25.3	29.6	23.4
SW [ 38 ]	27.1	34.9	33.8	32.2	37.0	30.3	16.6	26.3	20.0	13.9	9.4	13.7	39.5	33.2	29.6	33.4	26.1
FRCNN [ 41 ]	30.6	37.8	32.6	33.5	34.6	32.0	21.1	28.0	18.7	17.5	9.0	14.2	43.3	33.7	32.2	35.8	27.9
CDSD [ 51 ]	28.5	39.3	32.9	33.5	38.9	32.2	18.5	28.2	21.8	19.7	11.2	16.6	42.0	35.2	34.0	36.6	28.7
ClipGap [ 47 ]	30.7	46.0	38.9	38.5	40.1	38.8	22.8	32.3	25.8	22.7	11.3	18.7	40.3	38.6	33.5	36.9	31.6

Appendix

Table 11: Detailed results on Diverse Weather Dataset [ 51 ] , using Faster R-CNN as the detection head. mPC denotes the average performance across 4 different adverse weather conditions. Bold and underline results represent best and second-best performance, respectively. GP denotes geometric preservation.

	Daytime-Sunny
Methods	Bus	Bike	Car	Motor	Person	Rider	Truck	mAP
InterNorm [ 21 ]	58.4	34.2	42.2	44.1	31.6	40.8	55.5	43.9
SW [ 38 ]	62.3	42.9	53.3	49.9	39.2	46.2	60.6	50.6
ISW [ 7 ]	62.9	44.6	53.5	49.2	39.9	48.3	60.9	51.3
Faster R-CNN [ 41 ]	66.9	45.9	69.8	46.5	50.6	49.6	64.0	56.2
CDSD [ 51 ]	68.8	50.9	53.9	56.2	41.8	52.4	68.7	56.1

Appendix

Table 12: Detailed results on Daytime-Sunny (source domain) of DWD. Baseline is a frozen DINOv2 [ 37 ] backbone with a Faster R-CNN [ 41 ] detector. ‘F’ and ‘D’ denote Faster R-CNN and DETR-based [ 60 ] detector.

	Daytime-Foggy								Dusk-Rainy
Methods	Bus	Bike	Car	Mot.	Per.	Rid.	Tru.	mAP	Bus	Bike	Car	Mot.	Per.	Rid.	Tru.	mAP
FR [ 41 ]	34.5	29.6	49.3	26.2	33.0	35.1	26.7	33.5	34.2	21.8	47.9	16.0	22.9	18.5	34.9	28.0
CDSD [ 51 ]	32.9	28.0	48.8	29.8	32.5	38.2	24.1	33.5	37.1	19.6	50.9	13.4	19.7	16.3	40.7	28.2
ClipGap [ 47 ]	36.2	34.2	57.9	34.0	38.7	43.8	25.1	38.5	37.8	22.8	60.7	16.8	26.8	18.7	42.4	32.3
G-NAS [ 52 ]	32.4	31.2	57.7	31.9	38.6	38.5	24.5	36.4	44.6	22.3	66.4	14.7	32.1	19.6	45.8	35.1
PDD [ 31 ]	36.1	34.5	58.4	33.3	40.5	44.2	26.2	39.1	39.4	25.2	60.9	20.4	29.9	16.5	43.9	33.7

Appendix

Table 13: Detailed results on unseen target domain of DWD. ‘Mot’, ‘Per.’, ‘Rid.’, and ‘Tru.’ denote ‘Motor’, ‘Person’, ‘Rider’, and ‘Truck’, respectively. ‘F’ and ‘D’ denote Faster R-CNN [ 41 ] and DETR-based [ 60 ] detector.

Figure 10: Comparison between our method and Cauvis [ 30 ] on Cityscapes-C from Level-1 to Level-5. ‘Avg.’ denotes the average mAP across the 5 corruption levels.

	GIFT (Ours)						Cauvis [ 30 ]
Types	Level-1	Level-2	Level-3	Level-4	Level-5	Avg.	Level-1	Level-2	Level-3	Level-4	Level-5	Avg.
Source	-	-	-	-	-	62.7	-	-	-	-	-	54.6
Gauss	48.6	40.0	21.9	5.7	0.0	23.2	38.0	27.8	14.5	3.9	0.0	16.8
Shot	51.2	42.8	26.4	4.7	0.6	25.1	43.1	32.3	19.7	3.6	0.1	19.8
Impulse	45.5	33.1	20.4	3.9	0.1	20.6	34.6	23.2	15.3	2.8	0.0	15.2
Defocus	58.9	57.3	51.3	44.6	36.7	49.8	50.9	49.9	41.6	35.2	29.4	41.4

Appendix

Table 14: Quantitative results on Cityscapes-C from Level-1 to Level-5. ‘Avg.’ denotes the average mAP across the 5 corruption levels.

Figure 11: More visualization of detection results on the DWD [ 51 ] . (a) to (d) correspond to the four weather conditions of the DWD target domain: DF., DR., NR., and NS. GIFT is compared with the traditional S-DGOD method ClipGap [ 47 ] , Baseline (Frozen DINOv2 [ 37 ] with Faster R-CNN [ 41 ] ) and the SOTA method Cauvis [ 30 ] . Best seen on computer, in color and zoomed in.

Cross-Domain Generalization

Cross-domain generalization studies how machine-learning models retain reliable performance when data, sensors, populations, or tasks differ from training.

See all 178 papers

VFM $^{4}$ SDG: Unveiling the Power of VFMs for Single-Domain Generalized Object Detection
Apr 23, 2026Yupeng Zhang, Ruize Han, Ningnan Guo +3Detection Transformer Cross-Domain Generalization
College of Intelligence and Computing, Tianjin University, Tianjin 300350, China · Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology, Shenzhen 518107, Guangdong, China
Bridge: Basis-Driven Causal Inference Marries VFMs for Domain Generalization
Apr 29, 2026Mingbo Hong, Feng Liu, Caroline Gevaert +2Domain Generalization Cross-Domain Generalization
University of Twente · 2Drexel University
Prototype-Anchored Generalized Manifold Regression for Unknown-Domain Object Detection
Jul 8, 2026Zihao Zhang, Aming Wu, Yang Li +1Domain Generalization OOD Generalization
College of Intelligence and Computing, Tianjin Key Lab of Machine Learning, Tianjin University, Tianjin 300072, China · School of Computer Science and Information Engineering, Hefei University of Technology, China

Apr 23, 2026J/K move · Enter open · S save

VFM $^{4}$ SDG: Unveiling the Power of VFMs for Single-Domain Generalized Object Detection

Yupeng Zhang, Ruize Han, Ningnan Guo, Wei Feng+2

College of Intelligence and Computing, Tianjin University, Tianjin 300350, China · Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology, Shenzhen 518107, Guangdong, China

Detection Transformer Cross-Domain Generalization Object Detection

Open paper AskPDF arXiv

Real-world weather, illumination, and imaging variations often induce severe domain shifts, degrading single-source detectors in unseen environments. Existing single-domain generalized object detection (SDGOD) methods mainly rely on data augmentation or domain-invariant learning, while largely overlooking how domain shift disrupts detector prediction stability. Through analytical experiments, we find that performance degradation is mainly dominated by increasing missed detections. Further analysis shows that this phenomenon stems from reduced cross-domain stability in DETR-style detectors: domain shift disrupts encoder-side object-background and inter-instance relations, and further weakens the semantic-spatial binding between decoder queries and real objects. Motivated by this, we find that vision foundation models (VFMs) still preserve stable relational structures and object responses under severe shifts, making them suitable cross-domain stability priors to compensate for detector degradation. To this end, we propose VFM

^{4}

SDG, a dual-prior learning framework for SDGOD, which introduces a frozen VFM into encoder representation learning and decoder query modeling. Specifically, we propose Cross-domain Stable Relational Prior Distillation to distill stable object-background and inter-instance relations from the VFM into the encoder, compensating for relational degradation. Meanwhile, we propose Semantic-Contextual Prior-based Query Enhancement, which injects category semantic prototypes and global object context into queries before they enter the decoder layer, enhancing semantic-spatial query-object binding stability. Extensive experiments show that VFM

^{4}

SDG significantly outperforms existing advanced methods on standard SDGOD benchmarks and two mainstream DETR-based detection frameworks, demonstrating its effectiveness, robustness, and generality.

Abstract

Cross-Domain Generalization

Explore similar work

VFM4^{4}4SDG: Unveiling the Power of VFMs for Single-Domain Generalized Object Detection

Bridge: Basis-Driven Causal Inference Marries VFMs for Domain Generalization

Prototype-Anchored Generalized Manifold Regression for Unknown-Domain Object Detection

VFM $^{4}$ SDG: Unveiling the Power of VFMs for Single-Domain Generalized Object Detection