Semantically Compatible Knowledge Distillation for Cross-Domain Object Detection with Vision Foundation Models

Qifeng Zhang; Ting Xiang; Zeyuan Bai; Changjian Chen

cs.CVAug 21, 2026

Semantically Compatible Knowledge Distillation for Cross-Domain Object Detection with Vision Foundation Models

Authors: Qifeng Zhang, Ting Xiang, Zeyuan Bai, Changjian Chen

Organizations: National Supercomputing Center in Changsha, Hunan University, Changsha, Hunan Province, China.

Abstract

Vision foundation models (VFMs) offer strong generalization capabilities for domain-adaptive object detection (DAOD). However, existing VFM-based methods overlook the spatial-scale discrepancy between teacher and student feature maps, resulting in semantic incompatibility that weakens both feature alignment and pseudo-label learning. Moreover, domain shift can cause source-trained VFM teachers to miss target-domain objects, limiting the quality of their pseudo-labels. To address these issues, we propose the Semantic Localization-Enhanced Teacher (SLE-T), a semantically compatible knowledge-distillation framework built around a lightweight SLE Adapter for DINOv2. SLE Adapter injects pretrained local-texture priors into DINOv2 to improve cross-domain recognition and reformulates its features into dense representations that are spatially and semantically compatible with the student detector. SLE-T transfers the resulting teacher knowledge through either pseudo-label learning or feature alignment. We instantiate SLE-T with DINOv2-B and DINOv2-L (the ViT-B and ViT-L variants) and compare them with the larger DINOv2-G teacher. Extensive experiments on three DAOD benchmarks demonstrate that our method achieves state-of-the-art performance, and ablation studies confirm the importance of teacher-student semantic compatibility. Notably, SLE-T with DINOv2-B produces competitive or superior pseudo-labels using approximately one-quarter of the training time of DINOv2-G and substantially less GPU memory, demonstrating efficient VFM knowledge transfer under limited computational resources.

Figures & tables

Figure 1: Motivation for semantically compatible knowledge transfer. (A) Resizing matches feature dimensions but not object-level semantics. (B) Our method reformulates teacher features and selects a stride-compatible student representation. (C) Coarse student features lose object details which are helpful for pseudo-label learning. (D) Compatible representations preserve these details for more effective transfer.

Figure 2: Overview of the SLE Teacher framework. An SLE-enhanced teacher is trained on source annotations and then frozen to transfer knowledge through either pseudo-label learning or feature alignment.

Figure 3: Architecture of SLE Adapter. The Spatial Prior Module extracts stride-16 local features from a pretrained CNN. The Injector uses deformable attention (

\mathrm{DAttn}

) to inject these local cues into DINOv2 tokens, while the Extractor reverses the attention direction to produce a dense stride-16 representation refined by ConvFFN. The resized final DINOv2 feature is added as a semantic residual, preserving pretrained VFM knowledge in the student-compatible teacher feature. Blue and orange blocks denote frozen and trainable components, respectively.

Method	Type	Person	Rider	Car	Truck	Bus	Train	Motor	Bicycle	mAP
DA-Faster ( Chen et al., 2018 )	FR	29.2	40.4	43.4	19.7	38.3	28.5	23.7	32.7	32.0
DICN ( Jiao et al., 2022 )	FR	47.3	57.4	64.0	22.7	45.6	29.6	38.6	47.4	44.1
NLTE ( Liu et al., 2022 )	FR+GA	43.1	50.7	58.7	33.6	56.7	42.7	33.7	43.3	45.4
PT ( Chen et al., 2022 )	FR+MT	43.2	52.4	63.4	33.4	56.6	37.8	41.3	48.7	47.1
MIC ( Hoyer et al., 2023 )	FR+MT	50.9	55.3	67.0	33.9	52.4	33.7	40.6	47.5	47.6
AT ( Li et al., 2022c )	FR+MT	45.5	55.1	64.2	35.0	56.3	54.3	38.5	51.9	50.9

Table 1: Class-wise

\mathrm{AP}_{50}

(%) on Cityscapes

\rightarrow

Foggy Cityscapes (all fog levels). GA and MT denote Graph Alignment and Mean Teacher framework, respectively. ‘*’ indicates results we reproduced using the official publicly available code.

Method	Type	Person	Rider	Car	Truck	Bus	Motor	Bicycle	mAP
DA-Faster ( Chen et al., 2018 )	FR	28.9	27.4	44.2	19.1	18.0	14.2	22.4	24.9
SIGMA ( Li et al., 2022b )	FCOS+GA	46.9	29.6	64.1	20.2	23.6	17.9	26.3	32.7
TDD ( He et al., 2022 )	FR+MT	39.6	38.9	53.9	24.1	25.5	24.5	28.8	33.6
PT ( Chen et al., 2022 )	FR+MT	40.5	39.9	52.7	25.8	33.8	23.0	28.8	34.9
NSA ( Zhou et al., 2023 )	FR+MT	–	–	–	–	–	–	–	35.5
REACT ( Li et al., 2024 )	FR+MT	–	–	–	–	–	–	–	35.8

Table 2: Class-wise

\mathrm{AP}_{50}

(%) on Cityscapes

\rightarrow

BDD100K Daytime.

Method	Fog	Night	Rain	Snow
AT ( Li et al., 2022c )	62.2	29.5	37.7	55.2
DT* ( Lavoie et al., 2025 )	62.0	33.3	33.4	49.4
SLE-T (ViT-B)	66.3	34.9	38.8	55.5
SLE-T (ViT-L)	66.0	35.2	42.9	55.7

Table 3:

\mathrm{mAP}_{50}

(%) comparison on Cityscapes

\rightarrow

ACDC under different adverse weather conditions.

Backbone	$\mathrm{mAP}_{50}$		Training Time (h)
	C	BD	Training Time (h)
ViT-G	62.8	50.7	26.35
SLE (ViT-B)	65.8	49.9	5.93
SLE (ViT-L)	64.9	52.6	7.95

Table 4: Labeller accuracy and training time on Cityscapes

\rightarrow

BDD100K Daytime. C and BD denote Cityscapes and BDD100K Daytime.

No.	Layer	FA	AlignM	FC	BD
1	VGG4	–	–	40.1	22.3
2	VGG4	$\checkmark$	ViT-B	46.6	32.2
3	VGG4	$\checkmark$	SLE (ViT-B)	47.4	33.1
4	VGG3	–	–	38.7	21.7
5	VGG3	$\checkmark$	ViT-B	48.9	33.3
6	VGG3	$\checkmark$	SLE (ViT-B)	51.0	34.3

Table 5: Feature-alignment ablation on Foggy Cityscapes (FC) and BDD100K Daytime (BD). FA and AlignM denote feature alignment and the alignment model, respectively.

Figure 4: Grad-CAM comparison between raw and SLE-enhanced DINOv2-B.

Feature Layer	PL	FC	BD
VGG4	–	40.1	22.3
VGG4	$\checkmark$	53.6	44.3
VGG3	–	38.7	21.7
VGG3	$\checkmark$	57.4	47.7

Table 6: Pseudo-label-learning ablation on Foggy Cityscapes (FC) and BDD100K Daytime (BD). PL denotes pseudo-label learning. All experiments use pseudo labels generated by SLE-enhanced Dinov2-B.

Backbone	Pre-SPM	FLB	$\mathbf{mAP}_{50}$
ViT-B	–	–	51.8
ViT-L	–	–	54.5
SLE (ViT-B)	–	–	54.9
	$\checkmark$	–	58.5
	–	$\checkmark$	56.8
	$\checkmark$	$\checkmark$	59.9

Table 7: SLE Adapter ablation on Cityscapes

\rightarrow

Foggy Cityscapes. Pre-SPM and FLB denote a pretrained spatial prior module and fine-tuning the final DINOv2 block, respectively.

Backbone	Model Size (M)	$\mathrm{mAP}_{50}$
		FC	BD
ViT-B	86	51.8	43.8
ViT-L	300	55.4	46.6
ViT-G	1,100	57.0	50.7
SLE (ViT-B)	108	59.9	49.9
SLE (ViT-L)	322	61.3	52.6

Table 8: DINOv2 model size and labeller performance on Foggy Cityscapes (FC) and BDD100K Daytime (BD).

Figure 5: Class-wise correct-pseudo-label ratio for DINOv2 and SLE-enhanced labellers in Foggy Cityscapes.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Model	FC	BD	ACDC
Model	FC	BD	Fog	Night	Rain	Snow
DINOv2-G	57.0	50.7	65.9	36.4	45.8	52.0
SLE-enhanced DINOv2-B	59.9	49.9	72.4	36.5	49.8	60.0
SLE-enhanced DINOv2-L	61.3	52.6	70.5	38.7	48.2	60.5
SLE-enhanced DINOv3-B	59.5	49.7	73.3	37.6	52.5	60.4

Appendix

Table 9: Labeller performance (

\mathrm{mAP}_{50}

, %) with different backbones on Foggy Cityscapes (FC), BDD100K Daytime (BD), and ACDC.

Method	FA	Pse	$\mathrm{mAP}_{50}$
Baseline (VGG3)	–	–	38.7
Baseline (VGG4)	–	–	40.1
DINO Teacher	$\checkmark$	–	42.6
	–	$\checkmark$	53.2
	$\checkmark$	$\checkmark$	52.5
SLE Teacher (Ours)	$\checkmark$	–	48.9

Appendix

Table 10: Ablation study of feature alignment and pseudo-label learning with different teacher models on Foggy Cityscapes. All experiments use pseudo labels generated by SLE-enhanced Dinov2-B.

Figure 6: Qualitative detection results on Foggy Cityscapes. From top to bottom, the rows show the input image, predictions from DINO Teacher (DT), predictions from SLE-T instantiated with DINOv2-L, and ground truth (GT).

Figure 7: Labeller performance versus model size on Foggy Cityscapes (FC) and BDD100K Daytime (BD). The parameter axis uses a logarithmic scale.

Weather Condition	Method	Person	Rider	Car	Truck	Bus	Train	Motor	Bicycle	mAP
Fog	DT (ViT-G)	59.9	62.0	78.6	40.0	81.8	70.1	36.5	67.5	62.0
	SLE-T (ViT-B)	64.9	70.6	83.5	33.5	100.0	63.6	44.2	69.8	66.3
	SLE-T (ViT-L)	68.6	59.0	84.1	41.8	97.0	63.6	47.0	66.8	66.0
Night	DT (ViT-G)	37.1	21.2	58.1	35.6	–	40.8	24.3	15.7	33.3
	SLE-T (ViT-B)	40.4	35.5	57.2	30.9	–	45.2	24.2	10.8	34.9
	SLE-T (ViT-L)	39.0	30.6	59.3	37.6	–	47.7	19.4	13.0	35.2

Appendix

Table 11: Class-wise performance of the Cityscapes

\rightarrow

ACDC results reported in the main paper (

\mathrm{AP}_{50}

, %).

Object Detection

Object detection locates and classifies objects in images or video, balancing localization accuracy, robustness, and inference speed.

See all 605 papers

Generalizable Knowledge Distillation from Vision Foundation Models for Semantic Segmentation
Mar 3, 2026Chonghua Lv, Dong Zhao, Shuang Wang +4Domain Generalization for Semantic Segmentation Knowledge Distillation
School of Artificial Intelligence, Xidian University, China · Department of Information Engineering and Computer Science, University of Trento, Italy · Department of Automation, Tsinghua University, China +1
Training a Student Expert via Semi-Supervised Foundation Model Distillation
Apr 4, 2026Pardis Taghavi, Tian Liu, Renjie Li +2Knowledge Distillation Instance Segmentation
Texas A&M University
VFM $^{4}$ SDG: Unveiling the Power of VFMs for Single-Domain Generalized Object Detection
Apr 23, 2026Yupeng Zhang, Ruize Han, Ningnan Guo +3Detection Transformer Cross-Domain Generalization
College of Intelligence and Computing, Tianjin University, Tianjin 300350, China · Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology, Shenzhen 518107, Guangdong, China

Mar 3, 2026J/K move · Enter open · S save

Generalizable Knowledge Distillation from Vision Foundation Models for Semantic Segmentation

Chonghua Lv, Dong Zhao, Shuang Wang, Dou Quan+3

School of Artificial Intelligence, Xidian University, China · Department of Information Engineering and Computer Science, University of Trento, Italy · Department of Automation, Tsinghua University, China+1

Domain Generalization for Semantic Segmentation Knowledge Distillation OOD Generalization Feature-Based Knowledge Distillation Domain Generalization Selective Knowledge Distillation+1

Open paper AskPDF arXiv

Knowledge distillation (KD) has been widely applied in semantic segmentation to compress large models, but conventional approaches primarily preserve in-domain accuracy while neglecting out-of-domain generalization, which is essential under distribution shifts. This limitation becomes more severe with the emergence of vision foundation models (VFMs): although VFMs exhibit strong robustness on unseen data, distilling them with conventional KD often compromises this ability. We propose Generalizable Knowledge Distillation (GKD), a multi-stage framework that explicitly enhances generalization. GKD decouples representation learning from task learning. In the first stage, the student acquires domain-agnostic representations through selective feature distillation, and in the second stage, these representations are frozen for task adaptation, thereby mitigating overfitting to visible domains. To further support transfer, we introduce a query-based soft distillation mechanism, where student features act as queries to teacher representations to selectively retrieve transferable spatial knowledge from VFMs. Extensive experiments on five domain generalization benchmarks demonstrate that GKD consistently outperforms existing KD methods, achieving average gains of +1.9% in foundation-to-foundation (F2F) and +10.6% in foundation-to-local (F2L) distillation. The code will be available at https://github.com/Younger-hua/GKD.

Abstract

Object Detection

Explore similar work

Generalizable Knowledge Distillation from Vision Foundation Models for Semantic Segmentation

Training a Student Expert via Semi-Supervised Foundation Model Distillation

VFM4^{4}4SDG: Unveiling the Power of VFMs for Single-Domain Generalized Object Detection

VFM $^{4}$ SDG: Unveiling the Power of VFMs for Single-Domain Generalized Object Detection