A Closer Look at Cross-Domain Few-Shot Object Detection: Fine-Tuning Matters and Parallel Decoder Helps

Xuanlong Yu; Youyang Sha; Longfei Liu; Xi Shen; Di Yang

cs.CVMar 30, 2026

A Closer Look at Cross-Domain Few-Shot Object Detection: Fine-Tuning Matters and Parallel Decoder Helps

Authors: Xuanlong Yu, Youyang Sha, Longfei Liu, Xi Shen, Di Yang

Organizations: Intellindust AI Lab · Suzhou Institute for Advanced Research, USTC

Abstract

Few-shot object detection (FSOD) is challenging due to unstable optimization and limited generalization arising from the scarcity of training samples. To address these issues, we propose a hybrid ensemble decoder that enhances generalization during fine-tuning. Inspired by ensemble learning, the decoder comprises a shared hierarchical layer followed by multiple parallel decoder branches, where each branch employs denoising queries either inherited from the shared layer or newly initialized to encourage prediction diversity. This design fully exploits pretrained weights without introducing additional parameters, and the resulting diverse predictions can be effectively ensembled to improve generalization. We further leverage a unified progressive fine-tuning framework with a plateau-aware learning rate schedule, which stabilizes optimization and achieves strong few-shot adaptation without complex data augmentations or extensive hyperparameter tuning. Extensive experiments on CD-FSOD, ODinW-13, and RF100-VL validate the effectiveness of our approach. Notably, on RF100-VL, which includes 100 datasets across diverse domains, our method achieves an average performance of 41.9 in the 10-shot setting, significantly outperforming the recent approach SAM3, which obtains 35.7. We further construct a mixed-domain test set from CD-FSOD to evaluate robustness to out-of-distribution (OOD) samples, showing that our proposed modules lead to clear improvement gains. These results highlight the effectiveness, generalization, and robustness of the proposed method. Code is available at: https://github.com/Intellindust-AI-Lab/FT-FSOD.

Figures & tables

Figure 1: Performance on large-scale cross-domain FSOD benchmarks. All methods are adapted from pretrained models. Our approach, based on open-source MMGroundingDINO (MMGDINO) [ 61 ] , surpasses prior SOTA (Domain-RAG [ 26 ] , MQ-GLIP-L [ 54 ] and MMGDINO-L [ 61 ] ) on CD-FSOD [ 12 ] , ODinW-13 [ 57 ] and RF100-VL [ 45 ] benchmarks, and achieves comparable results to fine-tuned SAM3 [ 2 ] , notably outperforming it on the largest RF100-VL benchmark.

Figure 2: Overview of the proposed hybrid ensemble decoder. Our method extends the original decoder in a transformer-based object detector by parallelizing partial decoder layers on top of the model for the FSOD task. The final detection result is the aggregation of the results given by the object query outputs from all decoder layers. We also randomly replace the original denoising query with the initialized ones during training to further introduce diversity for the parallelized decoder layer input.

	Method	Backbone	ArTaxOr	Clipart1k	DIOR	DeepFish	NEU-DET	UODD	Avg.
0-shot	MMGDINO-B [ 61 ]	Swin-B [ 32 ]	7.3	48.8	5.3	42.0	1.2	20.0	20.8
1-shot	DE-ViT-FT $\dagger$ [ 60 ]	DINOv2-L [ 35 ]	10.5	13.0	14.7	19.3	0.6	2.4	10.1
	CD-ViTO [ 12 ]	DINOv2-L [ 35 ]	21.0	17.7	17.8	20.3	3.6	3.1	13.9
	VFMDETR [ 28 ]	DINOv2-L [ 35 ]	26.1	20.1	20.6	24.2	9.1	9.0	18.2
	CDFormer [ 34 ]	DINOv2-L [ 35 ]	36.0	54.0	16.3	34.5	7.4	12.7	26.8
	ETS [ 37 ]	Swin-B [ 32 ]	28.1	57.5	12.7	40.7	11.7	21.2	28.7

Table 1: Comparison of results (mAP

\uparrow

) of different models and strategies on the CD-FSOD [ 53 ] benchmark under the 1/5/10-shot setting. Best and second-best results are highlighted in bold and underlined, respectively.

\dagger

marks the results reproduced in CD-ViTO [ 12 ] .

Model	Backbone	Ckpt avail.	0-shot	1-shot	3-shot	5-shot	10-shot
GLIPv2-H [ 57 ]	Swin-H [ 32 ]	✗	55.5	$61.7\pm 0.5$	$64.1\pm 0.8$	$64.4\pm 0.6$	$65.9\pm 0.3$
GLEE-Pro [ 52 ]	EVA02-L [ 9 ]	✓	53.4	$59.4\pm 1.5$	$61.7\pm 0.5$	$64.3\pm 1.3$	$65.6\pm 0.4$
MQ-GLIP-L [ 54 ]	Swin-L [ 32 ]	✓	54.1	62.4	64.2	65.4	66.6
Grounding DINO 1.5 Pro [ 43 ]	EVA02-L [ 9 ]	✗	58.7	$62.4\pm 1.1$	$\underline{66.3}\pm 1.0$	$66.9\pm 0.2$	$67.9\pm 0.3$
SAM3 [ 2 ]	PE-L+ [ 1 ]	✓	59.9	$63.0\pm 2.4$	$\textbf{67.3}\pm 0.7$	$\textbf{69.1}\pm 1.1$	$\textbf{71.6}\pm 0.2$
Ours (MMGDINO-L [ 61 ] )	Swin-L [ 32 ]	✓	55.3	$\textbf{63.1}\pm 0.6$	$65.8\pm 0.7$	$\underline{67.6}\pm 0.3$	$\underline{68.6}\pm 0.6$

Table 2: Comparison of results (mAP

\uparrow

) of different models and strategies on ODinW-13 [ 25 ] benchmark under 1/3/5/10-shot setting. 0-shot performance is for reference. The results are averaged by three official seeds of runs. Best and second-best results are highlighted in bold and underlined, respectively.

Model	Backbone	Aerial	Document	Flora-Fauna	Industrial	Medical	Sports	Other	Avg.
Detic [ 60 ]	Swin-L [ 32 ]	12.2 / 19.5	4.5 / 19.6	17.9 / 28.4	6.0 / 25.9	0.8 / 8.5	7.6 / 26.6	11.2 / 25.7	9.5 / 22.8
SAM3 [ 2 ]	PE-L+ [ 1 ]	20.7 / 33.9	11.9 / 34.0	23.4 / 38.8	8.2 / 39.7	2.0 / 24.9	16.4 / 40.3	15.7 / 34.7	14.3 / 35.7
MMGDINO-L [ 61 ]	Swin-L [ 32 ]	21.5 / 32.4	9.2 / 30.6	27.9 / 41.3	10.3 / 37.8	2.1 / 18.3	13.3 / 33.2	17.5 / 32.0	15.7 / 33.6
Ours (MMGDINO-L)	Swin-L [ 32 ]	21.5 / 41.5	9.2 / 38.9	27.9 / 45.9	10.3 / 46.7	2.1 / 25.8	13.3 / 36.2	17.5 / 47.2	15.7 / 41.9

Table 3: Comparison of results (mAP

\uparrow

) of different models and strategies on RF100-VL [ 45 ] benchmark . The results are illustrated as 0/10-shot setting. 0-shot performance is for completeness. Best and second-best results are highlighted in bold and underlined, respectively.

Baseline (Naive aug & Plateau scheduler)		✓	✓	✓	✓
Progressive Fine-tuning			✓		✓
Hybrid Ensemble Decoder				✓	✓
CD-FSOD (0-shot: 20.8)	1-shot	30.8	33.3	33.1	34.9
	5-shot	43.1	44.6	43.4	45.0
	10-shot	47.1	47.1	47.2	47.9
ODinW-13 (0-shot: 55.3)	1-shot	60.7	61.8	62.0	63.1

Table 4: Ablation study on the proposed approach on CD-FSOD [ 53 ] and ODinW-13 [ 25 ] under 1/5/10-shot settings. The results are averaged by three official seeds of runs on ODinW-13. Best results are highlighted in bold.

Figure 3: Illustration on performance reduction of different fine-tuning strategies when the test set contains OOD samples. The performance reductions are highlighted in bold red, and the most robust results are underlined.

Decoder Structure	DnQ re-init rate $\tau$ (Eq. 7 )	CD-FSOD		ODinW-13
Decoder Structure	DnQ re-init rate $\tau$ (Eq. 7 )	1-shot	5-shot	1-shot	5-shot
6-stacked (Baseline+prog. ft.) –		33.3	44.6	61.8	66.1
6-parallel	0	27.4	40.0	58.4	63.6
4-stacked+2-parallel	0	33.9	44.8	60.3	66.6
3-stacked+3-parallel	0	32.9	44.3	60.0	67.3
2-stacked+4-parallel	0	34.2	44.3	61.9	66.4

Table 5: Ablation study on proposed hybrid ensemble decoder. Experiments are conducted on CD-FSOD [ 53 ] and ODinW-13 [ 25 ] (only on seed3 subset) 1/5-shot settings . Best results are in bold.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Hyperparameters
Init. learning-rate for detection transformers	1e-4
Init. learning-rate for image and language backbones	2e-5
Minimum learning rate	1e-6
Weight decay	0.05
Batch size	4
Number of epochs	100

Appendix

Table 6: Hyperparameter list applied for training on all benchmark experiments.

CD-FSOD \| CD-Mixed	1-shot	5-shot	10-shot
Randomly Initialized	33.7 \| 25.04	44.1 \| 36.08	47.2 \| 38.02
Pre-trained	34.9 \| 25.68	45.0 \| 37.14	47.9 \| 40.34

Appendix

Table 7: Fine-tuning results with and without randomly initialized parallel layers for HED.

CD-FSOD \| CD-Mixed	1-shot	5-shot	10-shot
1-stage (cosine) + HED	34.6 \| 24.92	43.6 \| 33.52	47.2 \| 38.30
2-stage (cosine) + HED	34.6 \| 25.46	44.2 \| 35.34	47.4 \| 38.42
2-stage (plateau) + HED (Ours)	34.9 \| 25.68	45.0 \| 37.14	47.9 \| 40.34

Appendix

Table 8: Fine-tuning results for the model using cosine and plateau scheduler with and without our progressive fine-tuning strategies.

Datasets	1-shot
AerialMaritimeDrone	26.267
Aquarium	47.500
CottontailRabbits	76.967
EgoHands	67.133
NorthAmericaMushrooms	81.067
Packages	65.833

Appendix

Table 9: Average results of ODinW-13 of the three official seeds of runs for 1-shot setting.

Datasets	3-shot
AerialMaritimeDrone	34.033
Aquarium	52.767
CottontailRabbits	77.067
EgoHands	69.033
NorthAmericaMushrooms	83.567
Packages	67.733

Appendix

Table 10: Average results of ODinW-13 of the three official seeds of runs for 3-shot setting.

Datasets	5-shot
AerialMaritimeDrone	38.200
Aquarium	53.733
CottontailRabbits	76.533
EgoHands	72.767
NorthAmericaMushrooms	87.000
Packages	70.833

Appendix

Table 11: Average results of ODinW-13 of the three official seeds of runs for 5-shot setting.

Datasets	10-shot
AerialMaritimeDrone	39.367
Aquarium	55.167
CottontailRabbits	75.200
EgoHands	71.900
NorthAmericaMushrooms	88.600
Packages	73.033

Appendix

Table 12: Average results of ODinW-13 of the three official seeds of runs for 10-shot setting.

Aerial Datasets	10-shot
aerial-airport	48.1
aerial-cows	34.4
aerial-sheep	48.2
apoce-aerial-photographs -for-object-detection- of-construction-equipment	46.2
electric-pylon-detection-in-rsi	21.1
floating-waste	36.5

Appendix

Table 13: Results on RF100-VL - Aerial category for 10-shot Performance (mAP)

Document Datasets	10-shot
activity-diagrams	30.4
all-elements	39.5
circuit-voltages	34.5
invoice-processing	23.7
label-printing-defect-version-2	54.1
macro-segmentation	34.1

Appendix

Table 14: Results on RF100-VL - Document category for 10-shot Performance (mAP)

Flora-Fauna Datasets	10-shot
aquarium-combined	52.6
bees	30.9
deepfruits	64.9
exploratorium-daphnia	26.7
grapes-5	39.3
grass-weeds	46.7

Appendix

Table 15: Results on RF100-VL - Flora-Fauna category for 10-shot Performance (mAP)

Industrial Datasets	10-shot
-grccs	50.2
13-lkc01	32.6
2024-frc	62.0
aircraft-turnaround-dataset	35.4
asphaltdistressdetection	21.2
cable-damage	24.2

Appendix

Table 16: Results on RF100-VL - Industrial category for 10-shot Performance (mAP)

Medical Datasets	10-shot
canalstenosis	46.6
crystal-clean-brain-tumors-mri-dataset	56.1
dentalai	20.4
inbreast	34.2
liver-disease	19.7
nih-xray	8.8

Appendix

Table 17: Results on RF100-VL - Medical category for 10-shot Performance (mAP)

Other Datasets	10-shot
buoy-onboarding	26.1
car-logo-detection	73.9
clashroyalechardetector	40.6
cod-mw-warzone	46.4
countingpills	85.8
everdaynew	50.0

Appendix

Table 18: Results on RF100-VL - Other category for 10-shot Performance (mAP)

Baseline + Progressive ft. + HED w/ re-init Dn Query	1-shot		5-shot		10-shot
Baseline + Progressive ft. + HED w/ re-init Dn Query	w/o OOD	w/ OOD	w/o OOD	w/ OOD	w/o OOD	w/ OOD
ArTaxOr	49.1	44	76.8	73.2	79.2	75.7
DIOR	24.6	23.6	35.3	33.8	41.5	40.3
NEU-DET	15.5	5.3	25.2	14.2	28.4	15.7
UODD	22.1	13.3	27.5	17.0	32.3	25.7
clipart1k	55.6	42.2	59.4	47.5	59.6	44.3

Appendix

Table 19: Full results on the prediction performance (mAP) on the clean target test set and the proposed CD-Mixed test set, with the performance reduction in percentage. The best results are highlighted in bold.

Few-Shot Object Detection

Few-shot object detection adapts detectors to recognize and localize new object classes from only a handful of labeled examples.

See all 27 papers

GiPL: Generative augmented iterative Pseudo-Labeling for Cross-Domain Few-Shot Object Detection
May 28, 2026Jiacong Liu, Shu Luo, Yikai Qin +3Synthetic Data Augmentation Few-Shot Domain Adaptation
Huazhong University of Science and Technology
Prompt-Driven Simulation with Feature Perturbation for Cross-Domain Few-Shot Object Detection
Aug 2, 2026Linhai Zhuo, Junxi Cai, Tianwen Qian +2Synthetic Data Augmentation Few-Shot Domain Adaptation
Fuzhou University · East China Normal University · Xiamen University +1
Geometry over Density: Few-Shot Cross-Domain OOD Detection
May 5, 2026Shawn Li, You Qin, Jiate Li +4Diffusion-Based OOD Detection Distribution Shift Detection
University of Southern California · National University of Singapore · Amazon

May 28, 2026J/K move · Enter open · S save

GiPL: Generative augmented iterative Pseudo-Labeling for Cross-Domain Few-Shot Object Detection

Jiacong Liu, Shu Luo, Yikai Qin, Yaze Zhao+2

Huazhong University of Science and Technology

Synthetic Data Augmentation Few-Shot Domain Adaptation Vision Foundation Model Adaptation Few-Shot Object Detection Few-Shot Learning Pseudo-Labeling+1

Open paper AskPDF arXiv

Vision-language foundation models have shown promising zero-shot generalization for Cross-Domain Few-Shot Object Detection (CD-FSOD). However, they face two critical challenges in fine-tuning: insufficient support set utilization due to sparse single-instance annotations, and severe overfitting under extremely limited target-domain samples. To address these issues, this paper proposes GiPL, an efficient two-branch training framework. In the first branch, we design an iterative pseudo-label self-training paradigm, which performs zero-shot inference on the support set to generate reliable pseudo-annotations, fuses them with ground-truth labels, and iteratively optimizes the model to fully exploit support set data. In the second branch, we introduce generative data augmentation pipeline using large vision-language models, which synthesizes domain-aligned, multi-object annotated images to enrich training samples and suppress overfitting. Extensive experiments on three challenging CD-FSOD datasets (RUOD, CARPK, CarDD) under 1/5/10-shot settings demonstrate that GiPL consistently outperforms state-of-the-art methods with significant performance gains. Code is available at CDiscover.

CD-FSOD \| CD-Mixed	1-shot	5-shot	10-shot
Randomly Initialized	33.7 \| 25.04	44.1 \| 36.08	47.2 \| 38.02
Pre-trained	34.9 \| 25.68	45.0 \| 37.14	47.9 \| 40.34

CD-FSOD \| CD-Mixed	1-shot	5-shot	10-shot
1-stage (cosine) + HED	34.6 \| 24.92	43.6 \| 33.52	47.2 \| 38.30
2-stage (cosine) + HED	34.6 \| 25.46	44.2 \| 35.34	47.4 \| 38.42
2-stage (plateau) + HED (Ours)	34.9 \| 25.68	45.0 \| 37.14	47.9 \| 40.34

Abstract

Few-Shot Object Detection

Explore similar work

GiPL: Generative augmented iterative Pseudo-Labeling for Cross-Domain Few-Shot Object Detection

Prompt-Driven Simulation with Feature Perturbation for Cross-Domain Few-Shot Object Detection

Geometry over Density: Few-Shot Cross-Domain OOD Detection