Cross-domain few-shot object detection (CD-FSOD) aims to adapt pretrained detectors from a source domain to target domains with limited annotations, suffering from severe domain shifts and data scarcity problems. In this work, we find a previously overlooked phenomenon: models exhibit dispersed and unfocused attention in target domains, leading to imprecise localization and redundant predictions, just like a human cannot focus on visual objects. Therefore, we call it the target-domain Astigmatism problem. Analysis on attention distances across transformer layers reveals that regular fine-tuning inherently shows a trend to remedy this problem, but results are still far from satisfactory, which we aim to enhance in this paper. Biologically inspired by the human fovea-style visual system, we enhance the fine-tuning's inherent trend through a center-periphery attention refinement framework, which contains (1) a Positive Pattern Refinement module to reshape attention toward semantic objects using class-specific prototypes, simulating the visual center region; (2) a Negative Context Modulation module to enhance boundary discrimination by modeling background context, simulating the visual periphery region; and (3) a Textual Semantic Alignment module to strengthen center-periphery distinction through cross-modal cues. Our bio-inspired approach transforms astigmatic attention into focused patterns, substantially improving adaptation to target domains. Experiments on six challenging CD-FSOD benchmarks consistently demonstrate improved detection accuracy and establish new state-of-the-art results.
Figures & tables
Figure 1: Visualization and quantification of the target-domain Astigmatism problem. Top: Attention maps measured across transformer blocks show that, in source domains (first row), attention progressively focuses on foreground objects, while in target domains (second row), attention remains persistently dispersed, resulting in oversized boxes and redundant predictions in object detection. Bottom: Attention distance across network depth ( SxBy denotes Stage x, Block y in the Swin Transformer) reveals: (1) a observed rise-then-fall trend in attention distance, reflecting an initial broad attention that gradually concentrates on objects for precise localization; (2) consistently higher attention dispersion in target domains compared to source domains; and (3) regular fine-tuning only marginally reduces this attention dispersion.
Figure 2: The inspiration from human fovea-style vision to remedy the Astigmatism problem. Our method mimics the human visual system: the Core Perception Zone (green) with high-detail processing guides the Positive Pattern Refinement (PPR) module to reshape attention more effectively toward foreground objects, while the Peripheral Zone (orange) with reduced details informs the Negative Context Modulation (NCM) module to enhance object-background boundaries by modeling background contexts. The Textual Semantic Alignment (TSA) module enforces the distinctions between center and peripheral regions, analogous to the center-surround mechanism of biological perception.
Figure 3: Overview of our human-vision-inspired framework to remedy the Astigmatism problem in CD-FSOD. The architecture integrates three complementary modules: (1) Positive Pattern Refinement (PPR) reshapes attention toward foreground objects using class prototypes; (2) Negative Context Modulation (NCM) enhances object-background boundaries through explicit background modeling; and (3) Textual Semantic Alignment (TSA) enhances these distinctions via cross-modal knowledge integration with negative descriptors (“not [class]”). During training (top), the model is optimized with both detection and alignment objectives, and extracts discriminative prototypes from support examples, stored in positive and negative repositories. At inference (bottom), stored prototypes turn dispersed attention patterns in query images into crystallized object-centric representations, analogous to the human center-peripheral visual system.
Figure 4: Attention distribution around a foreground patch x0 . Dashed arrows labeled Ak indicate attention from x0 to neighbors xk . Same‑object neighbors x2,x3 are spatially near and should receive high attention, whereas the background patch x1 is distant and should be weak. In target‑domain Astigmatism, domain shift diverts attention toward the distant background ( A1↑ ; A2,A3↓ ), yielding dispersed attention and a larger attention distance. Since the spatial distance between patches are fixed, our method reverses this dispersion by (i) down‑weighting background responses A1 via a learned background prototype and simple “not [class]” cues (NCM/TSA module) to suppress spurious foreground–background affinity, and (ii) up‑weighting same‑object responses A2,A3 via class‑specific foreground prototypes (PPR module) to strengthen intra‑object compatibility, thereby shortening attention distance and restoring a focused, object‑centric pattern.
Method
Backbone
Venue
ArTaxOr
Clipart1k
DIOR
DeepFish
NEU-DET
UODD
Avg.
Distill † [ 32 ]
ResNet50
ICASSP’23
5.10
7.60
10.50
–
–
5.90
/
ViTDet † [ 16 ]
ViT-B/14
ECCV’22
5.90
6.10
12.90
0.90
2.40
4.00
5.37
ViTDet* [ 16 ]
Swin-T
ECCV’22
8.85
10.22
18.50
1.06
4.54
4.90
8.01
Detic † [ 46 ]
ViT-L/14
ECCV’22
3.20
15.10
4.10
9.00
3.80
4.20
6.57
Detic* [ 46 ]
Swin-T
ECCV’22
4.45
19.22
5.70
10.49
7.15
5.12
8.69
DE-ViT † [ 36 ]
ViT-L/14
arXiv’23
10.50
13.00
14.70
19.30
0.60
2.40
10.08
Table 1: 1-shot cross-domain detection performance comparison (mAP). † indicates results reported in CD-ViTO [ 5 ] . Methods with * are reimplemented by us using Swin-Tiny backbone. GLIP and our method are also implemented with Swin-Tiny backbone.
Method
Backbone
Venue
ArTaxOr
Clipart1k
DIOR
DeepFish
NEU-DET
UODD
Avg.
Distill † [ 32 ]
ResNet50
ICASSP’23
12.50
23.30
19.10
15.50
16.00
12.20
16.43
ViTDet † [ 16 ]
ViT-B/14
ECCV’22
20.90
23.30
23.30
9.00
13.50
11.10
16.85
ViTDet* [ 16 ]
Swin-T
ECCV’22
21.05
23.45
23.45
11.03
19.96
15.07
19.00
Detic † [ 46 ]
ViT-L/14
ECCV’22
8.70
20.20
12.10
14.30
14.10
10.40
13.30
Detic* [ 46 ]
Swin-T
ECCV’22
10.63
20.31
14.78
17.46
20.80
14.07
16.34
DE-ViT † [ 36 ]
ViT-L/14
arXiv’23
38.00
38.10
23.40
21.20
7.80
5.00
22.25
Table 2: 5-shot cross-domain detection performance comparison (mAP). † indicates results reported in CD-ViTO [ 5 ] . Methods with * are reimplemented by us using Swin-Tiny backbone. GLIP and our method are also implemented with Swin-Tiny backbone.
Method
Backbone
Venue
ArTaxOr
Clipart1k
DIOR
DeepFish
NEU-DET
UODD
Avg.
Distill † [ 32 ]
ResNet50
ICASSP’23
18.10
27.30
26.50
15.50
21.10
14.50
20.50
ViTDet † [ 16 ]
ViT-B/14
ECCV’22
23.40
25.60
29.40
6.50
15.80
15.60
19.38
ViTDet* [ 16 ]
Swin-T
ECCV’22
24.20
25.93
30.10
8.04
22.52
19.55
21.72
Detic † [ 46 ]
ViT-L/14
ECCV’22
12.00
22.30
15.40
17.90
16.80
14.40
16.47
Detic* [ 46 ]
Swin-T
ECCV’22
13.80
22.59
17.20
22.04
23.97
16.58
19.36
DE-ViT † [ 36 ]
ViT-L/14
arXiv’23
49.20
40.80
25.60
21.30
8.80
5.40
25.18
Table 3: 10-shot cross-domain detection performance comparison (mAP). † indicates results reported in CD-ViTO [ 5 ] . Methods with * are reimplemented by us using Swin-Tiny backbone. GLIP and our method are also implemented with Swin-Tiny backbone.
Dataset
Classes
Images
Boxes
Domain
Challenge
ArTaxOr
7
1383
1628
photorealistic
small objects
Clipart1k
20
500
1526
cartoon
style abstraction
DIOR
20
5000
28810
aerial
medium-scale
DeepFish
1
909
3029
aquatic
poor visibility
NEU-DET
6
360
834
industrial
subtle features
UODD
3
506
3218
underwater
complex clutter
Table 4: Summary of target datasets used in our evaluation.
Components
Datasets
NCM
PPR
TSA
ArTaxOr
Clipart1k
DIOR
DeepFish
NEU-DET
UODD
48.11
39.28
27.39
28.40
19.55
11.40
✓
49.95
40.52
27.75
29.58
20.48
12.21
✓
✓
52.68
42.95
28.62
31.96
22.38
14.26
✓
✓
✓
54.98
44.83
29.41
33.87
23.64
15.66
Table 5: Ablation study showing each component’s contribution to overall performance (mAP %) in 5-shot setting.
Figure 5: Top: Visualization of attention maps demonstrating the Astigmatism problem and our solution. Each row displays two different samples: including the original image with ground truth (left), conventional fine-tuning that only marginally alleviates dispersed attention patterns (middle), and ours, which indicates focused attention that precisely concentrates on target regions. Bottom: Change in attention distance relative to pretrained model across datasets. Negative values indicate reduction in attention dispersion (larger magnitude means better focus). While fine-tuning reduces dispersion, our method achieves substantially greater reductions, validating our superior ability to address the Astigmatism problem by reshaping attention toward foreground objects.
Figure 6: Validation of Negative Context Modulation (NCM) module through background ratio analysis. Top: Performance comparison with varying background ratios. Our method demonstrates consistent advantages over the baseline, particularly at higher background ratios (0.3-1.0), indicating NCM exploits negative background contexts as complementary cues to strengthen foreground feature learning and boundary discrimination , thereby mitigating the attention dispersion in Astigmatism problem. Bottom: Visual examples showing detection with increasing background ratios from left to right (R=0.0 to R=1.0).
Figure 7: Qualitative detection results on cross-domain scenarios. Our method produces significantly more precise predictions with fewer candidate boxes compared to alternatives. The numbers indicate displayed boxes/total generated boxes, demonstrating our method’s ability to reduce redundant detections while maintaining precision . In the maritime scene (top), our approach avoids misclassifying background as objects, while in the transportation (middle) and indoor (bottom) examples, it generates tighter bounding boxes with minimal redundancy. These results validate our framework’s effectiveness in remedying the Astigmatism problem.
Text Count
Performance
GPU Mem. (MB) ↓
Eff. Ratio ↑
AP (%) ↑
Gain ↑
50
39.34
-
8762
-
100
40.31
+0.97%
8850
1.10%
150
41.48
+1.17%
8943
1.26%
200
42.87
+1.39%
9027
1.65%
250
43.47
+0.60%
9108
0.74%
Table 6: Effect of background text quantity on performance. Eff. Ratio = Δ AP / Δ Mem., indicating AP gain per MB increase.
Figure 8: Analysis of key parameters in our approach. Left : Effect of TSA loss weight ( λ ) on detection performance. Performance remains stable when λ≤1 , peaks at λ=103 , and declines beyond that, underscoring the importance of appropriate weighting in enhancing object-background separation. Right : Impact of threshold selection on NCM effectiveness. Performance remains above baseline across a wide threshold range (0.6-0.9), with optimal results at 0.75, demonstrating that selectively enhancing background features effectively reinforces object-background boundaries.
Figure 9: Parameter analysis of temperature-weight interactions in PPR. The heatmap shows peak performance at moderate weights (0.1-1.0) with robust temperature tolerance, while extreme weights ( ≥ 1.0) cause significant performance degradation. This confirms that balanced prototype utilization is essential for effective attention refinement without disrupting original feature representations.
Few-shot object detection (FSOD) is challenging due to unstable optimization and limited generalization arising from the scarcity of training samples. To address these issues, we propose a hybrid ensemble decoder that enhances generalization during fine-tuning. Inspired by ensemble learning, the decoder comprises a shared hierarchical layer followed by multiple parallel decoder branches, where each branch employs denoising queries either inherited from the shared layer or newly initialized to encourage prediction diversity. This design fully exploits pretrained weights without introducing additional parameters, and the resulting diverse predictions can be effectively ensembled to improve generalization. We further leverage a unified progressive fine-tuning framework with a plateau-aware learning rate schedule, which stabilizes optimization and achieves strong few-shot adaptation without complex data augmentations or extensive hyperparameter tuning. Extensive experiments on CD-FSOD, ODinW-13, and RF100-VL validate the effectiveness of our approach. Notably, on RF100-VL, which includes 100 datasets across diverse domains, our method achieves an average performance of 41.9 in the 10-shot setting, significantly outperforming the recent approach SAM3, which obtains 35.7. We further construct a mixed-domain test set from CD-FSOD to evaluate robustness to out-of-distribution (OOD) samples, showing that our proposed modules lead to clear improvement gains. These results highlight the effectiveness, generalization, and robustness of the proposed method. Code is available at: https://github.com/Intellindust-AI-Lab/FT-FSOD.
Xuanlong Yu, Youyang Sha, Longfei Liu +2
Intellindust AI Lab · Suzhou Institute for Advanced Research, USTC
Data augmentation, which simulates diverse visual variations to expand the source distribution and induce synthetic domain shifts, is a simple yet effective strategy for mitigating severe domain shifts and limited labeled target data in cross-domain few-shot object detection (CD-FSOD). Existing approaches rely on conventional data augmentation, such as Color-Jitter, Mosaic, and background-centric adaptation (e.g., Domain-RAG), which are limited in modeling complex domain shifts and often lead to suboptimal performance. In this paper, we propose PSP-FSOD, a principled framework that integrates prompt-driven domain simulation with feature perturbation regularization to improve generalization in CD-FSOD. To enable controllable domain synthesis, we design a prompt-driven strategy that leverages the visual grounding capability of large VLMs to jointly model foreground and background variations, generating semantically consistent yet domain-diverse training samples. Moreover, we adopt a grounding-aware generation scheme that guides object placement and alleviates semantic-spatial misalignment, thereby improving foreground adaptation. To ensure training stability and robustness, we further introduce a noise-induced feature perturbation mechanism that injects Gaussian noise into multi-scale intermediate features with distribution correction, encouraging consistent predictions under perturbations and reducing reliance on domain-specific cues. Extensive experiments demonstrate that PSP-FSOD produces high-quality domain-diverse supervision and learns domain-invariant representations, consistently improving performance across CD-FSOD benchmarks.
Linhai Zhuo, Junxi Cai, Tianwen Qian +2
Fuzhou University · East China Normal University · Xiamen University +1
Cross-Domain Few-Shot Object Detection (CD-FSOD) aims to detect novel classes in unseen target domains given only a few labeled examples. While open-vocabulary detectors built on vision-language models (VLMs) transfer well, they depend almost entirely on text prompts, which encode domain-invariant semantics but miss domain-specific visual information needed for precise localization under few-shot supervision. We propose a dual-branch detector that Learns Multi-modal Prototypes, dubbed LMP, by coupling textual guidance with visual exemplars drawn from the target domain. A Visual Prototype Construction module aggregates class-level prototypes from support RoIs and dynamically generates hard-negative prototypes in query images via jittered boxes, capturing distractors and visually similar backgrounds. In the visual-guided branch, we inject these prototypes into the detection pipeline with components mirrored from the text branch as the starting point for training, while a parallel text-guided branch preserves open-vocabulary semantics. The branches are trained jointly and ensembled at inference by combining semantic abstraction with domain-adaptive details. On six cross-domain benchmark datasets and standard 1/5/10-shot settings, our method achieves state-of-the-art or highly competitive mAP.
Wanqi Wang, Jingcai Guo, Yuxiang Cai +1
University of Chinese Academy of Sciences · The Hong Kong Polytechnic University · Zhejiang University +1