Remedying Target-Domain Astigmatism for Cross-Domain Few-Shot Object Detection

Yongwei Jiang; Yixiong Zou; Yuhua Li; Ruixuan Li

Figure 1: Visualization and quantification of the target-domain Astigmatism problem. Top: Attention maps measured across transformer blocks show that, in source domains (first row), attention progressively focuses on foreground objects, while in target domains (second row), attention remains persistently dispersed, resulting in oversized boxes and redundant predictions in object detection. Bottom: Attention distance across network depth ( SxBy denotes Stage x, Block y in the Swin Transformer) reveals: (1) a observed rise-then-fall trend in attention distance, reflecting an initial broad attention that gradually concentrates on objects for precise localization; (2) consistently higher attention dispersion in target domains compared to source domains; and (3) regular fine-tuning only marginally reduces this attention dispersion.

Figure 2: The inspiration from human fovea-style vision to remedy the Astigmatism problem. Our method mimics the human visual system: the Core Perception Zone (green) with high-detail processing guides the Positive Pattern Refinement (PPR) module to reshape attention more effectively toward foreground objects, while the Peripheral Zone (orange) with reduced details informs the Negative Context Modulation (NCM) module to enhance object-background boundaries by modeling background contexts. The Textual Semantic Alignment (TSA) module enforces the distinctions between center and peripheral regions, analogous to the center-surround mechanism of biological perception.

Figure 3: Overview of our human-vision-inspired framework to remedy the Astigmatism problem in CD-FSOD. The architecture integrates three complementary modules: (1) Positive Pattern Refinement (PPR) reshapes attention toward foreground objects using class prototypes; (2) Negative Context Modulation (NCM) enhances object-background boundaries through explicit background modeling; and (3) Textual Semantic Alignment (TSA) enhances these distinctions via cross-modal knowledge integration with negative descriptors (“not [class]”). During training (top), the model is optimized with both detection and alignment objectives, and extracts discriminative prototypes from support examples, stored in positive and negative repositories. At inference (bottom), stored prototypes turn dispersed attention patterns in query images into crystallized object-centric representations, analogous to the human center-peripheral visual system.

Figure 4: Attention distribution around a foreground patch

x_{0}

. Dashed arrows labeled

A_{k}

indicate attention from

x_{0}

to neighbors

x_{k}

. Same‑object neighbors

x_{2},x_{3}

are spatially near and should receive high attention, whereas the background patch

x_{1}

is distant and should be weak. In target‑domain Astigmatism, domain shift diverts attention toward the distant background (

A_{1}\uparrow

;

A_{2},A_{3}\downarrow

), yielding dispersed attention and a larger attention distance. Since the spatial distance between patches are fixed, our method reverses this dispersion by (i) down‑weighting background responses

A_{1}

via a learned background prototype and simple “not [class]” cues (NCM/TSA module) to suppress spurious foreground–background affinity, and (ii) up‑weighting same‑object responses

A_{2},A_{3}

via class‑specific foreground prototypes (PPR module) to strengthen intra‑object compatibility, thereby shortening attention distance and restoring a focused, object‑centric pattern.

Method	Backbone	Venue	ArTaxOr	Clipart1k	DIOR	DeepFish	NEU-DET	UODD	Avg.
Distill † [ 32 ]	ResNet50	ICASSP’23	5.10	7.60	10.50	–	–	5.90	/
ViTDet † [ 16 ]	ViT-B/14	ECCV’22	5.90	6.10	12.90	0.90	2.40	4.00	5.37
ViTDet* [ 16 ]	Swin-T	ECCV’22	8.85	10.22	18.50	1.06	4.54	4.90	8.01
Detic † [ 46 ]	ViT-L/14	ECCV’22	3.20	15.10	4.10	9.00	3.80	4.20	6.57
Detic* [ 46 ]	Swin-T	ECCV’22	4.45	19.22	5.70	10.49	7.15	5.12	8.69
DE-ViT † [ 36 ]	ViT-L/14	arXiv’23	10.50	13.00	14.70	19.30	0.60	2.40	10.08

Table 1: 1-shot cross-domain detection performance comparison (mAP). † indicates results reported in CD-ViTO [ 5 ] . Methods with * are reimplemented by us using Swin-Tiny backbone. GLIP and our method are also implemented with Swin-Tiny backbone.

Method	Backbone	Venue	ArTaxOr	Clipart1k	DIOR	DeepFish	NEU-DET	UODD	Avg.
Distill † [ 32 ]	ResNet50	ICASSP’23	12.50	23.30	19.10	15.50	16.00	12.20	16.43
ViTDet † [ 16 ]	ViT-B/14	ECCV’22	20.90	23.30	23.30	9.00	13.50	11.10	16.85
ViTDet* [ 16 ]	Swin-T	ECCV’22	21.05	23.45	23.45	11.03	19.96	15.07	19.00
Detic † [ 46 ]	ViT-L/14	ECCV’22	8.70	20.20	12.10	14.30	14.10	10.40	13.30
Detic* [ 46 ]	Swin-T	ECCV’22	10.63	20.31	14.78	17.46	20.80	14.07	16.34
DE-ViT † [ 36 ]	ViT-L/14	arXiv’23	38.00	38.10	23.40	21.20	7.80	5.00	22.25

Table 2: 5-shot cross-domain detection performance comparison (mAP). † indicates results reported in CD-ViTO [ 5 ] . Methods with * are reimplemented by us using Swin-Tiny backbone. GLIP and our method are also implemented with Swin-Tiny backbone.

Method	Backbone	Venue	ArTaxOr	Clipart1k	DIOR	DeepFish	NEU-DET	UODD	Avg.
Distill † [ 32 ]	ResNet50	ICASSP’23	18.10	27.30	26.50	15.50	21.10	14.50	20.50
ViTDet † [ 16 ]	ViT-B/14	ECCV’22	23.40	25.60	29.40	6.50	15.80	15.60	19.38
ViTDet* [ 16 ]	Swin-T	ECCV’22	24.20	25.93	30.10	8.04	22.52	19.55	21.72
Detic † [ 46 ]	ViT-L/14	ECCV’22	12.00	22.30	15.40	17.90	16.80	14.40	16.47
Detic* [ 46 ]	Swin-T	ECCV’22	13.80	22.59	17.20	22.04	23.97	16.58	19.36
DE-ViT † [ 36 ]	ViT-L/14	arXiv’23	49.20	40.80	25.60	21.30	8.80	5.40	25.18

Table 3: 10-shot cross-domain detection performance comparison (mAP). † indicates results reported in CD-ViTO [ 5 ] . Methods with * are reimplemented by us using Swin-Tiny backbone. GLIP and our method are also implemented with Swin-Tiny backbone.

Dataset	Classes	Images	Boxes	Domain	Challenge
ArTaxOr	7	1383	1628	photorealistic	small objects
Clipart1k	20	500	1526	cartoon	style abstraction
DIOR	20	5000	28810	aerial	medium-scale
DeepFish	1	909	3029	aquatic	poor visibility
NEU-DET	6	360	834	industrial	subtle features
UODD	3	506	3218	underwater	complex clutter

Table 4: Summary of target datasets used in our evaluation.

Components			Datasets
NCM	PPR	TSA	ArTaxOr	Clipart1k	DIOR	DeepFish	NEU-DET	UODD
			48.11	39.28	27.39	28.40	19.55	11.40
✓			49.95	40.52	27.75	29.58	20.48	12.21
✓	✓		52.68	42.95	28.62	31.96	22.38	14.26
✓	✓	✓	54.98	44.83	29.41	33.87	23.64	15.66

Table 5: Ablation study showing each component’s contribution to overall performance (mAP %) in 5-shot setting.

Figure 5: Top: Visualization of attention maps demonstrating the Astigmatism problem and our solution. Each row displays two different samples: including the original image with ground truth (left), conventional fine-tuning that only marginally alleviates dispersed attention patterns (middle), and ours, which indicates focused attention that precisely concentrates on target regions. Bottom: Change in attention distance relative to pretrained model across datasets. Negative values indicate reduction in attention dispersion (larger magnitude means better focus). While fine-tuning reduces dispersion, our method achieves substantially greater reductions, validating our superior ability to address the Astigmatism problem by reshaping attention toward foreground objects.

Figure 6: Validation of Negative Context Modulation (NCM) module through background ratio analysis. Top: Performance comparison with varying background ratios. Our method demonstrates consistent advantages over the baseline, particularly at higher background ratios (0.3-1.0), indicating NCM exploits negative background contexts as complementary cues to strengthen foreground feature learning and boundary discrimination , thereby mitigating the attention dispersion in Astigmatism problem. Bottom: Visual examples showing detection with increasing background ratios from left to right (R=0.0 to R=1.0).

Figure 7: Qualitative detection results on cross-domain scenarios. Our method produces significantly more precise predictions with fewer candidate boxes compared to alternatives. The numbers indicate displayed boxes/total generated boxes, demonstrating our method’s ability to reduce redundant detections while maintaining precision . In the maritime scene (top), our approach avoids misclassifying background as objects, while in the transportation (middle) and indoor (bottom) examples, it generates tighter bounding boxes with minimal redundancy. These results validate our framework’s effectiveness in remedying the Astigmatism problem.

Text Count	Performance		GPU Mem. (MB) $\downarrow$	Eff. Ratio $\uparrow$
Text Count	AP (%) $\uparrow$	Gain $\uparrow$	GPU Mem. (MB) $\downarrow$	Eff. Ratio $\uparrow$
50	39.34	-	8762	-
100	40.31	+0.97%	8850	1.10%
150	41.48	+1.17%	8943	1.26%
200	42.87	+1.39%	9027	1.65%
250	43.47	+0.60%	9108	0.74%

Table 6: Effect of background text quantity on performance. Eff. Ratio =

\Delta

AP /

\Delta

Mem., indicating AP gain per MB increase.

Figure 8: Analysis of key parameters in our approach. Left : Effect of TSA loss weight (

\lambda

) on detection performance. Performance remains stable when

\lambda\leq 1

, peaks at

\lambda=10^{3}

, and declines beyond that, underscoring the importance of appropriate weighting in enhancing object-background separation. Right : Impact of threshold selection on NCM effectiveness. Performance remains above baseline across a wide threshold range (0.6-0.9), with optimal results at 0.75, demonstrating that selectively enhancing background features effectively reinforces object-background boundaries.

Figure 9: Parameter analysis of temperature-weight interactions in PPR. The heatmap shows peak performance at moderate weights (0.1-1.0) with robust temperature tolerance, while extreme weights (

\geq

1.0) cause significant performance degradation. This confirms that balanced prototype utilization is essential for effective attention refinement without disrupting original feature representations.

Remedying Target-Domain Astigmatism for Cross-Domain Few-Shot Object Detection

Abstract

Figures & tables

Few-Shot Object Detection

A Closer Look at Cross-Domain Few-Shot Object Detection: Fine-Tuning Matters and Parallel Decoder Helps

Prompt-Driven Simulation with Feature Perturbation for Cross-Domain Few-Shot Object Detection

Learning Multi-Modal Prototypes for Cross-Domain Few-Shot Object Detection

Abstract

Few-Shot Object Detection

Explore similar work

A Closer Look at Cross-Domain Few-Shot Object Detection: Fine-Tuning Matters and Parallel Decoder Helps

Prompt-Driven Simulation with Feature Perturbation for Cross-Domain Few-Shot Object Detection

Learning Multi-Modal Prototypes for Cross-Domain Few-Shot Object Detection