SRasP: Self-Reorientation Adversarial Style Perturbation for Cross-Domain Few-Shot Learning
Authors: Wenqian Li, Pengfei Fang, Hui Xue
Organizations: the School of Computer Science and Engineering, Southeast University, Nanjing 210096, China · the Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, Nanjing 211189, China
Cross-Domain Few-Shot Learning (CD-FSL) aims to transfer knowledge from a seen source domain to unseen target domains, serving as a key benchmark for evaluating the robustness and transferability of models. Existing style-based perturbation methods mitigate domain shift but often suffer from gradient instability and convergence to sharp minima.To address these limitations, we propose a novel crop-global style perturbation network, termed Self-Reorientation Adversarial Style Perturbation (SRasP). Specifically, SRasP leverages global semantic guidance to identify incoherent crops, followed by reorienting and aggregating the style gradients of these crops with the global style gradients within one image. Furthermore, we propose a novel multi-objective optimization function to maximize visual discrepancy while enforcing semantic consistency among global, crop, and adversarial features. Applying the stabilized perturbations during training encourages convergence toward flatter and more transferable solutions, improving generalization to unseen domains. Extensive experiments are conducted on multiple CD-FSL benchmarks, demonstrating consistent improvements over state-of-the-art methods.
Figures & tables
Fig. 1: (a) Given an input image, multiple local crops are extracted. The training process applies one of three crop selection strategies, including random-crop selection, concept-crop selection and incoherent-crop selection, each leading to different gradient directions during optimization. (b) The gradient cosine similarity across training epochs shows that the proposed method maintains consistently higher stability compared with other perturbation methods, indicating a more reliable update trajectory. (c) Loss-surface visualizations further demonstrate that our incoherent-crop perturbation drives the model toward flatter and more generalizable minima.
Fig. 2: Comparison between Vanilla Global Perturbation and our Self-Reorientation Perturbation. Left : In the vanilla approach, a global style perturbation is directly applied to the feature map of the entire input. Right : In our Self-Reorientation Perturbation, the input image is first divided into incoherent crops. These crop style gradients are then reoriented and aggregated with the global style gradient.
Fig. 3: Overview of the proposed SRasP. SRasP first samples multiple localized crops from an input image and identifies incoherent regions that exhibit semantic inconsistency with the global content. Style gradients are extracted from both crop-level and global features, where incoherent crop gradients are reoriented toward the global semantic direction and aggregated through a self-reorientation gradient ensemble to resolve gradient conflicts. The resulting stable and semantically guided global style gradient is then used to synthesize hard yet meaningful adversarial style perturbations via AdaIN. In addition, a consistency–discrepancy triplet objective is employed to maximize visual style diversity while preserving semantic alignment among global, crop, and adversarial representations, enabling SRasP to generate stronger style variations and improve cross-domain generalization.
Method
Venue
Back.
FT
ChestX
ISIC
EuroSAT
CropDisease
CUB
Cars
Places
Plantae
Aver.
1-shot
GNN [ 39 ]
ICLR’18
RN10
✗
22.00 ± 0.46
32.02 ± 0.66
63.69 ± 1.03
64.48 ± 1.08
45.69 ± 0.68
31.79 ± 0.51
53.10 ± 0.80
35.60 ± 0.56
43.55
FWT [ 13 ]
ICLR’20
RN10
✗
22.04 ± 0.44
31.58 ± 0.67
62.36 ± 1.05
66.36 ± 1.04
47.47 ± 0.75
31.61 ± 0.53
55.77 ± 0.79
35.95 ± 0.58
44.14
ATA [ 40 ]
IJCAI’21
RN10
✗
22.10 ± 0.20
33.21 ± 0.40
61.35 ± 0.50
67.47 ± 0.50
45.00 ± 0.50
33.61 ± 0.40
53.57 ± 0.50
34.42 ± 0.40
43.84
StyleAdv [ 11 ]
CVPR’23
RN10
✗
22.64 ± 0.35
33.96 ± 0.57
70.94 ± 0.82
74.13 ± 0.78
48.49 ± 0.72
34.64 ± 0.57
58.58 ± 0.83
41.13 ± 0.67
48.06
FLoR [ 41 ]
CVPR’24
RN10
✗
23.11 ± 0.31
38.11 ± 0.51
62.90 ± 0.65
73.64 ± 0.64
49.99 ± 0.62
37.41 ± 0.54
53.18 ± 0.79
40.10 ± 0.64
47.31
SVasP [ 12 ]
AAAI’25
RN10
✗
23.23 ± 0.35
37.63 ± 0.58
72.30 ± 0.82
75.87 ± 0.73
49.49 ± 0.72
35.27 ± 0.57
59.07 ± 0.81
41.22 ± 0.62
49.26
TABLE I: Quantitative comparison to state-of-the-arts methods on eight target datasets based on ResNet-10, which is pretrained on miniImageNet. Accuracy of 5-way 1-shot/5-shot tasks with 95 confidence interval are reported. “ FT ” with ✓ means finetuning is used, vice versa. “ Aver. ” means “Average Accuracy” of the eight datasets. The optimal results are marked in bold .
Method
Venue
Back.
FT
ChestX
ISIC
EuroSAT
CropDisease
CUB
Cars
Places
Plantae
Aver.
1-shot
StyleAdv [ 11 ]
CVPR’23
ViT-S
✗
22.92 ± 0.32
33.05 ± 0.44
72.15 ± 0.65
81.22 ± 0.61
84.01 ± 0.58
40.48 ± 0.57
72.64 ± 0.67
55.52 ± 0.66
57.75
FLoR [ 41 ]
CVPR’24
ViT-S
✗
22.78 ± 0.31
34.20 ± 0.45
72.39 ± 0.67
81.81 ± 0.60
84.60 ± 0.57
40.71 ± 0.58
73.85 ± 0.65
51.93 ± 0.65
57.78
CD-CLS [ 45 ]
NeurIPS’24
ViT-S
✗
22.93 ± 0.33
34.21 ± 0.45
74.08 ± 0.63
83.51 ± 0.60
82.90 ± 0.60
41.67 ± 0.57
69.17 ± 0.66
51.88 ± 0.67
57.54
SVasP [ 12 ]
AAAI’25
ViT-S
✗
22.68 ± 0.30
34.49 ± 0.46
72.50 ± 0.62
80.82 ± 0.62
85.56 ± 0.57
40.51 ± 0.59
75.93 ± 0.66
56.25 ± 0.65
58.59
REAP [ 46 ]
ICML’25
ViT-S
✗
23.62 ± 0.31
37.21 ± 0.44
74.69 ± 0.60
84.04 ± 0.59
82.09 ± 0.62
42.23 ± 0.60
72.02 ± 0.66
55.56 ± 0.67
58.93
ReCIT [ 26 ]
ICML’25
ViT-S
✗
23.27 ± 0.31
35.13 ± 0.44
74.56 ± 0.60
84.76 ± 0.59
82.30 ± 0.58
42.89 ± 0.58
72.64 ± 0.66
55.60 ± 0.65
58.89
TABLE II: Quantitative comparison to state-of-the-arts methods on eight target datasets based on ViT-small, which is pretrained on ImageNet1K by DINO. Accuracy of 5-way 1-shot/5-shot tasks with 95 confidence interval are reported.
SR
LCDTO
Lcon
ChestX
ISIC
EuroSAT
CropDisease
CUB
Cars
Places
Plantae
Aver.
1-shot
-
-
-
22.61 ± 0.35
33.99 ± 0.56
69.04 ± 0.84
70.25 ± 0.77
45.73 ± 0.68
32.77 ± 0.56
53.91 ± 0.78
36.19 ± 0.62
45.56
✓
23.29 ± 0.35
34.99 ± 0.56
70.67 ± 0.81
73.85 ± 0.78
45.75 ± 0.68
33.73 ± 0.56
54.55 ± 0.77
38.70 ± 0.59
46.94
✓
✓
22.62 ± 0.34
36.23 ± 0.57
70.17 ± 0.83
72.24 ± 0.77
46.08 ± 0.68
34.56 ± 0.56
55.46 ± 0.77
39.81 ± 0.62
47.15
✓
✓
22.92 ± 0.36
36.77 ± 0.53
72.40 ± 0.84
75.60 ± 0.78
48.43 ± 0.69
35.16 ± 0.59
57.91 ± 0.81
40.19 ± 0.62
48.67
✓
✓
✓
23.39 ± 0.37
37.92 ± 0.60
74.90 ± 0.82
76.82 ± 0.77
50.62 ± 0.68
36.20 ± 0.60
60.11 ± 0.75
41.94 ± 0.61
50.24
5-shot
-
-
-
25.16 ± 0.35
47.46 ± 0.54
86.30 ± 0.54
92.07 ± 0.44
63.37 ± 0.68
49.26 ± 0.63
75.09 ± 0.64
57.66 ± 0.65
62.07
TABLE III: Ablation study of the proposed method with different component combinations. “SR” indicates Self-Reorientation Gradient Ensemble module.
Mining Strategy
BSCD-FSL
mini -CUB
Aver.
1-shot
Baseline
48.97
42.15
45.56
Concept
52.31
46.37
49.34
Random
52.42
46.57
49.49
Incoherent (Ours)
53.26
47.22
50.24
5-shot
Baseline
62.75
61.35
62.05
Concept
65.28
65.21
65.24
TABLE IV: Performances on different crop mining strategies. The average accuracy (%) is reported.
Fig. 4: Meta-training loss curves of the baseline and the proposed SRasP.
ξ
BSCD-FSL
mini -CUB
AVer.
1-shot
ξ=0.01
52.57
46.41
49.49
ξ=0.1 (Ours)
53.26
47.22
50.24
ξ=1
52.68
46.47
49.58
5-shot
ξ=0.01
64.93
64.51
64.72
ξ=0.1 (Ours)
65.92
65.65
65.78
ξ=1
65.11
64.62
64.86
TABLE V: Performances on different ξ . The average accuracy (%) is reported.
λ
BSCD-FSL
mini -CUB
AVer.
1-shot
λ=0
52.77
46.61
49.69
λ=0.2
53.26
47.22
50.24
λ=0.4
52.91
46.64
49.78
λ=0.6
52.49
46.36
49.43
λ=0.8
52.30
45.88
49.09
λ=1
51.99
45.67
48.83
TABLE VI: Performances on different λ . The average accuracy (%) is reported.
κ1 , κ2
BSCD-FSL
mini -CUB
AVer.
1-shot
Same
52.43
46.18
49.30
Different (Ours)
53.26
47.22
50.24
5-shot
Same
65.01
64.68
64.85
Different (Ours)
65.92
65.65
65.78
TABLE VII: Performances on whether use same κ1 , κ2 . The average accuracy (%) is reported.
Fig. 5: Performances on different numbers of crops k .
Fig. 6: Loss landscape visualization of different methods on the BSCD-FSL benchmark. SRasP produces markedly flatter loss landscapes across all target domains compared to the baseline and SVasP.
Fig. 7: Grad-CAM visualizations of different methods on four representative datasets. SRasP produces cleaner and more object-centric activation patterns across diverse target domains.
Data augmentation, which simulates diverse visual variations to expand the source distribution and induce synthetic domain shifts, is a simple yet effective strategy for mitigating severe domain shifts and limited labeled target data in cross-domain few-shot object detection (CD-FSOD). Existing approaches rely on conventional data augmentation, such as Color-Jitter, Mosaic, and background-centric adaptation (e.g., Domain-RAG), which are limited in modeling complex domain shifts and often lead to suboptimal performance. In this paper, we propose PSP-FSOD, a principled framework that integrates prompt-driven domain simulation with feature perturbation regularization to improve generalization in CD-FSOD. To enable controllable domain synthesis, we design a prompt-driven strategy that leverages the visual grounding capability of large VLMs to jointly model foreground and background variations, generating semantically consistent yet domain-diverse training samples. Moreover, we adopt a grounding-aware generation scheme that guides object placement and alleviates semantic-spatial misalignment, thereby improving foreground adaptation. To ensure training stability and robustness, we further introduce a noise-induced feature perturbation mechanism that injects Gaussian noise into multi-scale intermediate features with distribution correction, encouraging consistent predictions under perturbations and reducing reliance on domain-specific cues. Extensive experiments demonstrate that PSP-FSOD produces high-quality domain-diverse supervision and learns domain-invariant representations, consistently improving performance across CD-FSOD benchmarks.
Linhai Zhuo, Junxi Cai, Tianwen Qian +2
Fuzhou University · East China Normal University · Xiamen University +1
Cross-Domain Few-Shot Learning (CDFSL) aims to adapt large-scale pretrained models to specialized target domains with limited samples, yet the few-shot fine-tuning of vision-language models like CLIP remains underexplored. By establishing multiple fine-tuning baselines of CLIP for CDFSL, we find adapter-based methods (e.g., LoRA) consistently outperform prompt-based ones (e.g., MaPLe), contrary to in-domain scenarios. To make those effective in-domain methods competitive again in CDFSL, we analyze this phenomenon and discover LoRA's superiority stems from rectifying the collapsed attention of visual CLS token, enhancing modality alignment and class separation by focusing on text-related visual regions. Further, we find textual EOS token exhibit much better attention to visual samples, and CLIP's standard contrastive loss weakly constrains modality alignment. Based on these insights, we propose Semantic Probe, a plug-and-play attention rectification framework for both adapter- and prompt-based methods. Extensive experiments on four CDFSL benchmarks validate our rationale, achieving state-of-the-art performance and benefiting both fine-tuning paradigms. Codes will be released.
Cross-domain Few-shot Segmentation (CD-FSS) aims to transfer knowledge learned from source domain to distinct target domains, segmenting unseen target classes with only a few annotated samples. Although existing methods have made significant progress, they still rely on training or fine-tuning processes, which incur high computational costs and risk overfitting. We observe that when powerful and general-purpose vision foundation models are incorporated into these methods, their performance shows only marginal improvement or even degrades due to overfitting. To address this, we eliminate trainable parameters and propose a training-free framework to avoid both training overhead and overfitting. Built upon the self-supervised vision encoder DINOv3, our framework addresses cross-domain challenges through three core modules. First, the Semantic-aware Feature Re-fusion (SAFR) module identifies and re-fuses features that emphasize semantic patterns, generating representations with enhanced semantic discriminability. Additionally, the Adaptive Support Enhancement (ASE) module narrows semantic gaps between support and query through robust query information aggregation. Finally, the Hybrid Prototype Matching (HPM) module integrates matching results from diverse prototypes to adapt to varying semantic complexity across domains. Extensive experiments on four target domain datasets demonstrate that our method achieves state-of-the-art performance in CD-FSS without any training.
Sujun Sun, Mingwu Ren, Haofeng Zhang
School of Computer Science and Engineering, Nanjing University of Science and Technology, China · State Key Laboratory of Intelligent Manufacturing of Advanced Construction Machinery, China