Dual Contrastive Network for Few-Shot Remote Sensing Image Scene Classification
Authors: Zhong Ji, Liyuan Hou, Xuan Wang, Gang Wang, Yanwei Pang
Organizations: School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China · CETC Key Laboratory of Aerospace Information Applications, Shijiazhuang, 050081, China
Few-shot remote sensing image scene classification (FS-RSISC) aims at classifying remote sensing images with only a few labeled samples. The main challenges lie in small inter-class variances and large intra-class variances, which are the inherent property of remote sensing images. To address these challenges, we propose a transfer-based Dual Contrastive Network (DCN), which incorporates two auxiliary supervised contrastive learning branches during the training process. Specifically, one is a Context-guided Contrastive Learning (CCL) branch and the other is a Detail-guided Contrastive Learning (DCL) branch, which focus on inter-class discriminability and intra-class invariance, respectively. In the CCL branch, we first devise a Condenser Network to capture context features, and then leverage a supervised contrastive learning on top of the obtained context features to facilitate the model to learn more discriminative features. In the DCL branch, a Smelter Network is designed to highlight the significant local detail information. And then we construct a supervised contrastive learning based on the detail feature maps to fully exploit the spatial information in each map, enabling the model to concentrate on invariant detail features. Extensive experiments on four public benchmark remote sensing datasets demonstrate the competitive performance of our proposed DCN.
Figures & tables
Fig. 1: Illustrations of remote sensing image characteristics: small inter-class variances and large intra-class variances. See texts for details.
Fig. 2: Overview of the proposed DCN framework. It consists of a Context-guided Contrastive Learning (CCL) branch and a Detail-guided Contrastive Learning (DCL) branch to facilitate the model to learn more discriminative and detail information to generate context and detail features, both of which are used for training their corresponding classifiers.
Fig. 3: Illustration of Condenser Network. With the Squeeze and Expand operations among channels, the scene-relevant information is enhanced while the irrelevant information is ignored. After the global average pooling, the context features are generated.
Fig. 4: Illustration of Smelter Network. The helpful information in the channel and spatial dimensions is extracted respectively to enhance the significant local detail regions of the feature maps.
Type
Method
Backbone
WHU-RS19
UC Merced
1-shot
5-shot
1-shot
5-shot
Optimization
LLSR [ 5 ]
Conv
57.10
70.65
39.47
57.40
MAML [ 6 ]
ResNet-12
59.19 ± 0.92
72.34 ± 0.75
50.23 ± 0.98
62.02 ± 0.72
Metric
DLA-MatchNet [ 7 ]
Conv
68.27 ± 1.83
79.89 ± 0.33
53.76 ± 0.62
63.01 ± 0.51
SCL-MLNet [ 8 ]
Conv
-
-
51.37 ± 0.79
68.09 ± 0.92
MatchingNet [ 9 ]
ResNet-12
75.99 ± 0.63
82.96 ± 0.40
50.99 ± 0.69
57.14 ± 0.59
TABLE I: Comparative results(%) on the WHU-RS19 and UC Merced datasets with ± 95 % confidence intervals. The top two results are shown in bold and underline.
Type
Method
Backbone
NWPU-RESISC45
AID
1-shot
5-shot
1-shot
5-shot
Optimization
LLSR [ 5 ]
Conv
51.43
72.90
45.18
61.76
MAML [ 6 ]
ResNet-12
56.01 ± 0.87
72.94 ± 0.63
62.54 ± 0.79
73.12 ± 0.85
Metric
DLA-MatchNet [ 7 ]
Conv
68.80 ± 0.70
81.63 ± 0.46
57.21 ± 0.82
73.45 ± 0.61
SCL-MLNet [ 8 ]
Conv
62.21 ± 1.12
80.86 ± 0.76
59.46 ± 0.96
76.31 ± 0.68
MatchingNet [ 9 ]
ResNet-12
61.57 ± 0.49
76.02 ± 0.34
64.30 ± 0.46
74.49 ± 0.35
TABLE II: Comparative results(%) on the NWPU-RESISC45 and AID datasets with ± 95 % confidence intervals. The top two results are shown in bold and underline.
Method
WHU-RS19
UC Merced
1-shot
5-shot
1-shot
5-shot
Backbone
67.65 ± 0.74
83.17 ± 0.36
51.87 ± 0.70
68.90 ± 0.55
CCL (w/o CL)
68.17 ± 0.72
84.08 ± 0.38
52.52 ± 0.72
70.02 ± 0.52
CCL (w/o CN)
67.13 ± 0.66
83.80 ± 0.37
54.37 ± 0.75
72.40 ± 0.48
CCL
69.03 ± 0.67
85.22 ± 0.40
54.93 ± 0.68
72.86 ± 0.50
DCL (w/o CL)
67.61 ± 0.69
83.11 ± 0.36
52.10 ± 0.65
69.57 ± 0.58
TABLE III: Ablation studies(%) on the WHU-RS19 and UC Merced datasets.
Method
NWPU-RESISC45
AID
1-shot
5-shot
1-shot
5-shot
Backbone
71.26 ± 0.86
86.19 ± 0.47
69.22 ± 0.79
83.48 ± 0.45
CCL (w/o CL)
71.66 ± 0.85
86.31 ± 0.48
69.35 ± 0.84
83.07 ± 0.49
CCL (w/o CN)
72.01 ± 0.82
87.20 ± 0.44
69.18 ± 0.79
84.46 ± 0.45
CCL
72.55 ± 0.71
87.92 ± 0.41
70.91 ± 0.80
85.66 ± 0.44
DCL (w/o CL)
71.91 ± 0.75
86.53 ± 0.47
69.65 ± 0.82
84.10 ± 0.51
TABLE IV: Ablation studies(%) on the NWPU-RESISC45 and AID datasets.
Fig. 5: The inter-class variances and intra-class variances on the NWPU-RESISC45 dataset on 5-way 1-shot with baseline (ResNet), Condenser Network-integrated (ResNet+CN) and Smelter Network-integrated (ResNet+SN).
Fig. 6: Grad-CAM visualization results of inter-class images with different networks: baseline (ResNet) and Condenser Network-integrated (ResNet+CN).
Fig. 7: Grad-CAM visualization results of intra-class images with baseline (ResNet) and Smelter Network-integrated (ResNet+SN), where the right of the red dotted line is the visualization results of feature maps from several channels with large weights extracted by ResNet+SN.
Fig. 8: Impact of weighting coefficient α on the NWPU-RESISC45 dataset.
Fig. 9: Impact of weighting coefficient β on the NWPU-RESISC45 dataset.
Fig. 10: Impact of weighting coefficient γ on the NWPU-RESISC45 dataset.
WHU-RS19
UC Merced
NWPU-RESISC45
AID
Epoch
150
150
200
150
Time(s)
403.50
883.50
21010.00
4778.25
TABLE V: Training runtime on the WHU-RS19, UC Merced, NWPU-RESISC45 and AID datasets.
Remote sensing scene classification is increasingly relying on foundation models pre-trained on large-scale Earth-observation data. Moreover, transductive inference, which exploits the collective statistical structure of the entire unlabeled query set, appears to naturally match remote sensing pipelines where large images are routinely split into patches and inferred as a batch. In this work, we introduce LC-TIM (Locally Consistent Transductive Information Maximization), which extends the state-of-the-art Transductive Information Maximization for Few-Shot CLIP (TIM++) objective with a local consistency regularizer that enforces prediction agreement between each query sample and its κ nearest feature-space neighbors. The regularizer enters as a single multiplicative factor in the closed-form q-update, adding negligible computational overhead. We further propose a multi-source extension that fuses the affinity graph from multiple remote sensing foundation model, further boosting classification accuracy. To assess these methods, we establish the first comprehensive, open-source benchmark for transductive few-shot RS scene classification, evaluating LP++, TransCLIP, TIM++, and LC-TIM across ten diverse datasets, two remote sensing vision-language models, and across various few-shot settings. Our experiments show that transductive methods consistently outperform zero-shot baselines, and that LC-TIM achieves state-of-the-art accuracy, with the largest gains in the low-shot regime where neighborhood cues are most informative. Code is publicly available at: https://github.com/elkhouryk/LC-TIM
Contrastive language-image learning (CLIP) has become a key paradigm for remote sensing vision-language understanding. However, existing remote sensing contrastive learning methods are mostly built on RGB-oriented CLIP architectures, making it difficult to exploit heterogeneous sensors such as SAR, multi-spectral imaging (MSI), and hyperspectral imaging (HSI). To address this limitation, we propose OmniRSCLIP, an end-to-end contrastive learning framework that supports multi-source sensor inputs for remote sensing vision-language modeling. The key idea is to extend CLIP beyond its fixed RGB input interface without breaking the pretrained visual knowledge. To this end, OmniRSCLIP introduces Spectral-Spatial Basis Decomposition (SSBD), which formulates arbitrary-channel adaptation as a basis recomposition problem: pretrained CLIP patch embeddings provide transferable spatial bases, while wavelength-conditioned coefficients span sensor-specific embedding kernels within a constrained visual prior space. This design avoids forcing heterogeneous sensors into a fixed-channel input space, while aligning them in a unified image-text semantic space. We further introduce a spectral-context-aware mask-based contrastive learning scheme to suppress modality-specific redundant features and enhance fine-grained image-text alignment. Finally, to support multi-modal training, we construct OmniRS5M, the first large-scale remote sensing image-text corpus covering RGB, SAR, MSI, and HSI. Experiments on retrieval, zero-shot classification, and semantic localization show that OmniRSCLIP preserves strong RGB-domain performance while effectively extending CLIP to heterogeneous remote sensing modalities.
Xiangyang Miao, Kelu Yao, Yekai Huang +7
School of Computer Science and Technology, Zhejiang University · Space-based Computing System Research Center, Zhejiang Lab
Few-shot segmentation asks a model to delineate a target class in a query image from only a handful of annotated examples, a setting most acute in remote sensing, where labels are scarce and the imagery departs sharply from the natural images on which vision backbones are pretrained. Prevailing approaches either train a segmenter on labelled episodes, which raises accuracy within the training distribution but binds the model to it, or reduce each class to a lossy summary of frozen features, a single prototype, a few cluster prototypes, or a discrete clustering, none of which preserves the internal structure of a multimodal class. We argue that a class is better described by a distribution than by a point, and that frozen self-supervised features already carry enough structure to estimate that distribution directly. We introduce FROST, a training-free few-shot segmenter that treats the reference foreground and background as two point clouds on the unit sphere of frozen DINOv3 features and labels each query token by a nonparametric density ratio, with a threshold the Bayes rule fixes at zero under equal priors. Because the variance of a density estimate shrinks as its sample grows, the decision sharpens as references accumulate, and every remaining quantity from the kernel bandwidth to the spatial gate is read from the support set rather than tuned. We develop FROST for overhead imagery, where a class is typically a scatter of many small and dissimilar instances that a density tracks but a lossy summary blurs. Across seventeen remote-sensing benchmarks FROST surpasses both training-free and learning-based methods, leading by 5.6 mIoU from a single annotated example and widening its lead as the support set grows, all while remaining among the smallest models compared. Code is available at https://github.com/jhpark-ai/FROST.