Organizations: State Key Laboratory of Physical Oceanography, Ocean University of China, Qingdao 266100, China. · Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA.
Multi-source remote sensing data classification has received growing attention recently. However, existing methods still struggle to jointly model long-range contextual dependencies and shared spatial patterns across heterogeneous modalities. To address this issue, we propose a Shared-Pattern Collaborative Network (SPCNet) for HSI and SAR/LiDAR joint classification. To enhance cross-modal collaborative learning, Shared-Pattern Collaborative Block (SPCB) is designed by introducing a shared large-kernel operator and a dynamic convolution branch, enabling the network to capture common spatial structures and content-adaptive local features. Furthermore, Region-Adaptive Modality Allocator (RAMA) is developed to assign spatially varying modality weights for reliable feature fusion. Experiments on Houston2013, Augsburg, and Houston2018 datasets demonstrate that SPCNet achieves superior classification performance over several state-of-the-art methods. The code will be made publicly available at https://github.com/oucailab/SPCNet
Figures & tables
Fig. 1: Comparison between existing multi-source data classification models and the proposed SPCNet. SPCNet shares a large-kernel operator across the two modality branches to collaboratively learn common spatial patterns while retaining modality-specific features for adaptive fusion.
Fig. 2: Overview of the proposed Shared-Pattern Collaborative Network (SPCNet) for multi-source remote sensing data classification. The input multi-source data are processed by stacked Shared-Pattern Collaborative Blocks (SPCB) that exchange and learn common spatial-spectral representations through shared large kernels. Region-Adaptive Modality Allocator (RAMA) dynamically integrates complementary modality information to generate the classification map.
Fig. 3: Details of the Shared-Pattern Collaborative Block (SPCB).
Fig. 4: Details of the Region-Adaptive Modality Allocator (RAMA).
No.
Name
Color
Train
Test
1
Healthy grass
198
1053
2
Stressed grass
190
1064
3
Synthetic grass
192
505
4
Trees
188
1056
5
Soil
186
1056
6
Water
182
143
TABLE I: Number of training and test samples for the Houston2013 dataset.
No.
Name
Color
Train
Test
1
Forest
146
13361
2
Residential area
264
30065
3
Industrial area
21
3830
4
Low plants
248
26609
5
Allotment
52
523
6
Commercial area
7
1638
TABLE II: Number of training and test samples for the Augsburg dataset.
No.
Name
Color
Train
Test
1
Healthy grass
1000
38196
2
Stressed grass
1000
129008
3
Artificial turf
1000
1736
4
Evergreen trees
1000
53322
5
Deciduous trees
1000
19172
6
Bare earth
1000
17064
TABLE III: Number of training and test samples for the Houston2018 dataset.
Fig. 5: Visualization of the three multi-source datasets. (a) Houston2013 dataset. (b) Augsburg dataset. (c) Houston2018 dataset.
Fig. 6: Depth of SPCB Layers.
Fig. 7: Number of Principal Components for HSI.
Fig. 8: Size of the Input Image Patches.
Fig. 9: Visualized classification results of different methods for the Houston2013 dataset. (a) Ground truth. (b) FusAtNet. (c) S2ENet. (d) AsyFFNet. (e) ExViT. (f) HCT. (g) MACN. (h) MICF-Net. (i) GCCQTNet. (j) CHNet. (k) MGMNet. (l) SPCNet.
Class
FusAtNet
S2ENet
AsyFFNet
ExViT
HCT
MACN
MICF-Net
GCCQTNet
CHNet
MGMNet
SPCNet
Healthy grass
82.53
81.48
82.72
82.72
81.10
82.90
82.91
82.05
83.10
83.00
82.91
Stressed grass
85.15
84.68
83.93
85.15
86.09
100.0
83.46
96.33
85.15
85.15
99.72
Synthetic grass
98.81
99.01
100.0
99.21
99.80
99.80
99.80
99.41
99.60
99.60
100.0
Trees
92.90
92.14
91.95
91.95
98.01
98.11
97.63
91.00
92.61
87.59
93.37
Soil
100.0
100.0
100.0
100.0
100.0
100.0
100.0
99.52
99.90
100.0
100.0
Water
98.60
100.0
95.80
98.60
95.10
95.80
95.80
95.80
95.80
89.51
100.0
TABLE IV: Classification performance of different methods on the Houston2013 dataset. The bold entries denote the best results.
Class
FusAtNet
S2ENet
AsyFFNet
ExViT
HCT
MACN
MICF-Net
GCCQTNet
CHNet
MGMNet
SPCNet
Forest
93.78
98.10
96.01
90.04
96.92
97.95
95.98
95.26
96.24
96.28
96.84
Residential area
97.58
99.08
97.96
95.44
98.96
98.68
98.46
96.80
99.22
97.76
98.71
Industrial area
26.48
12.19
53.29
34.58
33.05
45.30
58.38
68.20
44.23
49.61
64.10
Low plants
97.67
91.78
91.74
90.68
92.36
94.24
91.75
91.52
93.83
92.32
97.08
Allotment
52.77
45.12
65.39
51.82
54.11
51.24
40.73
41.68
54.68
56.60
52.39
Commercial area
24.66
1.22
4.58
28.63
6.62
6.74
23.53
13.97
19.85
5.15
2.32
TABLE V: Classification performance of different methods on the Augsburg dataset. The bold entries denote the best results.
Fig. 10: Visualized classification results of different methods for the Augsburg dataset. (a) Ground truth. (b) FusAtNet. (c) S2ENet. (d) AsyFFNet. (e) ExViT. (f) HCT. (g) MACN. (h) MICF-Net. (i) GCCQTNet. (j) CHNet. (k) MGMNet. (l) SPCNet.
Class
FusAtNet
S2ENet
AsyFFNet
ExViT
HCT
MACN
MICF-Net
GCCQTNet
CHNet
MGMNet
SPCNet
Healthy grass
60.69
99.15
96.90
93.34
97.38
89.53
94.40
91.71
91.24
92.83
92.26
Stressed grass
93.85
84.35
91.82
93.44
89.40
93.50
89.79
94.22
93.94
93.51
94.92
Artificial turf
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
Evergreen trees
99.02
98.13
99.29
95.87
98.90
99.10
99.24
98.58
98.50
99.63
99.40
Deciduous trees
94.73
86.39
99.52
96.85
98.89
98.49
98.85
98.71
99.17
99.34
99.04
Bare earth
99.87
99.33
99.98
100.0
100.0
99.96
99.84
99.96
99.98
99.96
99.99
TABLE VI: Classification performance of different methods on the Houston2018 dataset. The bold entries denote the best results.
Fig. 11: Visualized classification results of different methods for the Houston2018 dataset. (a) Ground truth. (b) FusAtNet. (c) S2ENet. (d) AsyFFNet. (e) ExViT. (f) HCT. (g) MACN. (h) MICF-Net. (i) GCCQTNet. (j) CHNet. (k) MGMNet. (l) SPCNet.
Model Variant
Houston 2013
Augsburg
Houston 2018
Baseline
88.56
89.75
89.66
w/o SPCB
90.33
91.42
91.75
w/o RAMA
90.92
90.19
91.13
Full SPCNet
93.53
92.68
93.77
TABLE VII: Ablation study of the proposed SPCNet in terms of OA (%).
Model Variant
Houston 2013
Augsburg
Houston 2018
Shared-LK only
92.67
92.19
93.24
DK only
91.13
91.62
92.12
Static-DW (generator)
93.21
91.78
93.33
Static-DW (pointwise)
92.66
91.90
93.26
Full SPCB
93.53
92.68
93.77
TABLE VIII: Ablation study of SPCB in terms of OA (%).
Fusion Strategy
Houston 2013
Augsburg
Houston 2018
1×1 convolution
91.19
90.74
92.58
Global allocator
93.53
92.26
93.53
Regional allocation
93.29
92.31
93.63
RAMA
93.53
92.68
93.77
TABLE IX: Ablation study of RAMA in terms of OA (%).
Modality Shared
Depth Shared
Houston 2013
Augsburg
Houston 2018
×
×
90.46
91.42
92.28
✓
×
90.52
92.26
93.63
×
✓
91.34
91.76
93.33
✓
✓
93.53
92.68
93.77
TABLE X: Ablation study of the large-kernel sharing strategies in terms of OA (%).
Metrics
FusAtNet
S2ENet
AsyFFNet
ExViT
HCT
MACN
MICF-Net
GCCQTNet
CHNet
MGMNet
SPCNet
Params (M)
36.5760
1.1752
1.0654
0.2339
5.9186
2.3472
0.1332
1.2322
20.8156
4.3010
1.0893
FLOPs (G)
6.6950
0.2708
0.1747
0.0642
0.0849
0.3338
0.0377
0.0493
0.3214
0.1455
0.2470
TABLE XI: Comparison of model parameters, floating-point operations (FLOPs).
Hyperspectral image (HSI) and SAR/LiDAR data offer complementary spectral and structural information for land-cover classification. However, their effective fusion remains challenging due to two major limitations: The spectral redundancy in high-dimensional HSI and the heterogeneous characteristics between multi-source data. To this end, we propose Representative Spectral Correlation Network (RSCNet), a novel multi-source image classification framework specifically designed to address the above challenges through spectral selection and adaptive interaction. The network incorporates two key components: (1) Key Band Selection Module (KBSM) that adaptively selects task-relevant spectral bands from the original HSI under cross-source guidance, thereby alleviating redundancy and mitigating information loss from conventional PCA-based spectral reduction. Moreover, the learned band subset exhibits highly discriminative spectral structures that align with discriminative semantic cues, promoting compact yet expressive representations. (2) Cross-source Adaptive Fusion Module (CAFM) that performs cross-source attention weighting and local-global contextual refinement to enhance cross-source feature interaction. Experiments on three public benchmark datasets demonstrate that our RSCNet achieves superior performance compared with state-of-the-art methods, while maintaining substantially lower computational complexity. Our codes are publicly available at https://github.com/oucailab/RSCNet.
Chuanzheng Gong, Feng Gao, Junyan Lin +2
State Key Laboratory of Physical Oceanography, Ocean University of China, Qingdao 266100, China · Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China · Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA
Deep learning-based methods have achieved significant success in remote sensing Earth observation data analysis. Numerous feature fusion techniques address multimodal remote sensing image classification by integrating global and local features. However, these techniques often struggle to extract structural and detail features from heterogeneous and redundant multimodal images. With the goal of introducing frequency domain learning to model key and sparse detail features, this paper introduces the spatial-spectral-frequency interaction network (S2Fin), which integrates pairwise fusion modules across the spatial, spectral, and frequency domains. Specifically, we propose a high-frequency sparse enhancement transformer that employs sparse spatial-spectral attention to optimize the parameters of the high-frequency filter. Subsequently, a two-level spatial-frequency fusion strategy is introduced, comprising an adaptive frequency channel module that fuses low-frequency structures with enhanced high-frequency details, and a high-frequency resonance mask that emphasizes sharp edges via phase similarity. In addition, a spatial-spectral attention fusion module further enhances feature extraction at intermediate layers of the network. Experiments on four benchmark multimodal datasets with limited labeled data demonstrate that S2Fin performs superior classification, outperforming state-of-the-art methods. The code is available at https://github.com/HaoLiu-XDU/SSFin.
Hao Liu, Yunhao Gao, Wei Li +3
Department of Information Engineering and Computer Science, University of Trento, 38123 Trento, Italy · School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China · Beijing Key Laboratory of Fractional Signals and Systems, Beijing Institute of Technology, Beijing 100081, China +3
Multi-source remote sensing image classification has attracted increasing attention due to the complementary spectral, structural, and geometric information. However, existing methods still suffer from two limitations: insufficient semantic contextual modeling and unreliable feature fusion caused by slight spatial misalignment. To address these issues, we propose a Semantic-Guided Fusion Network (SGFNet) for multi-source remote sensing image classification. Specifically, the Semantic Mixing Convolution Block (SMCB) is designed to dynamically generate semantic-aware convolution kernels according to contextual relationships among feature representations. In addition, the Frequency Modulated Fusion Block (FMFB) is introduced to perform cross-modal interaction in the frequency domain, which effectively alleviates the influence of slight spatial misalignment and improves complementary information fusion. Extensive experiments conducted on the Augsburg and Houston 2018 datasets demonstrate that the proposed SGFNet consistently outperforms several state-of-the-art methods. The codes are publicly available at https://github.com/oucailab/SGFNet .
Yuwei Zhao, Chuanzheng Gong, Baogui Huan +3
State Key Laboratory of Physical Oceanography, Ocean University of China, Qingdao 266100, China · Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA