Remote Sensing Image Classification Using Deep Ensemble Learning
Authors: Niful Islam, Md. Rayhan Ahmed, Nur Mohammad Fahad, Salekul Islam, A. K. M. Muzahidul Islam, Saddam Mukta, Swakkhar Shatabda
Organizations: Department of Computer Science and Engineering, Oakland University, Michigan, USA · Irving K. Barber Faculty of Science, University of British Columbia, Kelowna, Canada · Department of Computer Science and Engineering, United International University, Dhaka, Bangladesh · Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh · Department of Software Engineering, LUT School of Engineering Sciences, LUT University, Finland · Department of Computer Science and Engineering, BRAC University, Dhaka, Bangladesh
Remote sensing imagery plays a crucial role in many applications and requires accurate computerized classification techniques. Reliable classification is essential for transforming raw imagery into structured and usable information. While Convolutional Neural Networks (CNNs) are mostly used for image classification, they excel at local feature extraction, but struggle to capture global contextual information. Vision Transformers (ViTs) address this limitation through self attention mechanisms that model long-range dependencies. Integrating CNNs and ViTs, therefore, leads to better performance than standalone architectures. However, the use of additional CNN and ViT components does not lead to further performance improvement and instead introduces a bottleneck caused by redundant feature representations. In this research, we propose a fusion model that combines the strengths of CNNs and ViTs for remote sensing image classification. To overcome the performance bottleneck, the proposed approach trains four independent fusion models that integrate CNN and ViT backbones and combine their outputs at the final prediction stage through ensembling. The proposed method achieves accuracy rates of 98.10 percent, 94.46 percent, and 95.45 percent on the UC Merced, RSSCN7, and MSRSI datasets, respectively. These results outperform competing architectures and highlight the effectiveness of the proposed solution, particularly due to its efficient use of computational resources during training.
Figures & tables
Paper
Dataset
Model
Drawbacks
Our Solution
Yu et al. [ 34 ]
UCM, AID, RSSCN7
Attention GAN
No comparison with other transfer learning models, lower classification accuracy.
An ablation study comparing other approaches is present. Has higher classification accuracy.
Shen et al. [ 14 ]
UCM, AID, PatternNet, OPTIMAL-31
ResNet50, DenseNet121, Attention Cascade Net
No transfer learning employed. Therefore, has a large number of trainable parameters.
We integrated pretrained models. Therefore, total trainable parameters are slightly above 8 million.
Ghadi et al. [ 17 ]
AID, UCM, ResisC45
fuzzy c- means, CNN, Feature fusion.
Incorporates machine learning algorithms that cannot be sped up by GPU. Therefore, very inefficient to train
The model can be sped up by GPU.
Alem and Kumar [ 18 ]
UCM
Custom CNN model
Has low classification accuracy.
Achieves higher classification accuracy.
Wang et al. [ 22 ]
UCM, AID
Multiple-Instance Learning with a Residual Dense Attention CNN
Too much complexity and requires more computational cost.
Less computational cost with 8.13M trainable parameters and 80 epochs of training.
Yang et al. [ 35 ]
AID, RSSCN7
Node Entropy
No proper prediction task using deep learning approach.
Experimented with numerous DL architectures and configured the optimal model.
Table 1: Comparison with existing works.
Figure 1: Flow diagram of the proposed method.
Figure 2: Proposed model architecture.
Figure 3: The ASPP module used in the model.
Figure 4: Samples from the UCM dataset.
Figure 5: Samples from the RSSCN7 dataset.
Figure 6: Samples from the MSRSI dataset.
Figure 7: Loss graphs of four fusion models used for ensembling on the UCM, RSSCN7, and MSRSI datasets.
Figure 8: Confusion matrix of the proposed model on the UCM dataset.
Figure 9: Confusion matrix of the proposed model on the RSSCN7 dataset.
Figure 10: Confusion matrix of the proposed model on the MSRSI dataset.
Figure 11: ROC curve of the proposed model on the UCM dataset.
Figure 12: ROC curve of the proposed model on the RSSCN7 dataset.
Figure 13: ROC curve of the proposed model on the MSRSI dataset.
Measure
UCM
RSSCN7
MSRSI
Accuracy
98.10
94.46
95.45
Precision
98.31
94.61
95.49
Recall
98.10
94.46
95.45
F1-Score
98.11
94.48
95.45
True Positive Rate
100.00
92.13
97.97
False Positive Rate
0.00
4.00
4.18
Table 3: Performance analysis of the proposed approach.
Model Name
UCM
RSSCN7
MSRSI
Total parameters
Trainable Parameters
Training Epoch
Xception
94.05
87.32
82.13
22.62 M
0.57 M
100
Inception-ResnetV2
92.86
88.57
80.71
54.9 M
0.56 M
100
XCiT
80.95
80.89
71.81
84 M
0.56 M
100
Swin Transformer
95.95
91.79
88.10
195.95 M
0.95 M
100
ViT Base
95.71
89.11
86.55
86.36 M
0.95 M
100
DeiT
90.48
91.07
84.15
86.65 M
0.56 M
100
Table 4: Comparison with the state-of-the-art methods.
Model Type
Model Structure
No of ViTs
No of CNNs
UCM
RSS-CN7
MS-RSI
Epo-chs
Fusion
ViT Base + Xception + ResNet152v2 + MobileNetV3Large
1
3
95.95
92.14
86.83
100
Fusion
ViT Base + Xception + ResNet152v2 + MobileNetV3Large + EffecientNetB7
1
4
94.29
92.57
84.70
100
Fusion
ViT Base + Xception + ResNet152v2 + MobileNetV3Large + EffecientNetB7 + DenseNet121
1
5
93.86
92.68
88.27
100
Fusion
ViT Base + InceptionResNetV2 + DenseNet121 + EffecientNetB7 + Xception + ResNet152v2 + MobileNetV3Large
1
6
96.67
92.86
87.78
100
Fusion
ViT Base +ViT Large + Xception + ResNet152v2 + MobileNetV3Large
2
3
97.86
91.43
87.55
100
Fusion
ViT Base + ViT Large + Xception + ResNet152v2 + MobileNetV3Large + EffecientNetB7
2
4
96.43
92.14
87.20
100
Table 5: Comparison with different methods.
Figure 14: Accuracy over changing number of CNNs and ViTs.
Model
Accuracy
Precision
Recall
F1-Score
Snapshot ensemble CNN, dropCycle [ 58 ]
97.38
-
-
-
Residual Network-based Features Extraction, Self-adaptive Global Best Harmony Search [ 59 ]
Land Use Scene Classification (LUSC) from remote sensing imagery plays a critical role in environmental monitoring, urban planning, and sustainable resource management. In recent years, deep learning methods have significantly advanced the state of the art, with Convolutional Neural Networks (CNNs) dominating the field because of their strong ability to capture local spatial features. However, the emergence of Vision Transformers (ViTs) has introduced a new paradigm that models long-range dependencies through self-attention mechanisms, potentially enabling improved global context understanding. This paper presents a comparative assessment of Vision Transformers and CNN-based architecture for remote sensing land use scene classification. Representative CNN models, such as AlexNet, is evaluated alongside the Vision Transformer (ViT) using benchmark remote sensing datasets, including the UC Merced Land Use and EuroSAT Land Use datasets. The study examines classification accuracy, precision, recall, F1-score, and computational complexity to provide a comprehensive performance comparison. Experimental results demonstrate that CNNs perform robustly on datasets with limited training samples and strong local texture characteristics, whereas Vision Transformers exhibit superior performance in capturing global spatial relationships in complex scenes when sufficient training data are available. However, ViTs typically require greater computational resources and larger training datasets to achieve optimal performance. The findings of this study provide insights into the strengths and limitations of both architectures and offer guidance for selecting appropriate models for remote sensing land use scene classification applications.
Arun D. Kulkarni
Computer Science Department, University of Texas at Tyler, Tyler, TX 75799, USA
Multi-source remote sensing image classification has attracted increasing attention due to the complementary spectral, structural, and geometric information. However, existing methods still suffer from two limitations: insufficient semantic contextual modeling and unreliable feature fusion caused by slight spatial misalignment. To address these issues, we propose a Semantic-Guided Fusion Network (SGFNet) for multi-source remote sensing image classification. Specifically, the Semantic Mixing Convolution Block (SMCB) is designed to dynamically generate semantic-aware convolution kernels according to contextual relationships among feature representations. In addition, the Frequency Modulated Fusion Block (FMFB) is introduced to perform cross-modal interaction in the frequency domain, which effectively alleviates the influence of slight spatial misalignment and improves complementary information fusion. Extensive experiments conducted on the Augsburg and Houston 2018 datasets demonstrate that the proposed SGFNet consistently outperforms several state-of-the-art methods. The codes are publicly available at https://github.com/oucailab/SGFNet .
Yuwei Zhao, Chuanzheng Gong, Baogui Huan +3
State Key Laboratory of Physical Oceanography, Ocean University of China, Qingdao 266100, China · Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA
Deep learning has become increasingly important in remote sensing image classification due to its ability to extract semantic information from complex data. Classification tasks often include predefined label hierarchies that represent the semantic relationships among classes. However, these hierarchies are frequently overlooked, and most approaches focus only on fine-grained classification schemes. In this paper, we present a novel Semantics-Aware Hierarchical Consensus (SAHC) approach that integrates hierarchical-level-specific classification heads within a deep network architecture and combines their output through cross-level probability projectors. Direct and projected predictions are fused into a geometric consensus distribution, which is used for self-consistent training and optional hierarchy-aware inference. This mechanism acts as a geometric ensemble that leverages the inherent structure of the hierarchical classification task. The projectors are initialized from the user-defined taxonomy (i.e., the hierarchical label structure), and can be adaptively refined during optimization. The proposed SAHC method is evaluated on two benchmark datasets with different degrees of hierarchical complexity on different tasks, considering varying spectral and spatial resolutions. Experimental results show both the effectiveness of the proposed approach in guiding network learning and the robustness of the hierarchical consensus for remote sensing image classification tasks. The source code is available at https://github.com/rslab-unitrento/sahc.
Giulio Weikmann, Gianmarco Perantoni, Lorenzo Bruzzone
Department of Information Engineering and Computer Science, University of Trento, 38123 Trento, Italy