Remote Sensing Image Classification Using Deep Ensemble Learning
Authors: Niful Islam, Md. Rayhan Ahmed, Nur Mohammad Fahad, Salekul Islam, A. K. M. Muzahidul Islam, Saddam Mukta, Swakkhar Shatabda
Organizations: Department of Computer Science and Engineering, Oakland University, Michigan, USA · Irving K. Barber Faculty of Science, University of British Columbia, Kelowna, Canada · Department of Computer Science and Engineering, United International University, Dhaka, Bangladesh · Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh · Department of Software Engineering, LUT School of Engineering Sciences, LUT University, Finland · Department of Computer Science and Engineering, BRAC University, Dhaka, Bangladesh
Remote sensing imagery plays a crucial role in many applications and requires accurate computerized classification techniques. Reliable classification is essential for transforming raw imagery into structured and usable information. While Convolutional Neural Networks (CNNs) are mostly used for image classification, they excel at local feature extraction, but struggle to capture global contextual information. Vision Transformers (ViTs) address this limitation through self attention mechanisms that model long-range dependencies. Integrating CNNs and ViTs, therefore, leads to better performance than standalone architectures. However, the use of additional CNN and ViT components does not lead to further performance improvement and instead introduces a bottleneck caused by redundant feature representations. In this research, we propose a fusion model that combines the strengths of CNNs and ViTs for remote sensing image classification. To overcome the performance bottleneck, the proposed approach trains four independent fusion models that integrate CNN and ViT backbones and combine their outputs at the final prediction stage through ensembling. The proposed method achieves accuracy rates of 98.10 percent, 94.46 percent, and 95.45 percent on the UC Merced, RSSCN7, and MSRSI datasets, respectively. These results outperform competing architectures and highlight the effectiveness of the proposed solution, particularly due to its efficient use of computational resources during training.
Figures & tables
Paper
Dataset
Model
Drawbacks
Our Solution
Yu et al. [ 34 ]
UCM, AID, RSSCN7
Attention GAN
No comparison with other transfer learning models, lower classification accuracy.
An ablation study comparing other approaches is present. Has higher classification accuracy.
Shen et al. [ 14 ]
UCM, AID, PatternNet, OPTIMAL-31
ResNet50, DenseNet121, Attention Cascade Net
No transfer learning employed. Therefore, has a large number of trainable parameters.
We integrated pretrained models. Therefore, total trainable parameters are slightly above 8 million.
Ghadi et al. [ 17 ]
AID, UCM, ResisC45
fuzzy c- means, CNN, Feature fusion.
Incorporates machine learning algorithms that cannot be sped up by GPU. Therefore, very inefficient to train
The model can be sped up by GPU.
Alem and Kumar [ 18 ]
UCM
Custom CNN model
Has low classification accuracy.
Achieves higher classification accuracy.
Wang et al. [ 22 ]
UCM, AID
Multiple-Instance Learning with a Residual Dense Attention CNN
Too much complexity and requires more computational cost.
Less computational cost with 8.13M trainable parameters and 80 epochs of training.
Yang et al. [ 35 ]
AID, RSSCN7
Node Entropy
No proper prediction task using deep learning approach.
Experimented with numerous DL architectures and configured the optimal model.
Table 1: Comparison with existing works.
Figure 1: Flow diagram of the proposed method.
Figure 2: Proposed model architecture.
Figure 3: The ASPP module used in the model.
Figure 4: Samples from the UCM dataset.
Figure 5: Samples from the RSSCN7 dataset.
Figure 6: Samples from the MSRSI dataset.
Figure 7: Loss graphs of four fusion models used for ensembling on the UCM, RSSCN7, and MSRSI datasets.
Figure 8: Confusion matrix of the proposed model on the UCM dataset.
Figure 9: Confusion matrix of the proposed model on the RSSCN7 dataset.
Figure 10: Confusion matrix of the proposed model on the MSRSI dataset.
Figure 11: ROC curve of the proposed model on the UCM dataset.
Figure 12: ROC curve of the proposed model on the RSSCN7 dataset.
Figure 13: ROC curve of the proposed model on the MSRSI dataset.
Measure
UCM
RSSCN7
MSRSI
Accuracy
98.10
94.46
95.45
Precision
98.31
94.61
95.49
Recall
98.10
94.46
95.45
F1-Score
98.11
94.48
95.45
True Positive Rate
100.00
92.13
97.97
False Positive Rate
0.00
4.00
4.18
Table 3: Performance analysis of the proposed approach.
Model Name
UCM
RSSCN7
MSRSI
Total parameters
Trainable Parameters
Training Epoch
Xception
94.05
87.32
82.13
22.62 M
0.57 M
100
Inception-ResnetV2
92.86
88.57
80.71
54.9 M
0.56 M
100
XCiT
80.95
80.89
71.81
84 M
0.56 M
100
Swin Transformer
95.95
91.79
88.10
195.95 M
0.95 M
100
ViT Base
95.71
89.11
86.55
86.36 M
0.95 M
100
DeiT
90.48
91.07
84.15
86.65 M
0.56 M
100
Table 4: Comparison with the state-of-the-art methods.
Model Type
Model Structure
No of ViTs
No of CNNs
UCM
RSS-CN7
MS-RSI
Epo-chs
Fusion
ViT Base + Xception + ResNet152v2 + MobileNetV3Large
1
3
95.95
92.14
86.83
100
Fusion
ViT Base + Xception + ResNet152v2 + MobileNetV3Large + EffecientNetB7
1
4
94.29
92.57
84.70
100
Fusion
ViT Base + Xception + ResNet152v2 + MobileNetV3Large + EffecientNetB7 + DenseNet121
1
5
93.86
92.68
88.27
100
Fusion
ViT Base + InceptionResNetV2 + DenseNet121 + EffecientNetB7 + Xception + ResNet152v2 + MobileNetV3Large
1
6
96.67
92.86
87.78
100
Fusion
ViT Base +ViT Large + Xception + ResNet152v2 + MobileNetV3Large
2
3
97.86
91.43
87.55
100
Fusion
ViT Base + ViT Large + Xception + ResNet152v2 + MobileNetV3Large + EffecientNetB7
2
4
96.43
92.14
87.20
100
Table 5: Comparison with different methods.
Figure 14: Accuracy over changing number of CNNs and ViTs.
Model
Accuracy
Precision
Recall
F1-Score
Snapshot ensemble CNN, dropCycle [ 58 ]
97.38
-
-
-
Residual Network-based Features Extraction, Self-adaptive Global Best Harmony Search [ 59 ]
State Key Laboratory of Physical Oceanography, Ocean University of China, Qingdao 266100, China · Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA