Small Object Detection Model with Spatial Laplacian Pyramid Attention and Multi-Scale Features Enhancement in Aerial Images
Authors: Zhangjian Ji, Huijia Yan, Shaotong Qiao, Kai Feng, Wei Wei
Organizations: Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, School of Computer and Information Technology, Shanxi University, Taiyuan 030006, China
Detecting objects in aerial images confronts some significant challenges, including small size, dense and non-uniform distribution of objects over high-resolution images, which makes detection inefficient. Thus, in this paper, we proposed a small object detection algorithm based on a Spatial Laplacian Pyramid Attention and Multi-Scale Feature Enhancement in aerial images. Firstly, in order to improve the feature representation of ResNet-50 on small objects, we presented a novel Spatial Laplacian Pyramid Attention (SLPA) module, which is integrated after each stage of ResNet-50 to identify and emphasize important local regions. Secondly, to enhance the model's semantic understanding and features representation, we designed a Multi-Scale Feature Enhancement Module (MSFEM), which is incorporated into the lateral connections of C5 layer for building Feature Pyramid Network (FPN). Finally, the features representation quality of traditional feature pyramid network will be affected because the features are not aligned when the upper and lower layers are fused. In order to handle it, we utilized deformable convolutions to align the features in the fusion processing of the upper and lower levels of the Feature Pyramid Network, which can help enhance the model's ability to detect and recognize small objects. The extensive experimental results on two benchmark datasets: VisDrone and DOTA demonstrate that our improved model performs better for small object detection in aerial images compared to the original algorithm.
Figures & tables
Fig. 1 : The improved ResNet50 architecture after inserting our proposed SLPA module.
Fig. 2 : The architecture of the improved FasterRCNN-FPN.
Fig. 3 : The whole process of multi-scale feature enhancement module.
methods
Input size
AP(%)
AP50 (%)
AP75 (%)
APs (%)
APm (%)
APl (%)
Baseline
1200×1999
33.2
58.3
33.2
26.1
42.6
43.4
Baseline+CBAM
1200×1999
33.9
59.0
34.0
26.6
43.5
44.6
Baseline+SLPA
1200×1999
34.3
59.5
34.8
26.9
43.9
44.4
TABLE I : Comparison between our proposed SLPA module and the CBAM on the VisDrone dataset. The best results are highlighted by the bold.
SLPA
MSFEM
DCN
AP(%)
AP50 (%)
AP75 (%)
APs (%)
APm (%)
APl (%)
33.2
58.3
33.2
26.1
42.6
43.4
√
34.3
59.5
34.8
27.1
43.9
44.2
√
34.5
59.7
35.0
26.9
43.5
44.5
√
34.7
60.1
35.3
27.2
44.3
41.6
√
√
34.7
60.3
34.5
27.0
44.0
44.9
√
√
35.1
60.6
35.9
27.8
45.0
44.6
TABLE II : Ablation study of three different modules on the VisDrone dataset, conducted by successively incorporating them into the baseline. The best results are highlighted by the bold.
Dilatation rates (r)
AP(%)
AP50 (%)
AP75 (%)
APs (%)
APm (%)
APl (%)
{1,2,3}
34.3
59.5
34.8
27.1
43.9
44.4
{1,2,4}
34.2
59.1
34.8
26.8
43.7
44.1
{1,2,5}
34.3
59.0
34.8
27.0
44.2
44.2
{1,3,5}
34.0
58.9
34.6
26.7
43.8
43.5
{1,3,6}
33.9
58.6
34.6
26.5
44.1
44.5
{1,3,9}
34.0
58.9
34.7
26.7
43.8
44.1
TABLE III : Ablation study of different dilatation rate configurations within our SLPA module. The best results are highlighted by the bold.
Dilatation rates (r)
AP(%)
AP50 (%)
AP75 (%)
APs (%)
APm (%)
APl (%)
{1,2,3,4}
34.5
59.7
35.0
26.9
43.5
44.5
{1,2,3,5}
34.3
59.6
34.2
26.1
43.2
42.6
{1,2,4,6}
34.4
59.7
34.5
26.5
44.1
42.7
{1,3,5,7}
34.1
59.5
34.1
26.0
43.2
42.3
TABLE IV : Ablation study of different dilatation rate configurations within our MSFEM module. The best results are highlighted by the bold.
Methods
Backbone
Input size
AP(%)
AP50 (%)
AP75 (%)
APs (%)
APm (%)
APl (%)
One-stage methods
QueryNet [ 22 ]
ResNet-50
-
28.3
48.1
28.8
19.8
35.9
40.3
SDPNet [ 51 ]
ResNet-50
-
33.7
56.6
34.3
26.7
42.9
45.7
Uniform crop [ 7 ]
ResNet-50
1024×1024
31.7
56.3
31.6
25.1
40.4
41.1
LMFF-MFEE [ 52 ]
YOLOv8s
640×640
27.2
44.5
-
-
-
-
Two-stage methods
TABLE V : Comparison results on the VisDrone2019 dataset. The symbol * denotes that the corresponding object detection methods use the improved backbone network, that is, our proposed SLPA module is plugged into each stage of ResNet-50. The symbol † indicates that the results of the corresponding object detection methods after their input image size is adjusted from 1000×1999 to 600×1000 . ‘MF’ stands for model fusion. The best result of one column is highlighted by the bold.
Methods
Backbone
FLOPs/G
Params/M
FPS
AP
CZ Det [ 10 ]
ResNet-50
213.12
100.7
12.0
33.2
CZ Det+ours
ResNet-50+SLPA
218.22
107.8
11.4
35.3
TABLE VI : Complexity comparison between the CZ Det and the improved CZ Det model with our proposed modules.
Methods
Backbone
Input size
AP(%)
AP50 (%)
AP75 (%)
APs (%)
APm (%)
APl (%)
FPS
Uniform crop [ 7 ]
ResNet-50
1024×1024
33.4
54.0
35.6
16.9
36.8
43.7
0.49
ClusterNet [ 8 ]
ResNet-50
1000×1000
32.2
47.6
39.2
16.6
32.0
50.0
-
CZ Det [ 10 ]
ResNet-50
1200×1999
34.6
56.9
36.2
18.2
37.8
43.8
0.30
CZ Det+ours
ResNet-50+SLPA
1200×1999
35.0
57.6
36.8
20.2
38.3
43.6
0.24
TABLE VII : Comparison results of the different object detection algorithms on DOTA-v1.0 dataset. The best result of one column is highlighted by the bold.
Fig. 4 : Visualization results of the original CZ Det (the first row) and the improved CZ Det (the second row) on several representative scene images.