Small Target Detection Based on Mask-Enhanced Attention Fusion of Visible and Infrared Remote Sensing Images

Qianqian Zhang; Xiaolong Jia; Ahmed M. Abdelmoniem; Li Zhou; Junshe An

cs.CVMar 6, 2026

Small Target Detection Based on Mask-Enhanced Attention Fusion of Visible and Infrared Remote Sensing Images

Authors: Qianqian Zhang, Xiaolong Jia, Ahmed M. Abdelmoniem, Li Zhou, Junshe An

Organizations: National Space Science Center, Chinese Academy of Sciences, Beijing, 100190, China · School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China · School of Electronic Engineering and Computer Science, Queen Mary University of London, London, UK · School of Astronomy and Space Science, University of Chinese Academy of Sciences, Beijing, China

Abstract

Targets in remote sensing images are usually small, weakly textured, and easily disturbed by complex backgrounds, challenging high-precision detection with general algorithms. Building on our earlier ESM-YOLO, this work presents ESM-YOLO+ as a lightweight visible infrared fusion network. To enhance detection, ESM-YOLO+ includes two key innovations. (1) A Mask-Enhanced Attention Fusion (MEAF) module fuses features at the pixel level via learnable spatial masks and spatial attention, effectively aligning RGB and infrared features, enhancing small-target representation, and alleviating cross-modal misalignment and scale heterogeneity. (2) Training-time Structural Representation (SR) enhancement provides auxiliary supervision to preserve fine-grained spatial structures during training, boosting feature discriminability without extra inference cost. Extensive experiments on the VEDAI and DroneVehicle datasets validate ESM-YOLO+'s superiority. The model achieves 84.71% mAP on VEDAI and 74.0% mAP on DroneVehicle, while greatly reducing model complexity, with 93.6% fewer parameters and 68.0% lower GFLOPs than the baseline. These results confirm that ESM-YOLO+ integrates strong performance with practicality for real-time deployment, providing an effective solution for high-performance small-target detection in complex remote sensing scenes.

Figures & tables

Figure 1: Overall architecture of the ESM-YOLO+. It comprises three key components: 1) the Mask-Enhanced Attention Fusion (MEAF) Module; 2) Detection Backbone and detection head; and 3) EDSR-based super-resolution branch used only during training to enhance spatial learning and removed at inference for faster detection.

Figure 2: Mask-Enhanced Attention Fusion (MEAF) Module. Pixel-level fusion module in the ESM-YOLO+ model.

Figure 3: Distribution of target sizes in the datasets.

Method	Car	Pickup	Camping	Truck	Other	Tractor	Boat	Van	$mAP_{50}$ ↑
YOLOv3 [ 20 ]	84.57	72.68	67.13	61.96	43.04	65.24	37.10	58.29	61.26
YOLOv4 [ 2 ]	85.46	72.84	72.38	62.82	48.94	68.99	34.28	54.66	62.55
YOLOv5s [ 27 ]	80.81	68.48	69.06	54.71	46.79	64.29	24.25	45.96	56.79
YOLOv5m [ 27 ]	82.53	72.32	68.41	59.25	46.20	66.23	33.51	57.11	60.69
YOLOv5l [ 27 ]	82.83	72.32	69.92	63.94	48.48	63.07	40.12	56.46	62.16
YOLOv5x [ 27 ]	84.33	72.95	70.09	61.15	49.94	67.35	38.71	56.65	62.65

Table 1: Comparative Experiments of the ESM-YOLO+ model on the VEDAI dataset.

Figure 4: Visual comparison between the ESM-YOLO Model and the ESM-YOLO+ Model. The red cycles represent the false alarms, the yellow ones denote the FP detection results, and the blue ones are FN detection results.

Figure 5: Comparison of P-R Curve for ESM-YOLO (a) and ESM-YOLO+ (b).

Figure 6: Comparison of F1 curves for ESM-YOLO (a) and ESM-YOLO+ (b).

	Car	Pickup	Camping	Truck	Other	Tractor	Boat	Van	$mAP_{50}$ ↑
Single-Mode (RGB)	89.31	87.31	69.71	83.34	61.04	80.08	54.17	75.52	75.06
Single-Mode (IR)	87.84	86.00	78.54	72.09	45.44	59.95	55.02	81.77	70.83
MultiModal (RGB+IR)	93.64	88.76	85.66	86.65	74.80	91.39	71.57	85.20	84.71

Table 2: Comparison of single modality and multimodalities (ESM-YOLO+) on the VEDAI dataset.

Figure 7: Visualization of BEF fusion and MEAF fusion results.

Method	Backbone	$mAP_{50}$ ↑	Params(M) ↓
CFT [ 31 ]	Transformer	85.3	196.9
ICAFusion [ 22 ]	Transformer	76.6	120.2
Cross-Channel-ViT [ 1 ]	Transformer	78.5	-
MOD-YOLO [ 21 ]	Transformer + CNN	59.3	24.9
MOD-YOLO-Tiny [ 21 ]	Transformer + CNN	58.4	16.0
ESM-YOLO+	CNN	84.7	5.1

Table 3: Comparative Experiments of the ESM-YOLO+ model on the VEDAI dataset.

Method	Car	Pickup	Camping	Truck	Other	Tractor	Boat	Van	$mAP_{50}$ ↑
MIR-YOLO [ 45 ]	88.8	77.2	70.8	68.7	71.0	72.8	60.8	47.8	69.7
ACDF-YOLO [ 9 ]	89.8	87.9	78.5	75.3	67.3	76.3	73.7	75.9	78.1
MMFDet [ 42 ]	88.3	78.5	81.6	59.8	63.5	86.2	76.0	88.3	77.9
FFCA-YOLO [ 41 ]	89.6	85.7	78.7	85.7	48.6	81.8	61.5	67.0	74.8
L-FFCA-YOLO [ 41 ]	91.3	85.5	72.8	79.7	47.3	79.0	56.1	73.9	73.3
ESM-YOLO+	93.6	88.8	85.7	86.7	74.8	91.4	71.6	85.2	84.7

Table 4: Comparison of Detection Accuracy Performance between Specialized Algorithms for Small Target Detection in Remote Sensing MultiModal Fusion and Our Method

Method	Architecture	Car	Truck	Bus	Van	Freight Car	$mAP_{50}$ ↑	Params(M) ↓	GFLOPs(G) ↓
UA-CMDet [ 25 ]	CNN	87.5	60.7	87.1	37.9	46.8	64.0	234.0	-
MKD [ 11 ]		93.5	62.5	91.9	44.5	52.7	69.0	242.0	-
Oriented R-CNN [ 33 ]		89.9	56.6	89.6	46.9	54.4	67.5	41.1	107.3
TSFADet [ 36 ]		89.9	67.9	89.9	54.0	63.7	73.1	104.7	109.8
C 2 Former-S 2 ANet [ 37 ]	Transformer	90.2	68.3	89.8	58.5	64.4	74.2	132.5	100.9
ViT-B+RVSA [ 30 ]		89.7	52.3	88.0	44.4	51.0	65.1	114.4	416.6

Table 5: Comparative Experiments of the ESM-YOLO+ model on the DroneVehicle dataset.

Figure 8: The F1 curve and P-R curve obtained by the ESM-YOLO+ model validated on the DroneVehicle dataset.

Figure 9: The visualization shows the comparison results of ESM-YOLO+ and other algorithms verified on the DroneVehicle dataset [ 25 ] . The red cycles represent the false alarms, the yellow ones denote the FP detection results, and the blue ones are FN detection results.

No.	EB	EB (-SR)	BEF	MEAF	$mAP_{50}$ ↑	Params(M) ↓
1	✓	-	-	-	72.68	80.16
2	✓	-	✓	-	82.42	80.18
3	-	✓	-	✓	84.71	5.14

Table 6: Ablation experiments on MEAF Module. EB: ESM-YOLO Backbone; EB (-SR): ESM-YOLO Backbone (SR removed during inference)

Remote Sensing Small Object Detection

Remote sensing small object detection finds tiny, crowded targets in aerial and satellite imagery despite low resolution, clutter, and scale variation.

See all 37 papers

LCMamNet: A Lightweight Cross-scale Mamba Network for Infrared Small Target Detection
Jul 27, 2026Yuhao Fan, Le Hui, Yuchao DaiImage Segmentation Infrared Small Target Detection
School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710072, China
Denoising-Enhanced Coarse-to-Fine Infrared Small Target Detection with Attention Prior-Guided Knowledge Distillation
Jun 20, 2026Houzhang Fang, Ruixuan Huang, Qiuhuan Chen +3Attention-Guided Distillation Infrared Small Target Detection
Xidian University, Xi’an, China · Huazhong University of Science and Technology, Wuhan, China
Point-to-Mask: From Arbitrary Point Annotations to Mask-Level Infrared Small Target Detection
Mar 17, 2026Weihua Gao, Wenlong Niu, Jie Tang +3Infrared Small Target Detection Object Detection
Key Laboratory of Electronics and Information Technology for Space Systems, National Space Science Center, Chinese Academy of Sciences, Beijing 100190, China. · School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100049, China.

Jul 27, 2026J/K move · Enter open · S save

LCMamNet: A Lightweight Cross-scale Mamba Network for Infrared Small Target Detection

Yuhao Fan, Le Hui, Yuchao Dai

School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710072, China

Image Segmentation Infrared Small Target Detection

Open paper AskPDF arXiv

Infrared small target detection (IRSTD) is important for low-altitude perception, unmanned-system warning, and security monitoring. However, weak targets in infrared imagery usually occupy only a few pixels and are easily submerged by cloud clutter, ground edges, and bright noise, making it difficult for lightweight segmentation-based methods to preserve local target structures while suppressing background interference. To address these challenges, we propose LCMamNet, a lightweight cross-scale Mamba network that progressively enhances local target structures, interacts cross-scale context in a latent space, and restores spatial details with background suppression. Specifically, a compact hierarchical encoder with cross-shaped directional bottleneck residual (CDBR) blocks strengthens direction-sensitive target structures under a small computation budget. A latent dense cross-scale fusion (LDCF) module then performs dense all-level interaction through bidirectional Mamba modeling and reorganizes the interacted features into stable hierarchical semantics. Finally, a progressive decoder selectively recovers shallow spatial details while suppressing irrelevant background textures. Extensive experiments on IRSTD-1k, NUAA-SIRST, and NUDT-SIRST show that the proposed network achieves mIoU scores of 71.25%, 79.60%, and 95.58%, respectively, with only 1.175M parameters and 6.91 GFLOPs. It also runs with a mean inference latency of 6.62 ms, and deployment results on an NVIDIA Jetson Orin NX 16G SUPER further demonstrate its practical potential for real-time edge inference. The code and checkpoints are publicly available at https://github.com/Haoyu096/LCMamNet.

Abstract

Remote Sensing Small Object Detection

Explore similar work

LCMamNet: A Lightweight Cross-scale Mamba Network for Infrared Small Target Detection

Denoising-Enhanced Coarse-to-Fine Infrared Small Target Detection with Attention Prior-Guided Knowledge Distillation

Point-to-Mask: From Arbitrary Point Annotations to Mask-Level Infrared Small Target Detection