MI-DETR: A Strong Baseline for Moving Infrared Small Target Detection with Motion Integration
Authors: Nian Liu, Jin Gao, Zhen Liang, Shubo Lin, Sikui Zhang, Fudong Ge, Liang Li, Weiming Hu
Organizations: School of Advanced Interdisciplinary Sciences (SAIS), University of Chinese Academy of Sciences (UCAS), Beijing 101408, China · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 101408, China · Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu 610209, China · University of Chinese Academy of Sciences (UCAS), Beijing 100049, China · Beijing Institute of Basic Medical Sciences, Beijing 100850, China · School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China
Detecting moving infrared small targets is challenging because tiny, low-contrast targets occupy few pixels and are easily obscured by dynamic backgrounds. Existing multi-frame methods aggregate temporal information across frames to capture motion. However, dynamic background changes can generate similar motion cues, making it difficult to distinguish between target motion and background interference. Furthermore, even when motion cues are extracted, combining them with current-frame appearance features remains difficult. To address these issues, we propose Motion Integration DETR (MI-DETR), a three-stage framework that explicitly models motion and fuses it with appearance features. First, to suppress background clutter while preserving target-related motion cues, Recurrent Interpretable Motion Cue Aggregation (RIMCA) maintains a recurrent temporal state that accumulates motion across consecutive frames, producing a causal and spatially aligned motion representation. Second, to integrate spatial and temporal information, Pathway Mutual Interaction (PMI) preserves separate appearance and motion pathways while enabling bidirectional feature exchange between them. Finally, an RT-DETR-based detector uses these refined features for end-to-end target localization. Experiments on DAUB-R, ITSDT-15K, and IRDST-H show that explicit motion modeling and pathway interaction effectively improve moving infrared small target detection.
Figures & tables
Fig. 1: MI-DETR combines 70.30% mAP 50 with 34.60 FPS on IRDST-H under the unified RTX 3090 protocol [ 7 ] .
Fig. 2: Overall architecture of MI-DETR. RIMCA constructs the motion representation, PMI performs bidirectional interaction between the appearance and motion pathways, and RT-DETR performs the final target classification and localization.
Fig. 3: Qualitative visualization of RIMCA over five consecutive infrared frames. (a, b) Appearance inputs without and with ground-truth (GT) annotations. (c, d) Corresponding RIMCA motion representations without and with GT annotations. Red boxes indicate GT targets, and blue boxes denote zoom-in regions.
Dataset
Train
Val
Test
Total
ITSDT-15K [ 43 ]
10,000
–
5,000
15,000
IRDST-H [ 28 ]
8,725
2,694
5,354
16,773
DAUB-R [ 28 ]
7,500
2,000
4,277
13,777
TABLE I: Summary of datasets used in our experiments. Train, Val, and Test denote the numbers of training, validation, and test samples, respectively; – indicates that no separate validation split is used.
Motion Representation
mAP 50 ↑
P ↑
R ↑
F1 ↑
Zero representation
87.55
85.83
78.32
81.90
Two-frame Difference [ 34 ]
93.94
90.59
86.54
88.52
Three-frame Difference [ 35 ]
90.11
86.12
81.45
83.72
Farnebäck [ 36 ]
94.07
94.57
85.32
89.71
MNWTH [ 38 ]
88.59
92.97
77.47
84.52
MOG2 [ 39 , 40 ]
84.36
83.84
74.79
79.06
TABLE II: Detection-oriented comparison of representative motion representations on DAUB-R under a fixed detector. Only the motion representation is changed, and all variants are reported with the common mAP 50 /P/R/F1 convention. Full RIMCA is the Stage I representation used by the complete MI-DETR model; its DAUB-R detector metrics are therefore identical to the MI-DETR entry in Table III .
Method
Temporal Context
Venue
DAUB-R
ITSDT-15K
IRDST-H
mAP 50
P
R
F1
mAP 50
P
R
F1
mAP 50
P
R
F1
Single-frame methods
SANet [ 48 ]
1
ICASSP 2023
83.64
91.62
92.26
91.94
62.17
87.78
71.23
78.64
33.02
51.86
64.50
57.49
AGPCNet [ 49 ]
1
IEEE TAES 2023
81.25
84.44
97.66
90.57
67.27
91.19
74.77
82.16
29.24
46.64
63.68
53.84
RDIAN [ 44 ]
1
IEEE TGRS 2023
82.55
87.65
95.23
91.28
68.49
90.56
76.06
82.68
30.57
42.18
73.50
53.60
DNANet [ 4 ]
1
IEEE TIP 2023
83.65
88.74
95.18
91.85
70.46
88.55
80.73
84.46
31.07
51.09
61.09
55.64
TABLE III: Comparison with representative methods on three moving infrared small-target benchmarks. Best and second-best results are shown in bold and underlined , respectively. Numeric temporal-context entries denote the number of input frames for single-frame and fixed-window methods; MI-DETR instead maintains a causal recurrent state whose historical contributions decay exponentially rather than being truncated to a fixed input window.
Fig. 4: Qualitative comparison on ITSDT-15K (confidence: 0.5, NMS: 0.3). Green boxes denote ground truth annotations, blue boxes indicate MI-DETR predictions, and red boxes show results from five multi-frame baselines (iMoPKL [ 28 ] , SSTNet [ 14 ] , STME [ 27 ] , Tridos [ 26 ] , TMP [ 16 ] ).
Fig. 5: Precision–Recall (PR) curves of MI-DETR and 11 representative methods on (a) ITSDT-15K , (b) DAUB-R , and (c) IRDST-H .
Method
T
mAP 50
P
R
F1
Par.
G
FPS
Single-frame methods
SIRST5K [ 50 ]
1
24.22
44.92
54.02
49.05
11.48
182.61
6.17
DNANet [ 4 ]
1
31.07
51.09
61.09
55.64
7.22
135.24
7.21
SCTransNet [ 53 ]
1
27.41
48.34
57.63
52.58
13.71
101.61
10.34
RPCANet [ 54 ]
1
29.17
45.17
65.18
53.36
3.21
382.69
14.81
MSHNet [ 51 ]
1
27.02
45.25
60.04
51.61
6.59
69.49
16.37
TABLE IV: Accuracy–complexity comparison on IRDST-H under the unified RTX 3090 protocol. T denotes temporal context: numeric entries are the numbers of input frames, while State denotes a recurrent-state method. Par. denotes millions of parameters, G denotes GFLOPs, and FPS denotes frames per second. MI-DETR FPS is measured for the complete RIMCA + detector pipeline.
ID
Readout
mAP 50
P
R
F1
Latency (ms)
FPS
R0
Zero representation
87.55
85.83
78.32
81.90
0.000
56.36
R1
Adapted response Pt
90.25
95.62
77.08
85.35
0.150
55.88
R2
Suppressed Ht
86.97
87.99
78.27
82.84
0.233
55.63
R3
Contrast Ct
87.92
80.02
80.90
80.45
0.312
55.38
R4
Recurrent state At
95.17
94.14
90.93
92.50
0.442
54.99
R5
Filtered response Gt
86.65
85.19
78.07
81.48
0.765
54.03
TABLE V: Intermediate RIMCA readout ablation on DAUB-R. R7 is the full RIMCA output. Latency is the cumulative RIMCA processing time per frame in milliseconds; profile FPS is measured on an A100 (40 GB). These two columns are separate from the unified RTX 3090 end-to-end result.
Fig. 6: One-factor-at-a-time sensitivity analysis of the main RIMCA configuration on DAUB-R. Panels (a)–(f) vary temporal retention α , temporal-response scale β , activation threshold θm , readout weight ηm , center–surround support km , and inhibition-kernel Gaussian width σG , respectively. The star marks the main-model configuration (98.00% mAP 50 and 94.35% F1); all remaining RIMCA and detector settings are fixed.
Parameter
Tested Values
Default
α
{0.50, 0.65, 0.80, 0.90, 0.95}
0.80
β
{0.60, 0.90, 1.20, 1.50, 1.80}
1.20
θm
{0.15, 0.225, 0.30, 0.375, 0.45}
0.30
ηm
{0.30, 0.50, 0.70, 0.85, 0.95}
0.70
km
{3, 5, 7}
5
σG (inhibition width)
{0.50, 1.00, 1.50}
1.00
TABLE VI: RIMCA parameters used in the one-parameter-at-a-time sensitivity analysis. The default column is the main-model configuration.
Variant
mAP 50
P
R
F1
G
Par.
Single-pathway baselines
(1) Appearance-only
90.20
87.50
81.70
84.50
57.20
19.88
(2) Motion-only
90.00
90.30
83.40
86.71
57.20
19.88
Non-interactive aggregation
(3) Element-wise Add
96.80
93.20
93.10
93.15
91.90
31.31
(4) Direct Concat
96.40
90.30
94.70
92.45
92.70
31.38
TABLE VII: Motion–appearance interaction ablation on DAUB-R. All variants use the same RT-DETR decoder, and variants with a motion pathway use the same RIMCA representations. LGAG denotes Large-kernel Grouped Attention Gate [ 57 ] . G denotes GFLOPs and Par. denotes millions of parameters.
Detector
Var.
mAP 50
P
R
F1
G
Par.
YOLOv8 [ 58 ]
App.
83.60
91.10
71.90
80.40
6.80
2.68
Mot.
86.90
84.70
78.90
81.70
6.80
2.68
+PMI
95.80
95.40
93.00
94.20
10.50
4.32
YOLOv10 [ 59 ]
App.
82.20
86.50
71.60
78.30
8.20
2.69
Mot.
80.90
80.90
71.80
76.10
8.20
2.69
+PMI
94.10
90.80
90.50
90.60
11.70
4.01
TABLE VIII: Cross-backbone evaluation of the dual-pathway RIMCA+PMI configuration on DAUB-R. Var. denotes the evaluated variant; App. and Mot. denote appearance-only and motion-only variants. R18 denotes ResNet-18. G denotes GFLOPs and Par. denotes millions of parameters. The +PMI rows include both the motion pathway and PMI; isolated PMI attribution is given in Table VII .
Infrared small target detection in dynamic scenes remains challenging due to the highly coupled motions among targets, imaging platforms, and dynamic backgrounds. Existing multi-frame methods usually perform implicit temporal modeling, where coherent background dynamics dominate motion correspondence learning, leading to an inherent trade-off between detection and false alarms. In this work, we observe that background motions exhibit strong global coherence, whereas small targets mainly correspond to sparse local motion anomalies. Moreover, many false-alarm responses maintain high consistency with globally coherent motion patterns, indicating that they mainly originate from coherent background dynamics rather than genuine target motions. Based on these observations, we propose a decoupled motion representation learning framework for moving infrared small target detection. Specifically, an explicit motion branch is introduced to model globally coherent motion dynamics using pretrained optical flow priors, together with a structure-preserving self-supervised adaptation strategy for infrared motion correspondence learning. Meanwhile, an implicit motion branch based on deformable feature alignment is designed to capture target-sensitive local motion anomalies under coherent motion guidance. Furthermore, a coherent-motion-guided local anomaly reasoning module is proposed to identify and suppress coherent-motion-induced false responses during localized motion modeling. Extensive experiments on two challenging infrared small target detection benchmarks demonstrate that the proposed method consistently outperforms existing state-of-the-art approaches, particularly in dynamic scenes with complex motions, while maintaining favorable inference efficiency.
Guoyi Zhang, Peiwen Wu, Han Wang +2
School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen 518107, Guangdong, China
Accurately localizing and segmenting small targets in low signal-to-noise ratio (SNR) infrared sequences remains a challenging task. Since targets are often indistinguishable from the background in individual frames, existing methods, even when equipped with advanced foundation model and powerful inter-frame association mechanisms, still fail to detect them. Motivated by the observation that targets tend to emerge gradually from the background over time and become distinguishable, we propose Temporal-Emerged Prompting for Segment Anything Model (TEP-SAM), a principled framework designed to explicitly exploit such temporal-emerged cues to modulate and prompt SAM. TEP-SAM operates by jointly modeling global motion patterns and local motion deviations to locate potential targets. It further enhances target region features by leveraging motion discrepancy, thereby generating temporal-emerged cues for SAM and enabling non-interactive segmentation. By bridging large-scale semantic pretraining with task-specific temporal modeling, TEP-SAM effectively adapts SAM to the challenging multiframe infrared small target detection task. Extensive experiments demonstrate the effectiveness of our approach, particularly under severely low-SNR conditions and in complex dynamic background.
Infrared small target detection (IRSTD) in high-resolution images is crucial for unmanned aerial vehicle (UAV) surveillance and UAV-based ground monitoring. However, small target size, weak features, and interference from complex dynamic backgrounds make IRSTD challenging. Existing methods incur redundant computation in non-target background regions and insufficiently exploit target context, limiting detection performance. To address these issues, we propose ECFNet, an efficient coarse-to-fine IRSTD framework with attention prior-guided knowledge distillation. In the coarse stage, we design a region binary classification network (RBCN) on grid-based multi-scale feature maps to efficiently identify target-containing context region proposals. A new denoising-assisted training strategy incorporates noisy ground-truth (GT) masks into RBCN feature maps and trains the network to reconstruct the original GT masks. This auxiliary task encourages explicit learning of target-background context to better distinguish target proposals from background regions. In the fine stage, we customize a lightweight target detector to the coarse-stage region proposals to balance accuracy and efficiency. Furthermore, we introduce a knowledge distillation strategy guided by a teacher-student cross-attention prior. This strategy directs the student to focus on critical target regions, enhancing discriminative feature representations for infrared small targets. Extensive experiments on three real infrared datasets demonstrate that ECFNet outperforms existing single-stage and two-stage approaches while maintaining high real-time processing efficiency. Code: https://github.com/IVPLabs/ECFNet.
Houzhang Fang, Ruixuan Huang, Qiuhuan Chen +3
Xidian University, Xi’an, China · Huazhong University of Science and Technology, Wuhan, China