MI-DETR: A Strong Baseline for Moving Infrared Small Target Detection with Motion Integration

Nian Liu; Jin Gao; Zhen Liang; Shubo Lin; Sikui Zhang; Fudong Ge; Liang Li; Weiming Hu

cs.CVMar 5, 2026

MI-DETR: A Strong Baseline for Moving Infrared Small Target Detection with Motion Integration

Authors: Nian Liu, Jin Gao, Zhen Liang, Shubo Lin, Sikui Zhang, Fudong Ge, Liang Li, Weiming Hu

Organizations: School of Advanced Interdisciplinary Sciences (SAIS), University of Chinese Academy of Sciences (UCAS), Beijing 101408, China · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 101408, China · Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu 610209, China ·

Abstract

Detecting moving infrared small targets is challenging because tiny, low-contrast targets occupy few pixels and are easily obscured by dynamic backgrounds. Existing multi-frame methods aggregate temporal information across frames to capture motion. However, dynamic background changes can generate similar motion cues, making it difficult to distinguish between target motion and background interference. Furthermore, even when motion cues are extracted, combining them with current-frame appearance features remains difficult. To address these issues, we propose Motion Integration DETR (MI-DETR), a three-stage framework that explicitly models motion and fuses it with appearance features. First, to suppress background clutter while preserving target-related motion cues, Recurrent Interpretable Motion Cue Aggregation (RIMCA) maintains a recurrent temporal state that accumulates motion across consecutive frames, producing a causal and spatially aligned motion representation. Second, to integrate spatial and temporal information, Pathway Mutual Interaction (PMI) preserves separate appearance and motion pathways while enabling bidirectional feature exchange between them. Finally, an RT-DETR-based detector uses these refined features for end-to-end target localization. Experiments on DAUB-R, ITSDT-15K, and IRDST-H show that explicit motion modeling and pathway interaction effectively improve moving infrared small target detection.

Figures & tables

Fig. 1: MI-DETR combines 70.30% mAP 50 with 34.60 FPS on IRDST-H under the unified RTX 3090 protocol [ 7 ] .

Fig. 2: Overall architecture of MI-DETR. RIMCA constructs the motion representation, PMI performs bidirectional interaction between the appearance and motion pathways, and RT-DETR performs the final target classification and localization.

Fig. 3: Qualitative visualization of RIMCA over five consecutive infrared frames. (a, b) Appearance inputs without and with ground-truth (GT) annotations. (c, d) Corresponding RIMCA motion representations without and with GT annotations. Red boxes indicate GT targets, and blue boxes denote zoom-in regions.

Dataset	Train	Val	Test	Total
ITSDT-15K [ 43 ]	10,000	–	5,000	15,000
IRDST-H [ 28 ]	8,725	2,694	5,354	16,773
DAUB-R [ 28 ]	7,500	2,000	4,277	13,777

TABLE I: Summary of datasets used in our experiments. Train, Val, and Test denote the numbers of training, validation, and test samples, respectively; – indicates that no separate validation split is used.

Motion Representation	mAP 50 $\uparrow$	P $\uparrow$	R $\uparrow$	F1 $\uparrow$
Zero representation	87.55	85.83	78.32	81.90
Two-frame Difference [ 34 ]	93.94	90.59	86.54	88.52
Three-frame Difference [ 35 ]	90.11	86.12	81.45	83.72
Farnebäck [ 36 ]	94.07	94.57	85.32	89.71
MNWTH [ 38 ]	88.59	92.97	77.47	84.52
MOG2 [ 39 , 40 ]	84.36	83.84	74.79	79.06

TABLE II: Detection-oriented comparison of representative motion representations on DAUB-R under a fixed detector. Only the motion representation is changed, and all variants are reported with the common mAP 50 /P/R/F1 convention. Full RIMCA is the Stage I representation used by the complete MI-DETR model; its DAUB-R detector metrics are therefore identical to the MI-DETR entry in Table III .

Method	Temporal Context	Venue	DAUB-R				ITSDT-15K				IRDST-H
Method	Temporal Context	Venue	mAP 50	P	R	F1	mAP 50	P	R	F1	mAP 50	P	R	F1
Single-frame methods
SANet [ 48 ]	1	ICASSP 2023	83.64	91.62	92.26	91.94	62.17	87.78	71.23	78.64	33.02	51.86	64.50	57.49
AGPCNet [ 49 ]	1	IEEE TAES 2023	81.25	84.44	97.66	90.57	67.27	91.19	74.77	82.16	29.24	46.64	63.68	53.84
RDIAN [ 44 ]	1	IEEE TGRS 2023	82.55	87.65	95.23	91.28	68.49	90.56	76.06	82.68	30.57	42.18	73.50	53.60
DNANet [ 4 ]	1	IEEE TIP 2023	83.65	88.74	95.18	91.85	70.46	88.55	80.73	84.46	31.07	51.09	61.09	55.64

TABLE III: Comparison with representative methods on three moving infrared small-target benchmarks. Best and second-best results are shown in bold and underlined , respectively. Numeric temporal-context entries denote the number of input frames for single-frame and fixed-window methods; MI-DETR instead maintains a causal recurrent state whose historical contributions decay exponentially rather than being truncated to a fixed input window.

Fig. 4: Qualitative comparison on ITSDT-15K (confidence: 0.5, NMS: 0.3). Green boxes denote ground truth annotations, blue boxes indicate MI-DETR predictions, and red boxes show results from five multi-frame baselines (iMoPKL [ 28 ] , SSTNet [ 14 ] , STME [ 27 ] , Tridos [ 26 ] , TMP [ 16 ] ).

Fig. 5: Precision–Recall (PR) curves of MI-DETR and 11 representative methods on (a) ITSDT-15K , (b) DAUB-R , and (c) IRDST-H .

Method	T	mAP 50	P	R	F1	Par.	G	FPS
Single-frame methods
SIRST5K [ 50 ]	1	24.22	44.92	54.02	49.05	11.48	182.61	6.17
DNANet [ 4 ]	1	31.07	51.09	61.09	55.64	7.22	135.24	7.21
SCTransNet [ 53 ]	1	27.41	48.34	57.63	52.58	13.71	101.61	10.34
RPCANet [ 54 ]	1	29.17	45.17	65.18	53.36	3.21	382.69	14.81
MSHNet [ 51 ]	1	27.02	45.25	60.04	51.61	6.59	69.49	16.37

TABLE IV: Accuracy–complexity comparison on IRDST-H under the unified RTX 3090 protocol.

T

denotes temporal context: numeric entries are the numbers of input frames, while State denotes a recurrent-state method. Par. denotes millions of parameters, G denotes GFLOPs, and FPS denotes frames per second. MI-DETR FPS is measured for the complete RIMCA + detector pipeline.

ID	Readout	mAP 50	P	R	F1	Latency (ms)	FPS
R0	Zero representation	87.55	85.83	78.32	81.90	0.000	56.36
R1	Adapted response $P_{t}$	90.25	95.62	77.08	85.35	0.150	55.88
R2	Suppressed $H_{t}$	86.97	87.99	78.27	82.84	0.233	55.63
R3	Contrast $C_{t}$	87.92	80.02	80.90	80.45	0.312	55.38
R4	Recurrent state $A_{t}$	95.17	94.14	90.93	92.50	0.442	54.99
R5	Filtered response $G_{t}$	86.65	85.19	78.07	81.48	0.765	54.03

TABLE V: Intermediate RIMCA readout ablation on DAUB-R. R7 is the full RIMCA output. Latency is the cumulative RIMCA processing time per frame in milliseconds; profile FPS is measured on an A100 (40 GB). These two columns are separate from the unified RTX 3090 end-to-end result.

Fig. 6: One-factor-at-a-time sensitivity analysis of the main RIMCA configuration on DAUB-R. Panels (a)–(f) vary temporal retention

\alpha

, temporal-response scale

\beta

, activation threshold

\theta_{m}

, readout weight

\eta_{m}

, center–surround support

k_{m}

, and inhibition-kernel Gaussian width

\sigma_{G}

, respectively. The star marks the main-model configuration (98.00% mAP 50 and 94.35% F1); all remaining RIMCA and detector settings are fixed.

Parameter	Tested Values	Default
$\alpha$	{0.50, 0.65, 0.80, 0.90, 0.95}	0.80
$\beta$	{0.60, 0.90, 1.20, 1.50, 1.80}	1.20
$\theta_{m}$	{0.15, 0.225, 0.30, 0.375, 0.45}	0.30
$\eta_{m}$	{0.30, 0.50, 0.70, 0.85, 0.95}	0.70
$k_{m}$	{3, 5, 7}	5
$\sigma_{G}$ (inhibition width)	{0.50, 1.00, 1.50}	1.00

TABLE VI: RIMCA parameters used in the one-parameter-at-a-time sensitivity analysis. The default column is the main-model configuration.

Variant	mAP 50	P	R	F1	G	Par.
Single-pathway baselines
(1) Appearance-only	90.20	87.50	81.70	84.50	57.20	19.88
(2) Motion-only	90.00	90.30	83.40	86.71	57.20	19.88
Non-interactive aggregation
(3) Element-wise Add	96.80	93.20	93.10	93.15	91.90	31.31
(4) Direct Concat	96.40	90.30	94.70	92.45	92.70	31.38

TABLE VII: Motion–appearance interaction ablation on DAUB-R. All variants use the same RT-DETR decoder, and variants with a motion pathway use the same RIMCA representations. LGAG denotes Large-kernel Grouped Attention Gate [ 57 ] . G denotes GFLOPs and Par. denotes millions of parameters.

Detector	Var.	mAP 50	P	R	F1	G	Par.
YOLOv8 [ 58 ]	App.	83.60	91.10	71.90	80.40	6.80	2.68
	Mot.	86.90	84.70	78.90	81.70	6.80	2.68
	+PMI	95.80	95.40	93.00	94.20	10.50	4.32
YOLOv10 [ 59 ]	App.	82.20	86.50	71.60	78.30	8.20	2.69
	Mot.	80.90	80.90	71.80	76.10	8.20	2.69
	+PMI	94.10	90.80	90.50	90.60	11.70	4.01

TABLE VIII: Cross-backbone evaluation of the dual-pathway RIMCA+PMI configuration on DAUB-R. Var. denotes the evaluated variant; App. and Mot. denote appearance-only and motion-only variants. R18 denotes ResNet-18. G denotes GFLOPs and Par. denotes millions of parameters. The +PMI rows include both the motion pathway and PMI; isolated PMI attribution is given in Table VII .

Infrared Small Target Detection

Infrared small target detection finds faint, often sub-pixel objects against cluttered thermal backgrounds while limiting false alarms.

See all 43 papers

Decoupled Motion Representation Learning for Moving Infrared Small Target Detection
Jun 13, 2026Guoyi Zhang, Peiwen Wu, Han Wang +2Representation Learning Infrared Small Target Detection
School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen 518107, Guangdong, China
Temporal-Emerged Prompting for Segment Anything in Multiframe Infrared Small Target Detection
Jun 26, 2026Yinghui Xing, Donghao Chu, Shizhou Zhang +1Image Segmentation Promptable Image Segmentation
Huawei Technologies Ltd.
Denoising-Enhanced Coarse-to-Fine Infrared Small Target Detection with Attention Prior-Guided Knowledge Distillation
Jun 20, 2026Houzhang Fang, Ruixuan Huang, Qiuhuan Chen +3Attention-Guided Distillation Infrared Small Target Detection
Xidian University, Xi’an, China · Huazhong University of Science and Technology, Wuhan, China

Jun 13, 2026J/K move · Enter open · S save

Decoupled Motion Representation Learning for Moving Infrared Small Target Detection

Guoyi Zhang, Peiwen Wu, Han Wang, Xiangpeng Xu+1

School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen 518107, Guangdong, China

Representation Learning Infrared Small Target Detection Motion Representation Learning

Open paper AskPDF arXiv

Infrared small target detection in dynamic scenes remains challenging due to the highly coupled motions among targets, imaging platforms, and dynamic backgrounds. Existing multi-frame methods usually perform implicit temporal modeling, where coherent background dynamics dominate motion correspondence learning, leading to an inherent trade-off between detection and false alarms. In this work, we observe that background motions exhibit strong global coherence, whereas small targets mainly correspond to sparse local motion anomalies. Moreover, many false-alarm responses maintain high consistency with globally coherent motion patterns, indicating that they mainly originate from coherent background dynamics rather than genuine target motions. Based on these observations, we propose a decoupled motion representation learning framework for moving infrared small target detection. Specifically, an explicit motion branch is introduced to model globally coherent motion dynamics using pretrained optical flow priors, together with a structure-preserving self-supervised adaptation strategy for infrared motion correspondence learning. Meanwhile, an implicit motion branch based on deformable feature alignment is designed to capture target-sensitive local motion anomalies under coherent motion guidance. Furthermore, a coherent-motion-guided local anomaly reasoning module is proposed to identify and suppress coherent-motion-induced false responses during localized motion modeling. Extensive experiments on two challenging infrared small target detection benchmarks demonstrate that the proposed method consistently outperforms existing state-of-the-art approaches, particularly in dynamic scenes with complex motions, while maintaining favorable inference efficiency.

Abstract

Infrared Small Target Detection

Explore similar work

Decoupled Motion Representation Learning for Moving Infrared Small Target Detection

Temporal-Emerged Prompting for Segment Anything in Multiframe Infrared Small Target Detection

Denoising-Enhanced Coarse-to-Fine Infrared Small Target Detection with Attention Prior-Guided Knowledge Distillation