Organizations: China-UAE Belt and Road Joint Laboratory on Intelligent Unmanned Systems of School of Aerospace Engineering, Beijing Institute of Technology, Beijing, 100081, China · School of Information and Computing Science, Xi’an Jiaotong-Liverpool University, Suzhou, 215123, China
Air-to-air tracking of swarm UAVs presents significant challenges due to the complex nonlinear group motion and weak visual cues for small objects, which often cause detection failures, trajectory fragmentation, and identity switches. Although existing methods have attempted to improve performance by incorporating trajectory prediction, they model each object independently, neglecting the swarm-level motion dependencies. Their limited integration between motion prediction and appearance representation also weakens the spatio-temporal consistency required for tracking in visually ambiguous and cluttered environments, making it difficult to maintain coherent trajectories and reliable associations. To address these challenges, we propose SCT-MOT, a tracking framework that integrates Swarm-Coupled motion modeling and Trajectory-guided feature fusion. First, we develop a Swarm Motion-Aware Trajectory Prediction (SMTP) module jointly models historical trajectories and posture-aware appearance features from a swarm-level perspective, enabling more accurate forecasting of the nonlinear, coupled group trajectories. Second, we design a Trajectory-Guided Spatio-Temporal Feature Fusion (TG-STFF) module aligns predicted positions with historical visual cues and deeply integrates them with current frame features, enhancing temporal consistency and spatial discriminability for weak objects. Extensive experiments on three public air-to-air swarm UAV tracking datasets, including AIRMOT, MOT-FLY, and UAVSwarm, demonstrate that SMTP achieves more accurate trajectory forecasts and yields a 1.21% IDF1 improvement over the state-of-the-art trajectory prediction module EqMotion when integrated into the same MOT framework. Overall, our SCT-MOT consistently achieves superior accuracy and robustness compared to state-of-the-art trackers across multiple metrics under complex swarm scenarios.
Figures & tables
Fig. 1: Key challenges in air-to-air swarm UAV tracking. (a) Swarm UAVs exhibit highly coupled and nonlinear group behavior. The left image is the raw input frame; the right shows the corresponding annotated ground-truth trajectories. (b) Due to extremely micro object sizes and weak visual features, tracking performance is easily affected by background noise and similar distractors, leading to issues such as misassociation, false tracking, and missed detections. The left image shows ground-truth annotations, while the right presents the tracking results generated by HOMATracker. Orange boxes denote correct tracklets, red boxes indicate false positive tracklets and yellow boxes indicate missed tracklets.
Fig. 5: Trajectory prediction comparison between different modules in air-to-air swarm tracking. Red dots and arrows denote the ground-truth positions and motion directions; yellow dots and arrows indicate the predicted positions and directions by the proposed SMTP module; green ones represent the predictions by EqMotion. Orange boxes highlight areas where SMTP predictions are more closely aligned with the ground-truth trajectories. The subfigures illustrate representative sequences from the UAVSwarm and AIRMOT datasets: (a) UAVSwarm-40, (b) UAVSwarm-58, and (c) AIRMOT-01.
T
MOTA ↑
IDF1 ↑
HOTA ↑
IDSW ↓
AIRMOT
4
33.63
30.04
25.51
457
8
32.30
30.15
25.54
476
12
31.89
29.61
25.47
498
16
31.12
28.85
25.38
513
UAVSwarm
TABLE I: Evaluation of the SCT-MOT with SMTP module on AIRMOT and UAVSwarm datasets. ↑ indicates higher is better, ↓ indicates lower is better. Best results are highlighted in bold .
algorithms
MOTA ↑
IDF1 ↑
HOTA ↑
IDSW ↓
AIRMOT
SCT-MOT-SMTP
32.30
30.15
25.54
476
SCT-MOT-EqMotion
31.91
28.94
24.92
478
UAVSwarm
SCT-MOT-SMTP
81.90
88.45
68.56
56
SCT-MOT-EqMotion
80.95
87.08
67.42
73
TABLE II: Comparison of different trajectory prediction modules for Multi-Object Tracking on AIRMOT and UAVSwarm test sets.
Fig. 6: Visualization of feature fusion in the TG-STFF module. From left to right: ground-truth frame, raw feature map, fused feature map using the IMA strategy, and fused feature map using TG-STFF module. Orange boxes: correct detections; red: false positives; yellow: missed detections.
Fig. 7: Performance comparison of SCT-MOT with existing MOT methods on AIRMOT and UAVSwarm datasets, different objects are bounding boxes with different colors. Red boxes/arrows denote false positives, while yellow boxes/arrows indicate missed detections.
Algorithms
IMA
TG-STFF
MOTA ↑
IDF1 ↑
HOTA ↑
IDSW ↓
FairMOT
18.19
17.41
17.56
476
FairMOT
✓
21.19
19.78
19.38
516
FairMOT
✓
21.86
21.58
20.41
415
HOMATracker
30.08
29.31
25.55
506
HOMATracker
✓
31.13
28.42
24.83
483
HOMATracker
✓
32.30
30.15
25.54
476
TABLE III: Performance comparison of FairMOT and HOMATracker using different feature fusion strategies on the AIRMOT test set.
Method
IMA
TG-STFF
MOTA ↑
IDF1 ↑
HOTA ↑
IDSW ↓
FairMOT
63.52
54.81
42.10
177
FairMOT
✓
62.13
61.90
47.68
184
FairMOT
✓
62.96
63.52
48.32
172
HOMATracker
72.42
77.64
58.38
20
HOMATracker
✓
72.46
77.90
58.53
18
HOMATracker
✓
72.52
79.31
59.02
18
TABLE IV: Performance comparison of FairMOT and HOMATracker using different feature fusion strategies on the MOT-FLY test set.
Method
IMA
TG-STFF
MOTA ↑
IDF1 ↑
HOTA ↑
IDSW ↓
FairMOT
67.70
73.20
59.20
590
FairMOT
✓
69.66
76.40
59.64
381
FairMOT
✓
71.29
77.50
60.55
372
HOMATracker
79.20
87.10
67.00
58
HOMATracker
✓
79.43
87.76
67.31
49
HOMATracker
✓
81.90
88.45
68.56
56
TABLE V: Performance comparison of FairMOT and HOMATracker using different feature fusion strategies on the UAVSwarm test set.
Method
Publication
MOTA ↑
IDF1 ↑
HOTA ↑
IDSW ↓
FPS ↑
AIRMOT
FairMOT [ 44 ]
IJCV 2021
18.19
17.41
17.56
476
35.2
DeepSORT [ 9 ]
ICIP 2017
19.60
20.45
19.42
2841
33.6
OCSORT [ 23 ]
CVPR 2023
20.88
23.27
22.07
522
40.9
ByteTrack [ 22 ]
ECCV 2022
24.35
25.72
23.56
1707
39.2
MOTRv3 [ 46 ]
ArXiv 2023
23.52
24.89
22.59
1805
20.6
TABLE VI: Performance comparison between the proposed SCT-MOT and other state-of-the-art (SOTA) MOT methods on the AIRMOT, UAVSwarm, and MOT-FLY test sets.
Method
Dataset
MOTA ↑
IDF1 ↑
HOTA ↑
FPS ↑
SCT-MOT
AIRMOT
30.68
28.72
24.31
24.0
MOT-FLY
72.11
75.34
56.12
22.1
UAVSwarm
78.22
84.04
66.71
24.5
ByteTrack-Light
AIRMOT
23.46
24.58
22.86
24.6
MOT-FLY
67.12
69.85
54.26
25.3
UAVSwarm
63.71
75.13
59.60
23.9
TABLE VII: Comparison of tracking accuracy and runtime of lightweight SCT-MOT and baseline tracker on the NVIDIA Jetson Orin NX across AIRMOT, UAVSwarm, and MOT-FLY test sets.
Airborne optical tracking of uncrewed aerial vehicle (UAV) swarms is challenging due to extremely small target scales, rapid viewpoint changes, and cluttered backgrounds, which can weaken target feature responses and lead to intermittent or temporarily missing detector responses. Existing multi-object tracking methods generally depend on reliable target-specific detector responses to maintain target states and identities across frames. When such responses become unreliable, target states cannot be reliably updated and cross-frame association cues become ambiguous, resulting in fragmented trajectories and identity switches. To address this problem, we propose EECTracker, a swarm-motion-prior-guided joint detection-and-tracking framework for airborne optical UAV swarm tracking. EECTracker constructs a probabilistic swarm motion prior from reliable historical tracklets to capture the shared short-term image-plane motion tendency of the swarm and its uncertainty, providing spatial guidance for cross-frame feature compensation. Building on this prior, we introduce Energy--Entropy Consistency Activation (EEC Activation) to evaluate motion-prior-conditioned feature consistency using feature residual energy and local residual entropy. The resulting Local EEC score guides pixel-level feature compensation by enhancing motion-prior-consistent feature responses in potential target regions while suppressing inconsistent background responses. Experiments on AIRMOT and UAVSwarm show that EECTracker achieves superior overall tracking performance compared with state-of-the-art methods. Compared with the strongest competing method SCT-MOT, EECTracker improves MOTA/IDF1 by 3.89/1.79 percentage points on AIRMOT and by 2.81/1.74 percentage points on UAVSwarm, while maintaining online inference speed.
Zhaochen Chu, Tao Song, Ren Jin +2
China-UAE Belt and Road Joint Laboratory on Intelligent Unmanned Systems, School of Aerospace Engineering, Beijing Institute of Technology, Beijing, 100081, China
Accurate 4D trajectory prediction and closed-loop tracking are essential for Unmanned Aerial Vehicle (UAV) swarms to achieve safe and efficient operations in complex low-altitude environments such as urban airspaces, industrial sites, and indoor facilities. However, this task remains challenging due to intrinsic nonlinearity of UAV swarm dynamics and strict real-time constraints of swarm formation control. To address these challenges, we propose a unified framework that couples coarse-to-fine trajectory forecasting with uncertainty-aware Distributed Nonlinear Model Predictive Control (DNMPC). Our approach features two key innovations: 1) a dimension-decoupled trajectory prediction module that reduces computational complexity by forecasting axis-wise motion, and 2) a diffusion-based residual dynamics refinement module that captures temporally correlated dynamic uncertainties. These refined predictions are then integrated into a DNMPC loop to ensure formation stability. We also introduce a synchronized multi-scenario 4D UAV swarm dataset spanning six representative airspace scenarios. The dataset contains over 7,900 frames of synchronized three-UAV trajectories with frame-level annotations of speed intention and target sector. Extensive experiments demonstrate that our approach outperforms state-of-the-art baselines, reducing trajectory tracking error by up to 10-15% and achieving sub-0.07,m average tracking error in complex urban and industrial environments, while maintaining real-time inference speeds of 34 FPS (sub-30 ms latency) suitable for agile flight.
Tianshun Li, Hongliang Lu, Haoang Li +1
The Hong Kong University of Science and Technology (Guangzhou), China · Southern University of Science and Technology, China and MoSense Technologies, China
Infrared unmanned aerial vehicle (UAV) tracking is challenging because the target is often small, low-contrast, and easily confused with thermal distractors or cluttered backgrounds. Recent Transformer-based trackers have achieved promising performance by learning strong appearance representations, but their responses can still be dominated by background structures when the target appearance is weak or ambiguous. A natural solution is to introduce temporal motion cues. However, in infrared UAV tracking, motion cues are not always reliable: camera jitter, dynamic backgrounds, sensor noise, and target disappearance may produce temporal variations that are stronger than the true target motion. Therefore, the key challenge is not simply how to use motion, but how to distinguish target-consistent motion from background-induced pseudo motion. To this end, we propose CMRTrack, a counterfactual motion reliability learning framework for robust infrared UAV tracking. CMRTrack first extracts temporal evidence from adjacent search regions using a lightweight motion evidence encoder. During training, a counterfactual target-erased history branch is introduced to construct hard motion references, encouraging the motion encoder to learn reliable target-consistent motion rather than arbitrary temporal changes. The learned motion evidence is then incorporated into a one-stream tracking framework through motion-guided token modulation and reliability-aware score fusion, enabling adaptive feature enhancement and response refinement. Extensive experiments on Anti-UAV410 demonstrate that CMRTrack consistently outperforms representative state-of-the-art trackers and significantly improves the OSTrack baseline, with ablation studies and qualitative analysis verifying the effectiveness of the proposed counterfactual motion reliability learning.
Yuehai Chen
Faculty of Electronics and Information Engineering and the Institute for Low-Altitude Regulation, Xi’an Jiaotong University, Xi’an 710049, China