The rapid movements and agile maneuvers of unmanned aerial vehicles (UAVs) induce significant observational challenges for multi-object tracking (MOT). However, existing UAV-perspective MOT benchmarks often lack these complexities, featuring predominantly predictable camera dynamics and linear motion patterns. To address this gap, we introduce DynUAV, a new benchmark for dynamic UAV-perspective MOT, characterized by intense ego-motion and the resulting complex apparent trajectories. The benchmark comprises 42 video sequences with over 1.7 million bounding box annotations, covering vehicles, pedestrians, and specialized industrial categories such as excavators, bulldozers and cranes. Compared to existing benchmarks, DynUAV introduces substantial challenges arising from ego-motion, including drastic scale changes and viewpoint changes, as well as motion blur. Comprehensive evaluations of state-of-the-art trackers on DynUAV reveal their limitations, particularly in managing the intertwined challenges of detection and association under such dynamic conditions, thereby establishing DynUAV as a rigorous benchmark. We anticipate that DynUAV will serve as a demanding testbed to spur progress in real-world UAV-perspective MOT, and we will make all resources available at link.
Figures & tables
Figure 1: Visualization of scene diversity and complexity in the DynUAV benchmark. Our benchmark encompasses three primary scenarios: campus, city and road, and nighttime. For each scenario, the upper montage depicts a variety of its constituent sequences, while the lower panel illustrates the temporal progression of a single representative sequence, accompanied with its ground-truth annotations.
Dataset
MOT-17 [ 25 ]
MOT-20 [ 7 ]
VisDrone [ 40 ]
UAVDT [ 9 ]
MDMT [ 19 ]
DanceTrack [ 33 ]
DynUAV
Task
General M
General M
UAV M,S,D
UAV M,S,D
UAV M
General M
UAV M
Seq.
42
8
92
50
88
100
42
BBox
614k
1337k
1621k
799k
2212k
574k
1720k
Frame
Min
450
429
58
265
151
183
1076
Avg
802
1676
417
815
451
1059
1828
Max
1500
3315
1424
2035
703
2402
2655
Table 1: Dataset composition and frame statistics. Overview of sequence length (Seq), bounding boxes (BBox, k= ×103 ), task type (UAV/general; S/M/D), and frame statistic (sum, minimum, average, maximum) compared to existing MOT benchmarks.
Figure 2: Comparison of the mean and variance of IOU across datasets. The horizontal and vertical axes represent the mean and variance of IoU respectively. The bubble radius corresponds to the proportion of non-overlapping bounding boxes in adjacent frames.
Dataset
avg.
MOT-17 [ 25 ]
1.3511×10−2
MOT-20 [ 7 ]
5.8318×10−3
VisDrone [ 40 ]
2.5820×10−3
UAVDT [ 9 ]
2.5922×10−3
MDMT [ 19 ]
9.7198×10−4
DanceTrack [ 33 ]
3.1923×10−2
Table 2: Average bounding box area ratio across datasets.
Figure 3: Bar charts comparing the distributions of trajectory discontinuities across different datasets. Each bar chart displays the proportion of target IDs relative to the number of continuous trajectory segments.
Algorithm
FP↓
FN↓
DetA↑
MOTA↑
HOTA↑
IDF1↑
AssA↑
IDSW↓
IDs↓
BoostTrack [ 32 ]
19097
94087
49.52
56.72
53.71
63.77
58.98
609
1696
BoT-SORT [ 1 ]
17724
68090
57.84
66.88
59.86
68.81
62.65
1274
2264
ByteTrack [ 38 ]
18935
75152
54.90
63.03
53.59
60.78
53.08
3136
4025
OC-SORT [ 5 ]
18085
83611
53.40
60.69
51.73
58.12
50.81
1660
2738
Deep OC-SORT [ 24 ]
19799
67890
57.58
66.44
61.09
72.25
65.49
567
1598
StrongSORT [ 10 ]
24965
57302
59.38
68.18
60.87
71.21
63.15
1394
1749
Table 3: Overall quantitative performance of representative MOT trackers on the DynUAV dataset.’↑’/’↓’ indicates higher/lower values are better, respectively. Bold numbers are superior results. Note: All metrics, except for IDSW, are reported as percentages (e.g., 63.54 means 63.54%).
Algorithm
MOT-17
MOT-20
DanceTrack
MOTA↑
IDF1↑
HOTA↑
MOTA↑
IDF1↑
HOTA↑
MOTA↑
IDF1↑
HOTA↑
BoostTrack [ 32 ]
-23.98
-18.43
-12.89
-20.98
-18.23
-12.69
-
-
-
BoT-SORT [ 1 ]
-13.72
-10.69
-4.74
-10.82
-7.49
-2.74
-
-
-
ByteTrack [ 38 ]
-17.27
-16.52
-9.51
-14.77
-14.42
-7.71
-
-
-
OC-SORT [ 5 ]
-17.31
-19.38
-11.47
-14.82
-17.78
-10.37
-31.51
3.22
-3.37
Deep OC-SORT [ 24 ]
-12.96
-8.35
-3.81
-9.16
-6.95
-2.81
-25.86
10.75
-0.21
Table 4: Performance Degradation: DynUAV vs MOT-17 [ 25 ] , MOT-20 [ 7 ] and DanceTrack [ 33 ] . Performance drop relative to conventional high-frequency test sets. Negative values indicate degradation, with the largest decrease in each metric highlighted in bold.
Figure 4: Fine-grained IDSW analysis. IDSW counts per algorithm across test sequences in DynUAV, revealing performance fluctuations in diverse scenarios.
Algorithm
FP↓
FN↓
DetA↑
MOTA↑
HOTA↑
IDF1↑
AssA↑
IDSW↓
IDs↓
AdapTrack [ 30 ]
o
19902
74779
55.72
63.57
56.99
65.53
58.97
1123
2035
w
21489
62181
58.84
67.96
62.33
73.26
66.71
583
1564
TrackTrack [ 31 ]
o
15030
75740
56.30
65.31
60.64
71.04
65.95
439
1343
w
15617
71034
57.65
66.95
62.74
74.81
68.89
256
1125
U2MOT [ 18 ]
o
19265
92843
49.42
54.81
50.95
58.85
53.34
6731
8919
w
20087
91359
49.97
55.16
51.47
58.92
53.90
6452
8881
Table 5: Comparison of MOT performance with and without CMC. ”w” and ”o” denote with and without CMC, respectively.
Figure 5: Per-Sequence IDSW with CMC. A radar plot showing the number of IDSW for each color-coded algorithm across all test sequences.
State key Laboratory of Extreme Environment Optoelectronic Dynamic Testing Technology and Instrument, Nanjing University of Science and Technology, Nanjing 210094, China · Jiangsu Key Laboratory of Visual Sensing and Intelligent Perception, Nanjing University of Science and Technology, Nanjing 210094, China