Organizations: School of Artificial Intelligence, Tianjin University, Tianjin 300072, China · College of Computer Science, VCIP, Nankai University, Tianjin 300350, China · School of Computer Science and Technology, Tianjin University, Tianjin 300072, China · School of Automation, Southeast University, Nanjing 210096, China · the Low-Altitude Intelligence Laboratory, Xiong’an National Innovation Center, Baoding, 071701, China · Xiong’an Guochuang Lantian Technology Company Ltd., Baoding, 071701, China
The reasoning capabilities of vision-language models (VLMs) can be limited by incomplete or inaccurate visual observations in the general domain. In unmanned aerial vehicle (UAV) scenes, small targets, subtle attributes, and changing viewpoints further complicate perception and understanding. Therefore, we introduce Drone-PUR, a dataset and benchmark organized around visual clue discovery, interpretation, and integration, corresponding to the three capability dimensions of perception, understanding, and reasoning. Reasoning samples are designed to require models to combine multiple visual clues to derive visually grounded conclusions. Its 11 tasks provide 175k training and 20k test samples spanning single images, image sequences, cross-modal images, and multiple viewpoints. All test samples undergo human review. Evaluating 22 general-purpose VLMs reveals persistent limitations in spatial judgment and cross-modal and cross-view reasoning. Across models, stronger perception is associated with better reasoning, while single-task fine-tuning further reveals selective bidirectional transfer between these capabilities. To examine domain adaptation, we develop UAV-PUR from Qwen3-VL through staged fine-tuning with auxiliary supervision. Its 4B and 8B variants improve overall multiple-choice accuracy by 18.33 and 17.52 percentage points over their pretrained counterparts, respectively. The dataset and project resources are available at https://drone-pur.github.io.
Figures & tables
Fig. 1: The capability framework and model evaluation on Drone-PUR. (a) The three capability dimensions illustrated through a shared scene. (b) Capability scores across three dimensions; marker sizes schematically indicate total parameter counts. (c) Task-level performance across the ten multiple-choice tasks.
Fig. 2: Task overview of the Drone-PUR dataset across perception, understanding, and reasoning. Representative questions and reference answers are shown.
Fig. 3: Construction pipeline of Drone-PUR, comprising clue preparation, VLM-based generation, cross-model verification and filtering, and human review.
Fig. 4: Statistical overview of the Drone-PUR dataset. (a) Image sources by task. (b–c) Task distributions in the benchmark and training sets, respectively. (d) Proportions of human-authored or revised samples by task and split. (e) The distribution of region areas normalized by image area.
Fig. 5: Overview of UAV-PUR, illustrating the model and training architecture, three-stage fine-tuning with auxiliary supervision, and unified inference through the original language modeling head. Auxiliary heads are used only in the first two training stages.
Method
Perception
Understanding
Reasoning
Overall ↑
OR
OC
MIG
CJ
PSJ
VG
CR
SI
MMR
MVR
Random
25.00
25.00
25.00
50.00
25.00
25.00
20.00
20.00
20.00
20.00
26.43
Text-only
35.07
22.37
25.86
54.62
28.14
27.04
28.90
26.75
23.43
21.86
30.16
Human
98.21
-
99.70
92.28
95.59
98.46
96.80
97.39
92.44
98.92
-
API-Evaluated Models
Claude-sonnet-4.6 [ 58 ]
86.61
29.64
38.48
78.54
31.34
48.86
64.99
70.46
41.17
27.93
50.38
TABLE I: Performance on the Drone-PUR multiple-choice benchmark. We report API-evaluated models, locally evaluated models, and fine-tuned models. The best and second-best scores within each model category are shown in bold and underlined, respectively.
Fig. 6: Confusion matrices of five representative VLMs on four PSJ subtask groups. Results are shown for cardinal vehicle headings, cardinal pedestrian headings, turning directions, and four-quadrant relative positions. Each row is normalized by the number of samples in its ground-truth direction.
Method
MMR-G
MMR-C
MMR-R
MVR-G
MVR-R
Claude-4.6
45.09
26.00
75.61
21.27
47.41
Qwen3.5-plus
64.16
30.68
76.10
36.10
61.08
Gemini-3.5
86.62
50.35
85.85
68.57
72.64
GPT-5.4
63.42
29.74
78.70
27.80
53.30
DeepSeek-V4.1
39.64
31.56
82.76
25.71
60.14
Grok-4.3
45.00
20.96
69.11
30.06
56.37
TABLE II: Performance comparison on the subtasks of MMR and MVR.
Method
VH
PH
TD
RP
HC
AVG.
Claude-4.6
23.40
33.14
35.89
30.26
33.73
31.34
Qwen3.5-plus
33.43
30.79
33.38
36.57
41.42
34.94
Gemini-3.5
36.21
38.42
32.79
36.23
47.34
36.34
GPT-5.4
30.08
29.91
38.40
26.51
41.42
31.30
DeepSeek-V4.1
34.82
39.59
28.36
33.84
30.18
33.10
Grok-4.3
25.91
34.90
28.21
27.54
31.95
28.69
TABLE III: Performance comparison on the five subtasks of PSJ.
Fig. 7: Pearson correlations across tasks and capability dimensions.
Fig. 8: Cross-task transfer following single-task fine-tuning on Drone-PUR.
Method
OD
MIG
VG
MMR-G
MVR-G
Qwen3-VL-2B
15.33
0.40
14.40
3.14
0.32
InternVL3.5-4B
2.61
0.00
7.27
0.83
10.96
Qwen3-VL-4B
29.50
4.04
43.86
23.20
9.43
InternVL3.5-8B
16.61
0.00
12.42
0.17
2.26
Qwen3-VL-8B
11.20
1.41
49.01
26.34
9.75
InternVL3.5-14B
11.42
0.00
14.70
2.15
12.81
TABLE IV: Open-Ended Detection and Grounding Performance.
Method
OC
MMR-C
Avg. Under(%)
MAE ↓
MRA(%) ↑
MAE ↓
MRA(%) ↑
Qwen3-VL-2B
97.43
10.90
15.35
36.23
87.07
Qwen3-VL-4B
76.41
12.08
9.29
48.53
84.76
Qwen3-VL-8B
69.20
18.64
9.02
51.04
80.11
InternVL3.5-8B
78.83
11.64
13.59
36.23
68.98
Qwen3.5-9B
73.58
26.52
8.96
57.70
82.69
TABLE V: Open-ended counting performance and underestimation rates.
Fig. 9: Paired comparison of multiple-choice and open-ended predictions across six models. Stacked bars show the proportions of samples answered correctly in both formats, only one format, or neither.
Fig. 10: Effects of visual clue masking on reasoning accuracy. On the same task subsets, progressive key-clue masking is compared with masking regions matched in area and shape. Zero denotes full input. Horizontal dotted lines indicate each model’s text-only baseline.
Fig. 11: Modality and view ablation study on MMR and MVR. Signed annotations indicate single-image accuracy changes relative to full-input performance.
Size
3 Stages
Configuration
Per.
Und.
Rea.
4B
×
Base
48.66
57.87
46.83
Stage 1 only
70.44
58.94
51.94
Stages 1+2
73.94
65.88
65.99
Qwen- sft
72.18
65.25
66.60
✓
w/o Aux.
72.31
65.58
65.10
Frozen Mergers
73.59
65.73
65.30
TABLE VI: Ablation of training stages, auxiliary supervision, and Merger adaptation. Within each size group, the best and second-best scores are shown in bold and underlined, respectively.
Fig. 12: Qualitative examples of multimodal and multi-view reasoning. Blue and orange borders mark essential clues from different input images; gray borders mark distracting regions. Model icons indicate selected options, with green and red denoting correct and incorrect answers, respectively. Only the GT option and options selected by the displayed models are shown.
Fig. 13: Qualitative comparisons of open-ended detection and grounding on Drone-PUR. OD predictions are displayed separately for each model, with TP, FP, FN, and F1 scores reported below. For VG, MMR-G, MVR-G, and MIG, predictions from four models are overlaid on the target image, with corresponding IoU scores. Green boxes indicate ground truth; model predictions follow the color legend. Enlarged crops highlight target details and grounding differences.
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Shanghai Innovation Institute