Unmanned Aerial Vehicles (UAVs) are expected to achieve semantic reasoning, intelligent decision-making, and long-horizon navigation under strict onboard constraints. While recent foundation-model-driven approaches have shown promising progress toward this goal, existing methods remain limited by challenges in 3D data availability, trajectory feasibility, long-horizon reasoning, and computational efficiency in complex environments. To bridge these gaps, we propose VLN-AR, a hierarchical vision-language-navigation (VLN) framework for aerial robots. VLN-AR decouples multimodal reasoning and low-level control, enabling structured prediction of 3D pose commands and navigation intentions that are translated into dynamically feasible actions. To mitigate data scarcity and domain gaps, we introduce a fully automated 3D Gaussian Splatting (3DGS)-based pipeline to generate scalable photorealistic UAV navigation data. A progressive three-stage training strategy further enhances grounding, scene reasoning, and complex navigation. In addition, a lightweight cognitive agent improves long-horizon reasoning via memory-aware task decomposition. Finally, we design onboard optimization techniques, including token sparsification and model lightweighting, achieving 7.1x-9.4x speedup while maintaining high navigation performance on resource-limited UAV platforms. Experiments show great improvements over existing methods in both simulation and real-world UAV navigation. Project website: https://wyz000.github.io/VLN-AR/
Figures & tables
Fig. 2 : Different UAV navigation foundation models.
Fig. 3 : The model architecture of the proposed VLN-AR model.
Fig. 4 : The improved ViT architecture and token sparsification .
Fig. 5 : Automated UAV navigation data generation and collection process.
Fig. 6 : Cognitive-driven UAV navigation agent pipeline for VLN-AR.
Training Stage
Hyperparameter
Value
Stage I (Grounding- Reasoning- enhanced SFT)
Batch size
32
Learning rate
5×10−5
Epochs
1
Weight decay
0.01
λvqa
1.0
λbbox
1.0
TABLE I: Hyperparameters for Multi-stage Training.
Types
Methods
AerialVLN Dataset [ 32 ]
OpenFly Dataset [ 31 ]
Our Simulation Dataset
SR(%) ↑
NE(m) ↓
TT(s) ↓
SR ↑
NE ↓
TT ↓
SR ↑
NE ↓
TT ↓
End-to-end VLN Methods
OpenUAV [ 33 ]
23.02
60.11
55.76
20.36
76.02
61.43
16.41
14.88
25.51
AerialVLN [ 32 ]
33.17
59.48
46.91
35.73
66.44
52.26
36.21
16.93
21.58
OpenFly [ 31 ]
34.03
64.21
47.02
30.94
77.54
48.11
26.73
14.14
20.02
UAV-Flow [ 34 ]
39.66
54.42
51.37
38.38
72.63
52.94
34.18
12.77
25.11
MLLM-based Navigation Agent
CityNavAgent [ 35 ]
31.84
63.02
113.57
21.75
74.93
128.20
34.96
14.60
26.58
TABLE II: Benchmark comparison of different methods on UAV navigation datasets
Fig. 7 : Task distribution of test data.
Fig. 8 : Simulation tests of VLN-AR across different navigation datasets using both visual and language inputs.
Fig. 9 : Hardware and software architectures of two different experimental drones.
Fig. 10 : Real-world experiments for navigation scenarios utilizing both visual and language inputs.
Model
Floating Point Operations
Baseline Decode Speed (s/token)
Our Decode Speed (s/token)
7B-AWQ
∼1815.8 GFLOPs
1.034
0.110
4B-AWQ
∼946.7 GFLOPs
0.508
0.062
2B-AWQ
∼563.3 GFLOPs
0.227
0.032
TABLE III : Onboard inference performance on different models
Fig. 11 : Robustness tests on 3D floating scenes, large height variations, dim lighting, dynamic disturbance, and zero-shot deployment.
Test scenes
Results
3D floating scene
18 / 20
Large height variation
19 / 20
Dim lighting environment
17 / 20
Dynamic obstacle interference
17 / 20
Zero-shot deployment
18 / 20
TABLE IV: Robustness test of different navigation scenes
Methods
Grounding-reasoning -enhanced SFT
Navigation -specific SFT
RFT for Reasoning -aware Navigation
Scene Understanding
Spatial Reasoning
Precise Navigation
Long-horizon Navigation
Baseline
×
×
×
77.4
68.2
21.3
17.8
Stage I
✓
×
×
87.2
94.5
28.1
22.5
Stage II
×
✓
×
75.6
64.3
68.5
64.4
Stage I + Stage II
✓
✓
×
80.2
88.3
60.6
61.5
Stage III
×
×
✓
82.6
76.1
45.4
36.3
Ours
✓
✓
✓
85.5
94.6
85.7
88.0
TABLE V: Ablation study of the multi-stage training strategy (Success rates %)
Fig. 12 : Ablation study of different training navigation datasets.
Fig. 13 : Impact of different token retention ratios for ViT.
Datasets
Tasks
Object Navigation
Spatial Reasoning
Long-horizon Navigation
Ours
98.1
94.6
88.0
w/o Agent
95.7
92.4
57.4
TABLE VI: Ablation study of the aerial navigation agent
Fig. 14 : Ablation study of the inference optimization and the few-shot transfer.
Pengcheng Laboratory, Shenzhen, Guangdong, China · School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou, Guangdong, China · Southern University of Science and Technology, Shenzhen, Guangdong, China
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China · School of Computer Science and Engineering, Beihang University, Beijing, China