cs.RONov 18, 2026

A Hierarchical Onboard Vision-Language-Navigation Framework for Aerial Robots in Complex Environments

Authors: Yuze Wu, Mo Zhu, Xingxing Li, Yuheng Du, Zhaoqi Wang, Yuxin Fan, Wenjun Li, Kenghou Hoi, 

Organizations: State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou 310027, China. · Differential Robotics, Huzhou 310027, China.

Abstract

Unmanned Aerial Vehicles (UAVs) are expected to achieve semantic reasoning, intelligent decision-making, and long-horizon navigation under strict onboard constraints. While recent foundation-model-driven approaches have shown promising progress toward this goal, existing methods remain limited by challenges in 3D data availability, trajectory feasibility, long-horizon reasoning, and computational efficiency in complex environments. To bridge these gaps, we propose VLN-AR, a hierarchical vision-language-navigation (VLN) framework for aerial robots. VLN-AR decouples multimodal reasoning and low-level control, enabling structured prediction of 3D pose commands and navigation intentions that are translated into dynamically feasible actions. To mitigate data scarcity and domain gaps, we introduce a fully automated 3D Gaussian Splatting (3DGS)-based pipeline to generate scalable photorealistic UAV navigation data. A progressive three-stage training strategy further enhances grounding, scene reasoning, and complex navigation. In addition, a lightweight cognitive agent improves long-horizon reasoning via memory-aware task decomposition. Finally, we design onboard optimization techniques, including token sparsification and model lightweighting, achieving 7.1x-9.4x speedup while maintaining high navigation performance on resource-limited UAV platforms. Experiments show great improvements over existing methods in both simulation and real-world UAV navigation. Project website: https://wyz000.github.io/VLN-AR/

Figures & tables

Explore similar work

CardsList
  1. ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planning

    May 31, 2026Xuchen Liu, Jiawei Huang, Shihao Xia +3UAV NavigationVision-Language Navigation

  2. FSD-VLN: Fast-Slow Dual-System Modeling for Aerial Long-Horizon Vision-Language Navigation

    Jul 9, 2026Xueke Zhu, Qingyan Meng, Liutao Yu +4Vision-Language ModelsUAV Navigation

  3. From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigation

    Aug 10, 2026Zeyuan Ma, Jiaxin Chen, Di HuangUAV NavigationVision-Language Navigation