cs.RONov 17, 2026

Fly as Instructed: Refining World-Action Policies for UAV Vision-Language Navigation

Authors: Zhexuan Zhou, Yanke Sun, Yuhan Zong, Youmin Gong, Jie Mei

Organizations: Harbin Institute of Technology · National University of Singapore · Harbin Institute of Technology, Shenzhen

Abstract

Natural language-guided UAV navigation enables intuitive human-robot interaction and flexible autonomous flight in complex environments. However, reliable navigation requires accurate instruction following, continuous task progress tracking, and safe obstacle avoidance, which remain challenging for existing world-action models that primarily focus on action prediction. To address these limitations, we propose a world-action learning framework that integrates three complementary mechanisms: a language-conditioned action interface with targeted post-training for instruction-responsive motion, recurrent instruction-consumption prediction to track task progress and determine termination, and Gaussian-scene post-training to enhance obstacle avoidance through trajectory imitation and differentiable geometric feedback. Closed-loop evaluations and ablation studies demonstrate improvements in sequential task completion, safe goal reaching, and timely termination. Furthermore, direct one-step denoising reduces inference overhead, and real-world indoor and outdoor flights validate the framework's effectiveness.

Figures & tables

Explore similar work

CardsList
  1. WorldFly: A World-Model-Based Vision-Language-Action Model for UAV Navigation

    Jun 4, 2026Shengtao Zheng, Kai Li, Weichen Zhang +5World Models for RoboticsRobot Navigation

  2. ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planning

    May 31, 2026Xuchen Liu, Jiawei Huang, Shihao Xia +3UAV NavigationVision-Language Navigation

  3. FreeFly-Thinking : Aligning Chain-of-Thought Reasoning with Continuous UAV Navigation

    Mar 7, 2026Jiaxu Zhou, Shaobo Wang, Zhiyuan Yang +2Aerial VLNUAV Navigation