Organizations: Guangdong Provincial Key Laboratory of Space-Aerial Networking and Intelligent Sensing, Harbin Institute of Technology, Shenzhen, China · Information Systems Technology and Design, Singapore University of Technology and Design, Singapore 487372 · School of Computer Science and Engineering, Central South University, Changsha, China · Peng Cheng Laboratory (PCL), Shenzhen, China
Natural language (NL) navigation for low-altitude unmanned aerial vehicles (UAVs) offers an intelligent and convenient solution for low-altitude aerial services by enabling an intuitive interface for non-expert operators. However, deploying this capability in urban environments necessitates the precise grounding of underspecified instructions into safety-critical, dynamically feasible motion plans subject to spatiotemporal constraints. To address this challenge, we propose a unified framework that translates NL instructions into Signal Temporal Logic (STL) specifications and subsequently synthesizes trajectories via mixed-integer linear programming (MILP). Specifically, to generate executable STL formulas from free-form NL, we develop a reasoning-enhanced large language model (LLM) leveraging chain-of-thought (CoT) supervision and group-relative policy optimization (GRPO), which ensures high syntactic validity and semantic consistency. Furthermore, to resolve infeasibilities induced by stringent logical or spatial requirements, we introduce a specification repair mechanism. This module combines MILP-based diagnosis with LLM-guided semantic reasoning to selectively relax task constraints while strictly enforcing safety guarantees. Extensive simulations and real-world flight experiments demonstrate that the proposed closed-loop framework significantly improves NL-to-STL translation robustness, enabling safe, interpretable, and adaptable UAV navigation in complex scenarios.
Figures & tables
Fig. 1: Overview of the proposed language-guided planning framework. A natural-language instruction is translated into an STL specification, which is enforced by a constrained planner. When infeasibility is detected, solver feedback is used to guide specification refinement and re-planning in a closed loop.
Fig. 2: GRPO-based RL framework for reasoning-enhanced NL-to-STL generation.
Fig. 3: CoT Data Engine for augmenting NL2TL with intermediate reasoning traces.
Fig. 4: NL-to-STL translation accuracy during SFT and GRPO training.
Fig. 5: Simulated trajectory for Task 1. The UAV reaches the goal region within the specified time window while avoiding obstacles.
Fig. 6: Simulated trajectory for Task 2. The UAV sequentially reaches the specified target regions while avoiding the no-fly zone at all times.
Fig. 7: Simulated trajectory for the repaired task. The UAV reaches the goal region after temporal repair while continuously avoiding the obstacle.
Fig. 8: Executed UAV trajectory for the real-world search-and-rescue experiment.
Fig. 9: Real-world experimental setup for the search-and-rescue task. (a) DJI Matrice 300 RTK equipped with an onboard 4G/5G wireless signal acquisition and localization device. (b) Annotated test site map showing designated rescue areas and no-fly zones.
We present a real-time semantic navigation framework for Unmanned Aerial Vehicles (UAVs) focused on improving time efficiency in the Object Goal Navigation (ObjectNav) task. Central to our approach is a Large Language Model (LLM) that interprets user-provided natural language instructions and performs semantic reasoning over detected objects and spatial context to prioritize high-probability search regions. The system combines real-time object detection, 3D spatial mapping, and polynomial spline interpolation for smooth and feasible UAV trajectory planning. Unlike prior methods that rely on offline reasoning or simulator-constrained action spaces, our framework can operate in real time, continuously updating semantic relevance based on new observations. Experiments in both simulated and real-world settings demonstrate reductions in mission duration while maintaining high search accuracy, underscoring the effectiveness of LLM-guided reasoning for time- efficient UAV-based ObjectNav.
Marin Maletic, Marijana Peti, Tamara Petrovic +1
University of Zagreb Faculty of Electrical Engineering and Computing, LARICS (Laboratory for Robotics and Intelligent Control Systems), Unska 3, 10000 Zagreb, Croatia
We propose an architecture for integrating high-level, human-provided safety rules and operator-aligned semantic preferences into autonomous robot navigation in unstructured outdoor environments. In our approach, natural-language rules are translated into Signal Temporal Logic (STL) specifications that guide planning and navigation during runtime. Persistent, environment-centric rules and terrain preferences are grounded into a 2D cost map, while temporally dynamic requirements are expressed as STL specifications to be monitored during runtime. We hypothesize the use of Vision-Language Models (VLMs) for zero-shot scene understanding, enabling mapping between human instructions, semantic features, and environmental constraints. Within this framework, we construct an illustrative navigation model that is designed to satisfy a set of STL-encoded specifications and soft operator preferences through formal satisfaction metrics embedded into environmental properties and runtime monitoring.
Kristy Sakano, Kalonji Harrington, Mumu Xu
Department of Computer Science, University of Maryland, College Park, USA. · Department of Aerospace Engineering, University of Maryland, College Park, USA.
Language-guided UAV agents must execute long-horizon semantic instructions while producing smooth, physically feasible continuous flight commands, yet existing Vision-Language Navigation (VLN) benchmarks typically use discrete or coarse actions and existing UAV Vision-Language-Action (VLA) tasks focus on short, atomic maneuvers. To address this gap in UAV task settings, we introduce FLIGHT, a Fine-grained Long-horizon Instruction-Guided benchmark for Hybrid UAV navigation and reasoning Tasks, which combines multi-stage instructions with dense 6-DoF trajectory annotations across two dataset splits: Fine-grained VLN and Long-horizon Flow. To endow the UAV agent with the capability of real-time in-flight reasoning over task execution status and mission planning, while simultaneously accommodating high-frequency, real-time precise control, we further propose FLIGHT VLA, an asynchronous architecture that decouples a low-frequency Streaming Pilot Vision-Language Model (VLM) for task-state reasoning from a high-frequency diffusion action model for continuous control, supervised by explicit Pilot Reasoning texts that summarize the current flight state and anticipate the next subgoal. In closed-loop evaluation, FLIGHT VLA consistently surpasses representative VLN and VLA baselines on our FLIGHT benchmarks, achieving stronger multi-stage completion, subgoal adherence, and terminal control. Its trained Streaming Pilot Reasoning VLM further improves UAV video reasoning, validating the effectiveness of our design.
Xiangyi Zheng, Xiangyu Wang, Qinan Liao +6
Colab, Beihang University · National University of Singapore · Meituan