FreeFly-Thinking : Aligning Chain-of-Thought Reasoning with Continuous UAV Navigation
Organizations: SJTU Paris Elite Institute of Technology, Shanghai Jiao Tong University, China · School of Physics and Astronomy, Shanghai Jiao Tong University, China · Tongji University, China
Abstract
Vision-Language Navigation aims to enable agents to understand natural language instructions and carry out appropriate navigation actions in real-world environments. Most work focuses on indoor settings, with little research in complex outdoor scenes. Current UAV Vision-and-Language Navigation models typically act as black boxes without explicit reasoning. We introduce FreeFly-thinking, an end-to-end VLN framework that converts the UAV agent's egocentric images and language instructions into a series of actions, inspired by environment of urban architecture proposed by OpenFly. We first construct a UAV dataset for navigation task, and then performing natural language chain of thought. We adopt a two-stage training strategy: Supervised fine-tuning and Reinforcement fine-tuning. Experiments on unseen test demonstrate a strong performance, presenting robustness and efficiency in UAV navigation issue.
Figures & tables
| Method | NE | SR | ADE |
|---|---|---|---|
| AerialVLN | 45.9 | 4.3% | 3.7 |
| OpenFly | 32.7 | 11.3% | 2.6 |
| Ours | 28.0 | 13.1% | 2.3 |
| Method | LM-head | Waypoint-head | |||||
|---|---|---|---|---|---|---|---|
| NE | SR | ADE | Act. Acc | NE | SR | ADE | |
| RFT | 16.5 | 30.4% | 1.4 | 84.5% | 35.1 | 9.6% | 3.2 |
| SFT w/o thinking | 19.1 | 27.2% | 1.5 | 83.0% | 31.8 | 11.0% | 2.5 |
| SFT dual head | 18.7 | 27.6% | 1.5 | 83.1% | 28.0 | 13.1% | 2.3 |
Aerial VLN
Aerial vision-and-language navigation (VLN) enables UAVs to follow natural-language goals using visual perception and flight control.
See all 61 papers