KinoUAV: A Benchmark for Safe and Stable UAV Vision-Language Navigation with Kinodynamic Modeling

Chenyao Yang; Peijin Wang; Yingchao Feng; Wenhui Diao

cs.CVOct 22, 2026

KinoUAV: A Benchmark for Safe and Stable UAV Vision-Language Navigation with Kinodynamic Modeling

Authors: Chenyao Yang, Peijin Wang, Yingchao Feng, Wenhui Diao

Organizations: the Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China · the University of Chinese Academy of Sciences, Beijing 100190, China · the School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100190, China · the Key Laboratory of Target Cognition and Application Technology, Beijing 100190, China

Abstract

Unmanned aerial vehicles (UAVs) have recently emerged as important remote sensing platforms, and UAV navigation has shown great potential in remote sensing applications. However, existing methods do not explicitly incorporate kinodynamic modeling or safety distances into trajectory generation, resulting in high absolute yaw rates, frequent collisions, and low navigation success rates, which limit safety and motion stability. To address this challenge, we introduce KinoUAV, a kinodynamic benchmark comprising a dataset and KinoUAV-Agent. First, we develop a hierarchical kinodynamic trajectory generation pipeline that drives UAV through acceleration actions. It uses waypoint chains from global planning and generates trajectories segment by segment under velocity, acceleration, and jerk constraints, reducing oscillations and improving trajectory stability. Furthermore, we introduce directional free-space distance (DFD) labels into the constructed dataset to describe the available clearance for safe motion decisions. The proposed KinoUAV-Agent predicts DFDs and uses them to generate safe actions, decoupling perception from action decision-making. This design enables autonomous obstacle avoidance and improves navigation safety. The benchmark incorporates the proposed Cruise-Seek task, in which the UAV first enters the target region and then searches for the target, reducing the difficulty of target search and improving navigation success. Experiments on KinoUAV show that KinoUAV-Agent outperforms existing methods, achieving the highest success rate while maintaining lower absolute yaw rates and collision rates. These results demonstrate the effectiveness of kinodynamic modeling for safe and stable UAV navigation.

Figures & tables

Fig. 1: Motivation of KinoUAV. Kinodynamic modeling is introduced into the dataset construction pipeline. Acceleration-driven control is used to generate stable UAV trajectories. Motion states, DFDs, front and bottom-view images are collected at each frame. The blue region highlights the data uniquely provided by KinoUAV.

Fig. 2: Overview of our dataset generation pipeline. Different waypoint chains are processed by hierarchical kinodynamic A ∗ to generate reference trajectories under motion constraints. After trajectory cleanup, the reference trajectories are tracked through AirSim’s flight-control API to collect observations and motion states for the Main Subset and Auxiliary Subset.


Dataset	Kinodynamic modeling	$N_{\mathrm{traj}}$	$\overline{\mathrm{len(traj)}}$	$\overline{\mathrm{len(instr)}}$	Action space	$\overline{N}_{\mathrm{act}}$	Environment	$N_{\mathrm{vocab}}$
R2R	No	7189	10	29	Graph	5	Matterport3D	3.1K
RxR	No	13992	14.9	129	Graph	8	Matterport3D	7.0K
TouchDown	No	9326	313.9	90	Graph	35	Google Street View	5.0K
VLN-CE	No	4475	11.1	19	2 DoF	56	Matterport3D	4.3K
LANI	No	6000	16	–	2 DoF	116	CHALET	2.3K

TABLE I: Comparison of VLN dataset properties.

N_{\mathrm{traj}}

is the number of trajectories;

\overline{\mathrm{len(traj)}}

and

\overline{\mathrm{len(instr)}}

are the average trajectory and instruction lengths;

\overline{N}_{\mathrm{act}}

is the average number of actions per trajectory; and

N_{\mathrm{vocab}}

is the vocabulary size. KinoUAV statistics refer to the language-annotated full-navigation data, excluding Auxiliary Subset.

Fig. 3: Dataset statistics. (a) Scenario categories used to annotate target descriptions; blue denotes urban scenarios and orange denotes non-urban scenarios. (b) Word cloud of the annotated descriptions. (c) and (d) Trajectory length distributions of Main Subset and Auxiliary Subset, respectively.

Fig. 4: Acceleration action statistics in our dataset. (a) Overall action distribution. (b) Action probabilities versus DFD. Actions are grouped by direction regardless of magnitude, with the infrequent stay action excluded.

Fig. 5: Overview of KinoUAV-Agent. It hierarchically integrates waypoint planning, DFD estimation, and acceleration-based motion control.

Fig. 6: Illustration of the Cruise–Seek navigation process and stage-specific instructions. Front- and bottom-view observations are shown at six successive navigation moments. During Cruise (1–4), the UAV approaches the target region using direction and distance guidance. During Seek (5–6), it visually localizes and approaches the target using its textual description. Red boxes highlight the target, and representative instructions for both stages are shown below.


Method	SR (%) $\uparrow$	OSR (%) $\uparrow$	NE (m) $\downarrow$	SPL (%) $\uparrow$	CR (%) $\downarrow$	$\overline{v}$ (m/s) $\uparrow$	$\overline{\|\omega_{z}^{k}\|}$ (rad/s) $\downarrow$
Existing Method
Random	0.0	0.2	138.0	0.0	42.5	0.588	0.215
CMA [ 3 ]	11.5	13.6	86.4	9.6	34.5	0.811	0.096
TravelUAV [ 12 ]	20.1	53.9	55.9	16.4	16.9	0.917	0.100
AerialVLA [ 52 ]	52.3	55.0	32.0	36.9	28.3	0.648	0.113

TABLE II: Navigation performance on KinoUAV Test Seen. SR, OSR, NE, SPL, and CR denote success rate, oracle success rate, navigation error, success weighted by path length, and collision rate, respectively. Mean speed and mean absolute yaw rate are reported in

\mathrm{m/s}

and

\mathrm{rad/s}

, respectively.


Method	SR (%) $\uparrow$	OSR (%) $\uparrow$	NE (m) $\downarrow$	SPL (%) $\uparrow$	CR (%) $\downarrow$	$\overline{v}$ (m/s) $\uparrow$	$\overline{\|\omega_{z}^{k}\|}$ (rad/s) $\downarrow$
Test Unseen Object
Random	0.0	0.0	152.1	0.0	32.8	0.634	0.189
CMA [ 3 ]	9.6	13.6	84.1	7.9	30.1	0.943	0.115
TravelUAV [ 12 ]	21.4	52.5	56.8	16.9	17.2	0.902	0.098
AerialVLA [ 52 ]	51.5	56.3	31.2	37.6	27.9	0.661	0.116

TABLE III: Navigation performance and selected kinematic statistics on KinoUAV Test Unseen Object and Test Unseen Map.


Subsets	Fraction	SR (%) $\uparrow$	OSR (%) $\uparrow$	NE (m) $\downarrow$	SPL (%) $\uparrow$	CR (%) $\downarrow$	$\overline{v}$ (m/s) $\uparrow$	$\overline{\|\omega_{z}^{k}\|}$ (rad/s) $\downarrow$
Only Main Subset	–	71.2	77.1	25.9	55.2	15.4	1.865	0.045
All	1%	0.9	10.5	170.4	0.2	68.5	1.583	0.023
All	10%	74.2	77.6	18.4	60.1	17.0	1.664	0.032
All	100%	89.3	90.8	8.8	71.9	6.0	1.680	0.045

TABLE IV: Ablation study on training subsets and data scale on KinoUAV Test Seen. The training fraction is applied to each selected subset independently. All variants use the same supervised DFD branch and navigation pipeline.


Configuration	SR (%) $\uparrow$	OSR (%) $\uparrow$	NE (m) $\downarrow$	SPL (%) $\uparrow$	CR (%) $\downarrow$	$\overline{v}$ (m/s) $\uparrow$	$\overline{\|\omega_{z}^{k}\|}$ (rad/s) $\downarrow$
Single stage: Without distance	79.5	80.4	12.0	64.9	17.9	1.698	0.037
Cruise–Seek: Without distance	76.8	83.9	16.7	59.8	12.5	1.674	0.043
Random initial heading	82.1	83.0	13.6	58.1	11.6	1.528	0.059
Without acceleration projection	87.9	88.6	9.7	70.5	9.8	1.741	0.083
Full configuration (KinoUAV-Agent)	89.3	90.8	8.8	71.9	6.0	1.680	0.045

TABLE V: Ablation of navigation and control configurations on KinoUAV Test Seen. The full configuration uses Cruise–Seek guidance, initializes the UAV heading toward the next ground-truth waypoint, and projects predicted accelerations onto

\mathcal{A}_{\mathrm{UAV}}

. Each variant modifies the indicated setting while retaining the remaining settings of the full configuration. Mean speed and mean absolute yaw rate are reported in

\mathrm{m/s}

and

\mathrm{rad/s}

, respectively.

Fig. 7: Qualitative comparison on the same navigation episode: (a) without the DFD branch, (b) without DFD supervision, and (c) the full model. The vectors and arrows indicate the selected acceleration actions.


DFD Configuration	SR (%) $\uparrow$	OSR (%) $\uparrow$	NE (m) $\downarrow$	SPL (%) $\uparrow$	CR (%) $\downarrow$	$\overline{v}$ (m/s) $\uparrow$	$\overline{\|\omega_{z}^{k}\|}$ (rad/s) $\downarrow$
No DFD branch	81.1	83.5	11.6	65.2	10.3	1.796	0.040
Predicted DFD, no supervision	85.7	88.4	10.3	68.9	8.5	1.791	0.044
Predicted DFD, supervised (KinoUAV-Agent)	89.3	90.8	8.8	71.9	6.0	1.680	0.045
+ noise level 0.1	86.7	88.9	10.1	69.3	6.3	1.640	0.040
+ noise level 0.3	85.0	86.8	12.3	68.1	8.0	1.706	0.036

TABLE VI: Analysis of DFD supervision and noise robustness on KinoUAV Test Seen. Noise is added to the supervised DFD variant.

Fig. 8: Qualitative comparison of generated trajectories, maximum absolute yaw rates, and navigation outcomes for TravelUAV, AerialVLA, and KinoUAV.

Safe Robot Navigation

Safe robot navigation develops methods for reaching goals while avoiding collisions and managing uncertainty, dynamic obstacles, and environmental hazards.

See all 139 papers

May 31, 2026

ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planning

Vision-language navigation (VLN) for UAVs demands grounding free-form instructions into 6-DoF flight under partial observability. While Vision-Language-Action (VLA) models excel at semantic reasoning, they suffer from brittleness due to geometric inconsistency and dynamics mismatch. To address this, we propose ImagineUAV, an imagination-driven framework leveraging cascaded world-action modeling. Instead of direct regression, ImagineUAV employs a latent video diffusion model to generate instruction-conditioned future observations, explicitly imagining environmental evolution, from which 6-DoF motions are inferred via an action extractor. A kinodynamic planner then refines these estimates into collision-free trajectories. Additionally, a step-distilled inference pipeline ensures real-time execution. With only 1.3B parameters, ImagineUAV outperforms prior VLN and VLA baselines on benchmarks and real-world flights, validating the practicality of imagination-driven aerial navigation.

Xuchen Liu, Jiawei Huang, Shihao Xia +3

Pengcheng Laboratory, Shenzhen, Guangdong, China · School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou, Guangdong, China · Southern University of Science and Technology, Shenzhen, Guangdong, China

UAV Navigation Vision-Language Navigation World Action Models Motion Planning Aerial VLN

Ask PDF

Jun 30, 2026

DynFly: Dynamic-Aware Continuous Trajectory Generation for UAV Vision-Language Navigation in Urban Environments

Recent advances in multimodal large models have significantly improved UAV vision-language navigation (UAV-VLN) by enhancing high-level perception and reasoning. However, existing methods mainly focus on predicting discrete actions, local targets, or sparse waypoints, while the continuous transition from navigation intent to executable UAV motion remains weakly modeled. This motion-interface gap limits the continuity, stability, and executability of generated UAV trajectories. To address this gap, we propose DynFly, a dynamic-aware continuous trajectory generation framework that bridges high-level navigation reasoning and executable UAV motion. DynFly bridges high-level navigation intent and continuous UAV motion through a lightweight trajectory generation layer. Specifically, it represents expert trajectories in B-spline control-point space and employs a Spline-DiT generator to learn conditional trajectory generation via flow matching. Furthermore, we introduce UAV-oriented dynamic-aware supervision over position, finite-difference velocity, finite-difference acceleration, heading consistency, and local target alignment, enabling the generated trajectories to better satisfy UAV motion characteristics. And our trajectory generation framework can also be integrated with an existing UAV-VLN framework while preserving its original visual-language reasoning pipeline. Extensive experiments on the OpenUAV UAV-VLN benchmark show that DynFly improves both navigation performance and trajectory quality. On the Test Unseen Full split, DynFly improves the strongest baseline by 4.69 NDTW, 2.40 SDTW, 2.14 SR points and 4.87 OSR points, while reducing NE by 4.51 m.

Wen Jiang, Hanfang Liang, Li Wang +9

School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China · Jianghan University, Wuhan 430056, China · Chongqing Innovation Center, Beijing Institute of Technology, Chongqing 401120, China +5

UAV Navigation Vision-Language Navigation Aerial VLN

Ask PDF

Jul 2, 2026

Lightweight Safe Reinforcement Learning for End-to-End UAV Navigation

With the rapid development of autonomous aerial systems, Unmanned Aerial Vehicles (UAVs) are increasingly deployed in applications such as inspection, environmental monitoring, and rescue, creating growing demand for reliable autonomous navigation. However, autonomous UAV navigation in dense environments remains challenging under sparse perception and dynamic constraints. Most reinforcement learning (RL) methods lack explicit safety mechanisms, leading to unsafe exploration, unstable training, and risky behaviors, especially during high-speed flight. Even in safe RL approaches, safety is often enforced by projecting policy outputs onto a safe action set, which may introduce instability. Meanwhile, many learning-based methods rely on dense inputs or large networks, increasing computational burden and limiting lightweight onboard deployment. Facing the above challenges, we propose a safety-constrained perception-control integrated framework for UAV navigation. A lightweight network encodes sparse observations into collision-risk-aware features using asymmetric and depthwise separable convolutions. We formulate the task as a constrained Markov decision process within a hierarchical control architecture and solve it using a Lagrangian-based safe PPO algorithm. Curriculum learning further improves training stability. Experiments with varying obstacle densities and flight speeds demonstrate higher success rates, improved safety, and better efficiency than existing reinforcement learning baselines.

Shenghui Zhang, YuXuan Gao, Songwei Zhao +3

Robot Policy Learning UAV Navigation Constrained RL Safe RL

Ask PDF

Abstract

Safe Robot Navigation

Explore similar work

ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planning

DynFly: Dynamic-Aware Continuous Trajectory Generation for UAV Vision-Language Navigation in Urban Environments

Lightweight Safe Reinforcement Learning for End-to-End UAV Navigation