Benchmarking Interaction-Oriented Aerial Mobility across Heterogeneous Environments

Zhibo Duan; Hanlin Kong; Tianran Ma; Yuze Wu; Yuxuan Xu; Kenghou Hoi; Changheng Xie; Xin Zhou

cs.ROOct 13, 2026

Benchmarking Interaction-Oriented Aerial Mobility across Heterogeneous Environments

Authors: Zhibo Duan, Hanlin Kong, Tianran Ma, Yuze Wu, Yuxuan Xu, Kenghou Hoi, Changheng Xie, Xin Zhou, +3 more

Organizations: Zhejiang University · Differential Robotics

Abstract

Aerial mobile manipulation requires unmanned aerial vehicles (UAVs) to traverse heterogeneous environments such as indoor and outdoor domains, and connect navigation with subsequent physical interaction within the same trajectories. Existing aerial benchmarks are typically confined to single-environment settings, limiting evaluation of agents across continuous environmental transitions. When interaction is evaluated, agents are often initialized near the target, leaving the transition from navigation to interaction largely unassessed. This work introduces CrossFly, a benchmark for long-horizon interaction-oriented aerial mobility across heterogeneous environments. CrossFly contains 15K trajectories across more than 20 reconstructed scenes under a unified simulation and evaluation framework, covering Indoor, Outdoor, and Indoor $\leftrightarrow$ Outdoor trajectories. It further introduces Interaction Readiness Rate (IRR) to assess whether terminal navigation states provide suitable conditions for downstream interaction without assuming a specific manipulation policy. Experiments with five representative aerial vision-language-action (VLA) agents and the proposed CrossFly-Agent show that Indoor $\leftrightarrow$ Outdoor settings are more challenging than single-environment settings, with the highest success rate reaching 9.7% on unseen scenes. A downstream manipulation case study further validates the relevance of interaction-oriented arrival to subsequent physical interaction. We hope CrossFly helps advance aerial embodied intelligence toward unified mobility and physical interaction in heterogeneous environments.

Figures & tables

Figure 1: Overview of CrossFly benchmark. It focuses on interaction-oriented aerial mobility tasks and consists of a large dataset for unmanned aerial vehicle (UAV) imitation learning, featuring multiple aerial embodiments, heterogeneous environments, photorealistic rendering, and diverse target objects. To enable systematic experimental analysis under interaction-oriented mobility task settings, we additionally provide an evaluation protocol and propose an interaction readiness metric.

Figure 2: Trajectory statistics and scene diversity. (a) Distribution of scene types and task instructions with different numbers of subtasks (§ 3.1 ). (b) Distribution of trajectory lengths and path distances, summarizing the temporal and spatial scales of the dataset. (c) Representative environments in CrossFly , where each scene covers indoor and outdoor environments.


Dataset	Platform	Scene	$N_{\text{traj}}$	$N_{\text{scenes}}$	Environment	Interaction	Dynamics	DoF
R2R ( Anderson et al., 2018 )	Ground Agent	Matterport3D	7,189	90	Indoor	✗	✗	Graph-based
VLN-CE ( Krantz et al., 2020 )	Ground Agent	Habitat	4,475	90	Indoor	✗	✗	2
OpenUAV ( Schmittle et al., 2018 )	UAV	AirSim + UE	12,149	22	Outdoor	✗	✓	6
AVDN ( Fan et al., 2023 )	UAV	xView	6,269	–	Outdoor	✗	✗	3
CityNav ( Lee et al., 2025 )	UAV	SensatUrban	32,637	34	Outdoor	✗	✗	4

Table 1: Comparison with related benchmarks (§ 3.1 ).

Figure 3: Overview of the dataset construction pipeline (§ 3.2 ). CrossFly begins with reconstructed scenes from point clouds and realistic rendering, followed by waypoint annotation and trajectory acquisition through UAV simulation. The collected trajectories are then assembled with RGB observations, UAV states, and task-related annotations to construct the benchmark dataset.

Figure 4: Architecture of CrossFly-Agent (§ 4.1 ) . A high-level VLM planner uses visual context spanning the observed trajectory to predict the active language subtask. Conditioned on this subtask, a low-level VLA policy combines the initial observation with recent visual context to generate flight action chunks and determine when to predict a manipulation-invocation signal.


	Indoor				Outdoor				Indoor $\leftrightarrow$ Outdoor
Model	NE $\downarrow$	SR $\uparrow$	OSR $\uparrow$	IRR $\uparrow$	NE $\downarrow$	SR $\uparrow$	OSR $\uparrow$	IRR $\uparrow$	NE $\downarrow$	SR $\uparrow$	OSR $\uparrow$	IRR $\uparrow$
$\pi_{0}$ ( Black et al., 2024 )	4.07	17.7	23.6	37.2	3.79	20.4	27.8	37.0	7.21	13.2	19.7	29.1
$\pi_{0}$ -FAST ( Pertsch et al., 2025 )	3.81	18.5	19.2	39.0	3.42	22.7	30.1	38.9	6.63	15.8	22.1	30.7
OpenFly-Agent ( Gao et al., 2026 )	3.48	15.1	21.4	46.7	3.39	18.1	20.6	43.4	5.93	13.9	17.7	34.2
OpenVLA-OFT ( Kim et al., 2025 )	3.84	17.2	21.7	47.9	3.51	19.5	24.4	38.1	6.87	14.9	21.3	38.8

Table 2: Quantitative comparison on seen scenes (§ 4.2 ).


	Indoor				Outdoor				Indoor $\leftrightarrow$ Outdoor
Model	NE $\downarrow$	SR $\uparrow$	OSR $\uparrow$	IRR $\uparrow$	NE $\downarrow$	SR $\uparrow$	OSR $\uparrow$	IRR $\uparrow$	NE $\downarrow$	SR $\uparrow$	OSR $\uparrow$	IRR $\uparrow$
$\pi_{0}$ ( Black et al., 2024 )	5.72	10.8	13.4	26.7	5.26	13.5	15.8	27.8	9.38	2.4	4.1	21.7
$\pi_{0}$ -FAST ( Pertsch et al., 2025 )	5.18	13.2	15.7	28.4	4.83	15.6	17.2	29.1	8.74	3.7	6.3	25.2
OpenFly-Agent ( Gao et al., 2026 )	4.49	10.6	13.9	32.7	4.22	13.4	15.1	26.4	7.61	7.2	7.9	26.2
OpenVLA-OFT ( Kim et al., 2025 )	5.45	11.9	14.6	27.5	4.97	14.1	16.4	28.2	9.02	3.1	4.7	27.9

Table 3: Quantitative comparison on unseen scenes (§ 4.2 ).

Figure 5: Qualitative results of CrossFly-Agent (§ 4.3 ). This example shows the instruction and selected third-person views. Blue text denotes navigation actions, while red text denotes manipulation actions. The top row shows the trajectory from the start to the interaction-ready state, while the bottom row covers the trajectory from the interaction-ready state to task completion.


Task Suite	Traj. Length	# Subtasks	Instr. Length	NE $\downarrow$	SR $\uparrow$
Indoor	11.2	6.8	18.9	4.15	23.64
Outdoor	11.8	6.7	18.5	4.36	25.28
Indoor $\leftrightarrow$ Outdoor	11.5	6.8	18.8	6.43	10.98

Table 4: Performance comparison under matched trajectories.

Figure 6: Manipulation case study (§ 4.4 ). Each row represents a manipulation case. The leftmost image shows the initial observation of UAV. The top case satisfies IRR criteria and achieves a successful grasp, whereas the bottom case violates IRR criteria and fails to grasp.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Qualitative results of CrossFly-Agent .

Figure 8: Failure case. Sometimes, UAV collides with obstacles when traversing passages across indoor–outdoor boundaries, indicating that navigation in transition regions remains challenging.


Split	# Scenes	Scene overlap with training	Used for training
Training	18	–	Yes
Seen	18	Same scenes	No
Unseen	5	None	No

Appendix

Table 5: Scene-level partition used in CrossFly (§ B.1 ).


Training Configuration	Setting
Batch size	32
Learning rate	$1\times 10^{-5}$
Learning rate schedule	Warmup + cosine decay
Optimizer	AdamW
AdamW $\beta$	$(0.9,0.95)$

Appendix

Table 6: Training configuration for VLA models.

Explore similar work

ActiveFly-Bench: Aligning Embodied Question Answering with Vision-Language-Action for Aerial Embodied Perception
Jul 11, 2026Weichen Zhang, Shiquan Yu, Yinan Zhu +9Active Perception Aerial Robotics
1Tsinghua University · 2Manifold AI
AeroManip-VLA: Scalable Vision-Language-Action Learning for Aerial Manipulation with RL-Generated Demonstrations
Sep 29, 2026Rui Huang, Yanlin Mu, Lidong Li +3Synthetic Data Generation for Robotics Vision-Language-Action Models
National University of Singapore · Beijing Institute of Technology
WorldFly: A World-Model-Based Vision-Language-Action Model for UAV Navigation
Jun 4, 2026Shengtao Zheng, Kai Li, Weichen Zhang +5World Models for Robotics Robot Navigation
Tsinghua Shenzhen International Graduate School · BNRist, Tsinghua University

Jul 11, 2026J/K move · Enter open · S save

ActiveFly-Bench: Aligning Embodied Question Answering with Vision-Language-Action for Aerial Embodied Perception

Weichen Zhang, Shiquan Yu, Yinan Zhu, Peizhi Tang+8

1Tsinghua University · 2Manifold AI

Active Perception Aerial Robotics Embodied QA Vision-Language-Action Models

Open paper AskPDF arXiv

We introduce ActiveFly-Bench, the first benchmark to bridge cyberspace reasoning and physical-world interaction for UAV embodied perception. The benchmark decomposes active perception into three hierarchical tasks: Aerial Embodied Question Answering (Air-EQA), Observation Behavior Planning (OBP), and Fine-grained Language-guided UAV Control (FLUC), explicitly connecting high-level task understanding, behavior planning, and low-level control. The datasets are collected from both real-world and simulated outdoor environments for training and evaluation. We further develop ActiveFly, a closed-loop UAV agent that integrates visual-language reasoning with fine-grained control, and deploy it on a physical UAV platform. Experiments with representative VLMs and VLA models show that current UAV agents still struggle with behavior planning, viewpoint adjustment, and robust task completion in active perception. These results establish ActiveFly-Bench as a new testbed for embodied aerial intelligence.

Abstract

Explore similar work

ActiveFly-Bench: Aligning Embodied Question Answering with Vision-Language-Action for Aerial Embodied Perception

AeroManip-VLA: Scalable Vision-Language-Action Learning for Aerial Manipulation with RL-Generated Demonstrations

WorldFly: A World-Model-Based Vision-Language-Action Model for UAV Navigation