AirPlayground: Benchmarking Aerial Navigation across Diverse Reconstructed Environments

Rangya Zhang; Yujing Shang; Zhuoyuan Yu; Mingsheng Li; Qingsong Yan; Chao Yan; Yuhang Zhang; Mir Feroskhan

cs.ROOct 21, 2026

AirPlayground: Benchmarking Aerial Navigation across Diverse Reconstructed Environments

Authors: Rangya Zhang, Yujing Shang, Zhuoyuan Yu, Mingsheng Li, Qingsong Yan, Chao Yan, Yuhang Zhang, Mir Feroskhan

Organizations: Nanyang Technological University · Wuhan University · Nanjing University of Aeronautics and Astronautics

Abstract

Large-scale aerial resources provide a foundation for learning navigation across diverse environments. However, existing resources remain limited in providing (1) densely sampled flight demonstrations for learning different policy formulations and (2) broad environment coverage for standardized closed-loop evaluation. To address these limitations, we introduce AirPlayground, a unified benchmark for training and evaluating aerial navigation policies across diverse environments. We first construct 94 photorealistic 3D Gaussian Splatting (3DGS) environments spanning indoor, urban outdoor, and vegetation settings, with aligned collision geometry. Building on these environments, we develop a scene-adaptive pipeline for automated expert flight collection, yielding 94K demonstrations and 46.5M synchronized observation-state-action samples at 50 Hz. We further provide an extensible API that supports training different policy formulations on this shared corpus and evaluating them under unified protocols in the corresponding environments. We benchmark seven baselines spanning four policy paradigms across three navigation settings in seen and unseen environments, and further examine scaling behavior with ACT. Scaling ACT's training corpus from 5 to 70 scenes raises median success rates on seen and unseen environments from 30.6% and 16.8% to 41.2% and 29.4%, respectively. Through this shared training and evaluation framework, AirPlayground supports analysis of how offline prediction errors relate to closed-loop outcomes. Our analyses highlight the roles of prediction errors, execution protocols, and scene interactions in closed-loop performance.

Figures & tables

Figure 1: The AirPlayground benchmark. Top: (1) 94 real-world scenes are reconstructed as executable 3DGS environments; (2) a unified API adapts the observation–state–action corpus to different policies; and (3) policies are evaluated under closed-loop protocols for point-goal, image-goal, and goal-free navigation. Bottom: representative indoor, urban, and vegetation-rich environments.

Figure 2: AirPlayground framework. Real-world captures become executable 3DGS scenes for closed-loop expert collection. A unified cross-policy API supports training on the resulting demonstrations and evaluation in the corresponding scenes.

Benchmark	Source	3DGS	Exec.	Scenes	Traj.	Supervision	Granularity
	Environment			Scale		Flight Experience
AerialVLN ( Liu et al., 2023 )	Simulation	–	✓	25	8.4K	Discrete 4-DoF	$\sim$ 204 steps/traj.
CityNav ( Lee et al., 2025 )	Real-city scans	–	✓	2 cities	32.6K	Discrete actions	–
OpenUAV ( Wang et al., 2025b )	Simulation	–	✓	22	12.1K	6-DoF trajectory	$\sim$ 264 steps/traj.
OpenFly ( Gao et al., 2026 )	Mixed	✓	Partial	18	100K	Discrete + waypoint	$\sim$ 35 actions/traj.
UAV-Flow ( Wang et al., 2025a )	Real + sim.	–	✓	3 real	40.8K	6-DoF state	5 Hz

Table 1: Comparison of aerial navigation resources. AirPlayground pairs executable 3DGS reconstructions with continuous supervision. “–” indicates an absent property or unreported quantity.

Figure 3: Visual appearance and scene geometry. (a) Reference photograph, (b) 3DGS rendering, and (c) mesh-derived depth at the matched viewpoint.

Figure 4: PointGoal rollouts with shared starts and goals in three scenes.

	Seen Offline				Seen Closed Loop				Unseen Offline				Unseen Closed Loop
Method	ADE $\downarrow$	FDE $\downarrow$	P95 $\downarrow$	Yaw $\downarrow$	SR $\uparrow$	A ∗ -SPL $\uparrow$	CR $\downarrow$	OB $\downarrow$	ADE $\downarrow$	FDE $\downarrow$	P95 $\downarrow$	Yaw $\downarrow$	SR $\uparrow$	A ∗ -SPL $\uparrow$	CR $\downarrow$	OB $\downarrow$
CNN-BC	0.54	0.81	2.52	0.0022	21.0	0.21	49.0	30.0	9.34	16.59	19.75	0.0054	15.0	0.14	47.5	37.5
ACT	1.67	3.07	16.86	0.0096	40.8	0.41	39.2	20.0	11.84	21.55	26.68	0.0171	29.5	0.30	65.0	5.5
DP	2.88	5.38	32.98	0.0316	20.5	0.19	65.5	14.0	22.15	40.27	58.04	0.1235	11.2	0.10	48.8	40.0
ViNT	7.11	12.81	19.85	0.0360	38.8	0.39	36.2	25.0	15.13	27.50	38.24	0.0422	33.0	0.33	64.5	2.5
OmniVLA	4.99	8.11	15.01	0.0454	47.5	0.47	38.8	13.7	20.64	37.95	45.58	0.0497	40.5	0.38	59.5	0.0

Table 2: Offline and closed-loop PointGoal performance on seen/unseen scenes. Offline errors use a 0.2 s horizon: ADE/FDE/P95 in cm; mean absolute endpoint yaw error in radians. P95 denotes scene-averaged 95th-percentile FDE. Closed-loop protocol: Sec. 4.1 .

Figure 5: Offline diagnostics and closed-loop outcomes. (a) Direction-aligned Spearman correlations: positive correlations link errors to worse outcomes. (b) Signed correlation differences for CR relative to mean FDE. Correlations use seven scores, averaging seen/unseen splits (Appendix B.4.1 ).

Figure 8: Offline prediction errors and closed-loop trajectories around an obstacle.

Change	$N$	FDE (cm)	Expert $\rightarrow$ Pred. (cm)	Closed-loop min. (cm)	Collision	Local pass
Decrease	43	8.4	25.6 $\rightarrow$ 18.9	5.2	16/43 (37.2%)	25/43 (58.1%)
Stable	76	7.9	26.2 $\rightarrow$ 26.5	10.8	16/76 (21.1%)	56/76 (73.7%)
Increase	61	8.8	25.9 $\rightarrow$ 32.1	15.7	8/61 (13.1%)	49/61 (80.3%)

Table 3: Outcomes by predicted clearance change. Group-mean distances; counts out of

N

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Scene diversity and corpus scale. Representative views across indoor, urban outdoor, and vegetation environments.

Navigable space and task sampling
Parameter	Reference value
Planning-grid cell width $\delta$	$0.10$ m
Endpoint candidates $N$	$128$
Minimum endpoint separation $d_{\min}$	$2.0$ m
Additional endpoint-clearance margin	$0.40$ m
Endpoint distance-quantile filter	$0.0$

Appendix

Table 4: Reference settings of the collection implementation. Scene files and launch arguments must be consulted when reproducing a particular collection run.

Figure 10: Reconstructed scenes across three environment categories. Indoor, urban outdoor, and vegetation examples.

Split	Indoor	Urban outdoor	Vegetation	Total	Trajectories	Min–max
	Scenes				Flight data	Area (m 2 )
Training	11	38	21	70	70,000	23–6,674
Validation	3	3	2	8	8,000	14–1,640
Unseen test	4	7	5	16	16,000	118–4,543
Total	18	48	28	94	94,000	14–6,674

Appendix

Table 5: Dataset splits.

		Seen		Unseen
Environment	Spatial structures	Scenes	Area range (m 2 )	Scenes	Area range (m 2 )
Indoor	Rooms, corridors, open halls	4	80–1,079	4	118–863
Urban outdoor	Covered passages, streets, plazas	7	200–6,674	7	502–4,543
Vegetation	Wooded paths, hillside gardens, coastal parks	5	141–1,191	5	128–1,678

Appendix

Table 6: Evaluation scenes.

Metric	Definition
Motion Direction Error	Angle between predicted and expert net displacements; undefined for zero displacement.
Band-limited DTW	Minimum cumulative position distance over monotone alignments, divided by $K$ ; band $\|i-j\|\leq\max(1,\lceil 0.1K\rceil)$ .
Lateral / Yaw Direction Error Rate	Opposite-sign fraction for expert lateral displacement $\|d_{y}\|\geq 0.02$ m / yaw change $\|\Delta\psi\|\geq 0.02$ rad. Lateral displacement uses the initial body frame.
Yaw Error	Absolute wrapped difference between predicted and expert yaw changes at the 0.2 s endpoint, averaged over valid anchors within trajectories, then equally across trajectories within scenes and across scenes.
P95 FDE	95th percentile of anchor-level FDE within each scene, then averaged equally across scenes.
Corner FDE	FDE for a $\geq 20^{\circ}$ angle between the first and last 0.1 s displacements of the next second; both lengths must exceed 0.02 m.

Appendix

Table 7: Offline diagnostics.

Method	RGB history	Native output	Pred. span (s)	Exec. $H$	Deployment adapter
CNN-BC	1/1	1 control	0.02	1	Direct / hold
ACT	1/1	50 controls	1.00	50	Prefix
DP	2/1	32 controls	0.64	16	Prefix
ViNT	6/5	5 waypoints	0.50	10	Tracker
OmniVLA	1/1	8 poses	0.16	8	Pose differences / prefix
NWM	4/5	Future view	0.80	40	Rank / hold

Appendix

Table 8: Policy adapters.

Method	Updates	Batch	Peak LR	Training scope
CNN-BC	5,280	256	(1–3) $\!\times\!10^{-4}$	Full policy
ACT	34,650	32	$10^{-5}$	Full policy
DP	54,450	32	$10^{-4}$	Full policy
ViNT	8,418	256	$10^{-6}$ – $10^{-4}$	Goal modules + decoder
OmniVLA	500	32	(2–5) $\!\times\!10^{-5}$	LoRA + adapters
NWM	604	64	$10^{-5}$	Full predictor

Appendix

Table 9: Training configurations.

Figure 11: Navigation under different goal guidance. Paths share a start and evaluation target; the target is withheld from Goal-free policies.

Method	PointGoal							ImageGoal							Goal-free
Method	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	CR $\downarrow$	OB $\downarrow$	TTS $\downarrow$	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	CR $\downarrow$	OB $\downarrow$	TTS $\downarrow$	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	CR $\downarrow$	OB $\downarrow$	TTS $\downarrow$
CNN-BC	6.44	20.8	18.0	0.18	48.2	33.8	7.21	5.60	10.2	8.3	0.07	59.2	32.5	5.49	5.84	10.5	6.9	0.06	19.8	73.3	5.46
ACT	4.38	39.1	35.1	0.35	52.1	12.7	4.74	4.08	26.7	24.8	0.22	54.3	20.9	3.21	4.34	16.4	12.1	0.10	63.5	24.4	2.42
DP	7.17	26.5	15.9	0.14	57.1	27.0	5.88	4.13	24.4	20.2	0.16	52.7	27.1	3.58	4.43	16.4	13.1	0.11	60.1	26.8	3.75
ViNT	4.94	39.0	35.9	0.36	50.4	13.7	8.05	2.20	52.1	51.1	0.48	39.7	9.2	6.01	4.36	17.1	13.4	0.11	60.8	25.8	3.45
OmniVLA	3.81	46.5	44.0	0.43	49.1	6.9	6.83	2.00	53.8	52.3	0.47	42.8	4.9	5.44	4.08	32.6	19.6	0.15	49.4	31.0	3.74

Appendix

Table 10: Closed-loop navigation across goal modes. The full protocol reuses 50 tasks per scene across settings in 16 seen and 16 unseen scenes; see Appendix B.3 for aggregation and coverage.

Figure 12: Execution-interval sweeps. Supplementary two-scene evaluation, separate from the main-text sweep. Every point uses the same ten tasks in its scene; each method retains its checkpoint across intervals. The two rows show SR and CR. The horizontal axis lists all evaluated intervals on a logarithmic scale.

Figure 13: How execution changes a maneuver. A separate matched-task ACT diagnostic. (a–b) Matched-start ACT trajectories on a flight-height occupancy slice: dark gray denotes obstacles and light gray lies outside the navigable polygon. Dots mark queries, stars goals, and crosses episode ends. (c) Executed yaw commands. (d) All four measurable turn-decay pairs, colored by $H=1$ outcome. States evolve independently after the shared start.

			Clean		Toward		Tangential		Away
Method	Margin	$N$	$\mathrm{CR}^{\mathrm{ref}}\downarrow$	$d_{\min}^{\mathrm{ref}}\uparrow$	$\mathrm{CR}^{\mathrm{ref}}\downarrow$	$d_{\min}^{\mathrm{ref}}\uparrow$	$\mathrm{CR}^{\mathrm{ref}}\downarrow$	$d_{\min}^{\mathrm{ref}}\uparrow$	$\mathrm{CR}^{\mathrm{ref}}\downarrow$	$d_{\min}^{\mathrm{ref}}\uparrow$
ACT	Low	17	30.0	0.221	30.0	0.192	32.1	0.202	20.0	0.254
ACT	Medium	15	20.0	0.451	25.0	0.368	10.0	0.480	15.0	0.476
ACT	High	15	5.0	0.836	5.0	0.766	0.0	0.850	5.0	0.867
NavMorph	Low	14	35.0	0.194	35.0	0.178	35.0	0.185	30.0	0.211
NavMorph	Medium	17	15.0	0.476	15.0	0.431	10.0	0.497	15.0	0.514

Appendix

Table 11: Local disturbance outcomes. Disturbance magnitude: 0.5 m/s.

\mathrm{CR}^{\mathrm{ref}}

is in percent and

d_{\min}^{\mathrm{ref}}

in meters;

N

counts accepted anchors. Results weight the two scenes equally.

Figure 14: Paired effects within a fixed local region. Controlled disturbances at 93 anchors, separate from the 180-sample clearance grouping. Top: local CR and common-window clearance contrasts with 95% task-bootstrap intervals. Bottom: the first eligible anchor in each displayed policy–scene–clearance group. Dashed circles delimit the 1 m observation region; tan cells show the reference patch. Crosses mark reference collisions and triangles region exits. Coordinates are anchor-relative. Zero-width CR intervals arise from all-zero paired differences and do not establish equivalence.

UAV Navigation

Unmanned aerial vehicle (UAV) navigation develops methods for locating and guiding drones safely through known, unknown, and changing environments.

See all 272 papers

AerialDojo-200K: A Large-Scale Benchmark Suite for Open-World Aerial Object-Goal Search
Sep 28, 2026Tongtong Feng, Xin Wang, Haoran Hou +9Aerial Robotics Object-Goal Navigation
Department of Computer Science and Technology, BNRist, Tsinghua University · School of Electrical & Electronic Engineering, University College Dublin · School of Electronics Engineering and Computer Science, Peking University +1
AirDreamer: Generalist Drone Navigation with World Models
Jun 2, 2026Zian Liu, Andong Yang, Chunkai Yang +3RL for Robotics World Models for Robotics
Institute for AI Industry Research, Tsinghua University, Beijing, China. · Department of Electronic Engineering, Tsinghua University, Beijing, China. · School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, China.
AirGroundVLN: A Large-Scale Benchmark for Goal-Oriented Air-Ground Collaborative Vision-and-Language Navigation
Oct 7, 2026Zhenxuan Zeng, Qingle Wu, Wei Suo +4Vision-Language Navigation Aerial VLN
School of Computer Science, Northwestern Polytechnical University, China.

Sep 28, 2026J/K move · Enter open · S save

AerialDojo-200K: A Large-Scale Benchmark Suite for Open-World Aerial Object-Goal Search

Tongtong Feng, Xin Wang, Haoran Hou, Ren Wang+8

Department of Computer Science and Technology, BNRist, Tsinghua University · School of Electrical & Electronic Engineering, University College Dublin · School of Electronics Engineering and Computer Science, Peking University+1

Aerial Robotics Object-Goal Navigation Agent Benchmarks

Open paper AskPDF arXiv

Open-world aerial object-goal search is a foundational yet challenging task, requiring aerial agents to autonomously explore large-scale, unstructured three-dimensional environments and reach target objects specified by semantic descriptions or reference images, rather than following route-specific instructions. However, research in this task remains at a nascent stage and relies on small, environment-specific benchmarks with heterogeneous action spaces and data formats. These limitations hinder large-scale training and cross-benchmark evaluation, constraining the scalability and generalizability of aerial agents. To address this problem, we propose AerialDojo-200K, a large-scale benchmark suite for open-world aerial object-goal search, with 3 times as many scenes and 18.7 times as many task instances as the largest existing benchmark for this task. Specifically, we construct 42 simulation scenes spanning four scene families and 21 scene types, including 18 urban, 12 natural, six infrastructure, and six disaster scenes. To ensure data quality, 12 annotators spent two months manually annotating 109 landmarks, 2099 target objects, and 2099 object anchors across these scenes. We further construct 205,732 task instances, comprising over 100K semantic-goal and over 100K image-goal instances across Base, Standard, and Long-Horizon settings. Each task instance includes a collision-free reference trajectory and corresponding multi-view video recordings. We also develop a unified evaluation framework with a scene partition comprising 21 in-distribution scenes and 21 out-of-distribution scenes. Finally, our evaluation of five open-source and four closed-source multimodal large language models reveals that there is still a long way to go toward achieving general-purpose aerial agents. All can be found at https://fengtt42.github.io/AerialDojo/.

Abstract

UAV Navigation

Explore similar work

AerialDojo-200K: A Large-Scale Benchmark Suite for Open-World Aerial Object-Goal Search

AirDreamer: Generalist Drone Navigation with World Models

AirGroundVLN: A Large-Scale Benchmark for Goal-Oriented Air-Ground Collaborative Vision-and-Language Navigation