Organizations: National Key Laboratory of Digital Intelligent Modeling and Simulation, National University of Defense Technology · College of Systems Engineering, National University of Defense Technology · Aerospace Information Research Institute, Chinese Academy of Sciences · BNRist, Tsinghua University · Southeast University
Autonomous unmanned aerial vehicle (UAV) object search involves a closed loop of perception, decision-making, and action under partial observability. Urban environments pose several challenges: large search areas and narrow egocentric views limit coverage, dense 3D geometry constrains safe motion, and open-world instructions require identifying a specific target among distractors. Many existing methods mitigate partial observability through explicit maps or memory representations, yet remain largely reactive, reasoning over past observations without explicitly predicting future states. World models enable prospective reasoning through imagined rollouts. However, image-generating world models can incur high inference latency, while spatially grounded planning remains challenging for latent world models. We propose SearchWorld, a recurrent state-space world model that connects explicit spatial memory with value-guided imagination. The model maintains BEV exploration and obstacle memory and decodes a task-aware spatial value layer to guide search. A cognition-action network uses this learned spatial value prior to improve the policy through imagined rollouts, without training a separate scalar critic. Training progresses from world-model learning to expert imitation and imagination-based exploration refinement. On UAV-ON, SearchWorld improves the success rate to 23.8% (19.5% for the strongest published agent) and raises oracle success to 35.5%, while remaining robust on unseen scenes (19.9% success rate). By grounding imagination in explicit spatial representations, SearchWorld enables UAV agents to plan prospectively rather than react.
Figures & tables
Figure 1: SearchWorld overview with its three-stage curriculum. A task-oriented RSSM world model encodes the RGB view, the instruction, and the explicit spatial-cognition memory, and decodes the observation, the spatial memory, the task-aware value layer, and the UAV action. The cognition–action network trains the action policy in Stage 2 and refines movement actions in Stage 3 by pooling the decoded value over imagined action footprints.
Method
Small
Medium
Large
Total
SR ↑
OSR ↑
SPL ↑
SR ↑
OSR ↑
SPL ↑
SR ↑
OSR ↑
SPL ↑
SR ↑
OSR ↑
SPL ↑
Random
4.14%
7.80%
2.80%
3.33%
8.10%
3.05%
2.48%
8.07%
1.62%
3.70%
8.00%
2.66%
CLIP-H
2.86%
8.43%
1.51%
10.95%
16.67%
7.17%
13.04%
19.25%
10.53%
6.20%
11.90%
4.15%
AOA-V
2.86%
25.44%
0.54%
5.71%
27.62%
1.73%
7.45%
27.95%
1.038%
4.20%
26.30%
0.87%
AOA-F
4.45%
16.38%
1.61%
10.48%
17.62%
6.36%
14.29%
21.74%
10.66%
7.30%
17.50%
4.06%
WMNav
5.64%
20.82%
1.77%
9.04%
23.91%
3.30%
16.20%
28.29%
6.56%
8.04%
22.67%
2.86%
Table 1: Performance across different object sizes and overall total, evaluated using three metrics: Success Rate (SR ↑ ), Oracle Success Rate (OSR ↑ ), and Success weighted by Path Length (SPL ↑ ).
Table 3
Configuration
SR ↑
OSR ↑
SPL ↑
SearchWorld (Ours, full)
23.80%
35.52%
8.03%
(Q1) World-model architecture
direct imitation (BC, no world model)
14.33%
25.68%
7.20%
X-Mobility-style (RSSM + imitation only)
12.91%
19.88%
6.95%
Dreamer-style (scalar critic in imagination)
19.07%
24.99%
7.64%
(Q2) Explicit spatial-cognition memory
Table 4: Ablation study on UAV-ON.
Figure 2: Case study of a successful search. The target is highlighted by the red box
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Parameter
Value
BEV grid
grid size
256×256
cell resolution
0.4m
region coverage
102.4m×102.4m
grid centering
search-region center
Exploration layer
sensing range ρ
20m
FOV width ϕ
90∘
Appendix
Table 5: System specifications of SearchWorld referenced in Section 3.2 and Section 3.5 .
Figure 3: A representative failure case. The instruction names the target instance (a gray utility box truck). The UAV explores its surroundings in the early panels but stays too far from the target to detect it, and the step budget is exhausted in the complex environment.
End-to-end Vision-Language-Action (VLA) models have shown promise in UAV navigation. However, existing approaches typically rely on historical observations to directly predict actions, often struggling in dense urban environments where severe occlusions and sharp turns result in drastic viewpoint transitions. We argue that the ability to "imagine" future states -- inherent in World Models -- is critical for robust decision-making under such partial observability. To address this, we construct a challenging Urban Canyon Traversal Benchmark, specifically designed to evaluate spatial understanding in scenarios characterized by severe occlusions and drastic viewpoint transitions. To this end, we propose WorldFly, a novel world-model-based VLA framework that employs a dual-branch coupled flow matching mechanism to jointly generate future video predictions and navigation actions, thereby explicitly guiding the agent's policy via spatial imagination. Extensive evaluations on our benchmark demonstrate that WorldFly outperforms other baselines, particularly in unseen environments, validating the effectiveness of integrating world models into embodied aerial agents.
Shengtao Zheng, Kai Li, Weichen Zhang +5
Tsinghua Shenzhen International Graduate School · BNRist, Tsinghua University
Vision-language navigation (VLN) for UAVs demands grounding free-form instructions into 6-DoF flight under partial observability. While Vision-Language-Action (VLA) models excel at semantic reasoning, they suffer from brittleness due to geometric inconsistency and dynamics mismatch. To address this, we propose ImagineUAV, an imagination-driven framework leveraging cascaded world-action modeling. Instead of direct regression, ImagineUAV employs a latent video diffusion model to generate instruction-conditioned future observations, explicitly imagining environmental evolution, from which 6-DoF motions are inferred via an action extractor. A kinodynamic planner then refines these estimates into collision-free trajectories. Additionally, a step-distilled inference pipeline ensures real-time execution. With only 1.3B parameters, ImagineUAV outperforms prior VLN and VLA baselines on benchmarks and real-world flights, validating the practicality of imagination-driven aerial navigation.
Xuchen Liu, Jiawei Huang, Shihao Xia +3
Pengcheng Laboratory, Shenzhen, Guangdong, China · School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou, Guangdong, China · Southern University of Science and Technology, Shenzhen, Guangdong, China
Aerial image-goal navigation requires an unmanned aerial vehicle (UAV) to reach a target location specified by a goal image. Existing world-model-based methods rank candidate trajectories using predicted futures, but typically rely on only one or a few point predictions, which is inadequate for large-scale outdoor environments with substantial future-state uncertainty. To address this limitation, we propose the Uncertainty-Aware Navigation World Model (UA-NWM), an efficient latent world model for aerial image-goal navigation, which formulates trajectory scoring as conditional out-of-distribution detection. UA-NWM represents plausible futures with an uncertainty subspace and decomposes the prediction--goal discrepancy into uncertainty-explainable and unexplainable components. Only the unexplainable residual is used for scoring, enabling robust selection without multiple future samples. Extensive experiments demonstrate that UA-NWM consistently outperforms existing navigation world models while maintaining low inference latency. Real-world UAV experiments further validate its practical applicability. Project page: https://duryi.github.io/UA-NWM-Project-Page
Deyi Zhu, Haoyu Fan, Yinan Zhu +4
Tsinghua Shenzhen International Graduate School, Tsinghua University