cs.RONov 8, 2026

ScanDreamer: A Map-Grounded World Model for Quadrotor Navigation

Authors: Yunyi Gao, Yixing Shi, Yize Liu, Hai-Duong Tran, Nhat-Hoang P. Nguyen, Shuai Wang, Yi Deng, Shengming Liu, 

Organizations: School of Artificial Intelligence and Robotics, Xiamen University Malaysia, Malaysia · Center for AI Research, VinUniversity, Hanoi, Vietnam · Air Traffic Management Research Institute, Nanyang Technological University, Singapore

Abstract

Quadrotors inspecting buildings and tunnels must navigate unknown clutter with a forward-facing depth camera whose limited field of view constrains scene understanding and collision-avoiding navigation. Recent world-model agents learn navigation through imagined experience (e.g., AirDreamer), but remain data-intensive and often lack explicit geometric memory and awareness of what the camera can observe. We propose ScanDreamer, a map-grounded world model that constructs an online 2D scan-memory map from depth images and vehicle pose. Our world model, actor, and critic are conditioned on this map, which also defines a geodesic progress reward. Moreover, a forward-cone penalty discourages horizontal motion outside the camera's viewing direction. During imagination, ScanDreamer propagates the quadrotor pose over a frozen map and derives progress and wall-contact signals from the mapped geometry. We train ScanDreamer for 0.8 million environment steps (5.5 h on one GPU) and evaluate it in NVIDIA Isaac Sim over 1560 episodes spanning unseen layouts and parameter ranges. Across three seeds (mean ±\pm SD), ScanDreamer achieves 96.3 ±\pm 0.3% success with 3.6 ±\pm 0.3% collision, compared with 59.6 ±\pm 3.0% success and 28.8 ±\pm 2.8% collision for AirDreamer at the same online-interaction budget. Removing the forward-cone penalty increases collisions to 9.9 ±\pm 3.1%. These results show that combining explicit map memory, map-grounded imagination, and perception-aware motion substantially improves navigation performance and data efficiency. The source code is available at https://github.com/WANGSHUAIzzzz/ScanDreamer.

Figures & tables

Explore similar work

Jun 3, 2026

MAD: Mapping-Aware World Models for Agile Quadrotor Flight

Agile quadrotor flight in cluttered scenes requires more than a reactive mapping from a depth image to a control command: the vehicle must remember which regions have been observed, infer nearby occupied space, and act under partial visibility and tight latency. In this paper, we present Mapping-Aware Dreamer (MAD), a geometry-aware world model for vision-based quadrotor flight. Instead of using raw-image reconstruction as the main self-supervised objective, MAD learns recurrent latent dynamics that reconstruct robocentric occupancy and visibility grid maps together with proprioceptive states. This design forces the latent state to encode local geometry, visibility history, and ego-motion in a form that is directly relevant to collision avoidance. MAD is trained in DiffAero using a GPU-parallel map-construction module that provides high-throughput supervision for occupancy and visibility. The learned representation is used in three policy-learning modes: imagination-based MAD-Dreamer and feature-extractor variants based on PPO and SHAC. Across visual navigation and racing tasks, MAD-based agents achieve higher success rates, faster flight, and better cross-task transfer than corresponding vision-only baselines. The model also produces interpretable map predictions and accurate ego-motion estimates from depth observations. We further deploy the learned policy on a physical quadrotor with an Intel RealSense D435i and demonstrate safe indoor and outdoor flight under limited sensing, reaching 9.66 m/s in simulation and 5.05 m/s in real-world forest experiments. These results show that mapping-aware world models provide a practical middle ground between modular aerial navigation and end-to-end learning.
Jun 2, 2026

AirDreamer: Generalist Drone Navigation with World Models

Navigating a drone in unseen and cluttered environments requires reliable generalization to unseen scene layouts and understanding of environmental structure relative to the robot's capabilities. Previous methods, which assume the same environment configuration, often rely heavily on human-designed perception pipelines and predefined rules to guide the robot toward the target. This process is environment-dependent and generalizes poorly across environments. Inspired by animal navigation behavior, we design a navigation framework that navigates with a reinforcement-learning-based policy on top of a world-model-based environment understanding to overcome these issues. In addition, a sparse reward function without hand-crafted shaping terms is designed to avoid local minima traps and encourage yaw control behaviors. In simulation and on real drones, our method exhibits emergent capabilities for navigating complex, unseen environments and escaping local optima where other methods fail. In challenging maps, it achieves a 5.3% higher navigation success rate than best baseline. Furthermore, the proposed framework achieves effective sim-to-real transfer without any tuning during deployment. The code will be publicly available.
Jun 3, 2026

Flying on Imagination: DreamerV3-Style World Models under Environment Distribution Shift for Quadrotor Navigation

DreamerV3-style reconstructive world models offer a route to vision-based navigation for aerial robots through learned latent dynamics. Yet their behavior under shifts in the training environment and their ability to support navigation using imagined observations, predictions generated by rolling the latent dynamics forward without new sensory input, remain insufficiently characterized. We study four training-environment regimes for depth-based quadrotor navigation, spanning a fixed layout, a finite set of layouts, low-discrepancy continuous obstacle sampling, and i.i.d. uniform sampling. We cross-evaluate reconstruction before and after Reinforcement Learning fine-tuning, separately evaluate the resulting policies in simulation, and transfer them to a real quadrotor without real-world fine-tuning. We demonstrate onboard flight driven by imagined state and depth: a 24.7m mission in which 66% of flight time and 76% of traveled distance occur during repeated 5s sensor-free imagination windows, with prediction quality degrading over each open-loop interval and recovering after sensory re-grounding. Additionally, in closed-loop hardware transfer, all policies succeed in a setting close to the simulated geometry, whereas the policy trained from a single fixed layout fails under a corridor-topology and obstacle-type shift, while the other three succeed.