ScanDreamer: A Map-Grounded World Model for Quadrotor Navigation
Organizations: School of Artificial Intelligence and Robotics, Xiamen University Malaysia, Malaysia · Center for AI Research, VinUniversity, Hanoi, Vietnam · Air Traffic Management Research Institute, Nanyang Technological University, Singapore
Abstract
Quadrotors inspecting buildings and tunnels must navigate unknown clutter with a forward-facing depth camera whose limited field of view constrains scene understanding and collision-avoiding navigation. Recent world-model agents learn navigation through imagined experience (e.g., AirDreamer), but remain data-intensive and often lack explicit geometric memory and awareness of what the camera can observe. We propose ScanDreamer, a map-grounded world model that constructs an online 2D scan-memory map from depth images and vehicle pose. Our world model, actor, and critic are conditioned on this map, which also defines a geodesic progress reward. Moreover, a forward-cone penalty discourages horizontal motion outside the camera's viewing direction. During imagination, ScanDreamer propagates the quadrotor pose over a frozen map and derives progress and wall-contact signals from the mapped geometry. We train ScanDreamer for 0.8 million environment steps (5.5 h on one GPU) and evaluate it in NVIDIA Isaac Sim over 1560 episodes spanning unseen layouts and parameter ranges. Across three seeds (mean SD), ScanDreamer achieves 96.3 0.3% success with 3.6 0.3% collision, compared with 59.6 3.0% success and 28.8 2.8% collision for AirDreamer at the same online-interaction budget. Removing the forward-cone penalty increases collisions to 9.9 3.1%. These results show that combining explicit map memory, map-grounded imagination, and perception-aware motion substantially improves navigation performance and data efficiency. The source code is available at https://github.com/WANGSHUAIzzzz/ScanDreamer.
Figures & tables
| Training family | Scenes | Key parameters (training intervals) |
|---|---|---|
| Pillar forest | 1,440 | density 0.02–0.10 m -2 , gap 1.0–2.0 m |
| Wall field | 1,440 | 1–8 walls of 3–10 m, slits 1.2–1.6 m |
| Traps | 2,880 | 1–3 U, C, V, or L traps, depth 2.0–4.0 m |
| Combination | 4,800 | pillar forest, 1–2 traps, 0–3 walls |
| Corridor tree | 6,240 | 2–3 branches, arms 3–9 m, width 2.1–2.5 m |
| Grid maze | 2,880 | 5 5 to 7 7 cells of 2.6–3.5 m |
| SR per suite (%) | All suites pooled | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Method | Steps | Seeds | In-dist. | Param. | Topo. | Struct. | Open | SR | CR | TO | SPL |
| NavRL [ 6 ] , released | – | – | 28.5 | 27.9 | 8.6 | 47.3 | 100.0 | 33.8 | 11.3 | 54.5 | 0.324 |
| NavRL, retrained | 0.8M | 1 | 5.3 | 4.3 | 0.5 | 12.3 | 45.0 | 8.4 | 37.1 | 54.5 | 0.077 |
| NavRL, retrained | 2.0M | 1 | 15.1 | 12.1 | 0.5 | 5.7 | 88.0 | 15.6 | 16.7 | 67.6 | 0.144 |
| TD-MPC2 [ 15 ] | 0.8M | 1 | 7.3 | 3.6 | 2.3 | 15.3 | 55.0 | 10.8 | 52.1 | 37.0 | 0.096 |
| TD-MPC2 | 1.0M | 1 | 5.6 | 3.6 | 2.7 | 11.7 | 48.0 | 8.9 | 51.9 | 39.0 | 0.076 |
| Variant | Map | Geo. | Cone | Imag. | Arr. | SR | CR | TO | SPL | Fwd. (%) | Off-axis (%) | Appr. (s) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Ours | ✓ | ✓ | ✓ | ✓ | ✓ | 96.3 0.3 | 3.6 0.3 | 0.1 0.0 | 0.831 0.008 | 92.4 0.9 | 35.8 10.8 | 0.38 0.01 |
| w/o forward cone | ✓ | ✓ | ✓ | ✓ | 89.9 3.1 | 9.9 3.1 | 0.1 0.1 | 0.770 0.021 | 40.1 7.0 | 77.0 3.2 | 0.40 0.04 | |
| w/o arrival handling | ✓ | ✓ | ✓ | ✓ | 96.2 1.6 | 3.4 1.6 | 0.3 0.1 | 0.815 0.027 | 86.4 2.3 | 40.2 9.2 | 1.39 0.30 | |
| w/o cone and arrival | ✓ | ✓ | ✓ | 90.3 3.3 | 9.2 3.1 | 0.5 0.2 | 0.757 0.029 | 36.4 7.4 | 74.2 13.6 | 1.02 0.27 | ||
| Geodesic reward only | ✓ | 51.6 2.5 | 32.5 8.8 | 14.4 9.7 | 0.431 0.034 | 48.7 3.5 | 83.6 4.1 | 0.99 0.38 |
World Models for Robotics
World models for robotics predict how scenes and robot states change under actions to support planning, control, and learning.
See all 420 papers