cs.ROFeb 25, 2026

Dream-SLAM: Dreaming the Unseen for Active SLAM in Dynamic Environments

Authors: Xiangqi Meng, Pengxu Hou, Zhenjun Zhao, Javier Civera, Daniel Cremers, Hesheng Wang, Haoang Li

Organizations: Thrust of Robotics and Autonomous Systems, Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · University of Zaragoza, Zaragoza, Spain. · School of Computation, Information and Technology, Technical University of Munich, Munich, Germany. · School of Automation and IntelligentSensing, Shanghai Jiao Tong University, Shanghai 200240, China.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In addition to the core tasks of simultaneous localization and mapping (SLAM), active SLAM additionally in- volves generating robot actions that enable effective and efficient exploration of unknown environments. However, existing active SLAM pipelines are limited by three main factors. First, they inherit the restrictions of the underlying SLAM modules that they may be using. Second, their motion planning strategies are typically shortsighted and lack long-term vision. Third, most approaches struggle to handle dynamic scenes. To address these limitations, we propose a novel monocular active SLAM method, Dream-SLAM, which is based on dreaming cross-spatio-temporal images and semantically plausible structures of partially observed dynamic environments. The generated cross-spatio-temporal im- ages are fused with real observations to mitigate noise and data incompleteness, leading to more accurate camera pose estimation and a more coherent 3D scene representation. Furthermore, we integrate dreamed and observed scene structures to enable long- horizon planning, producing farsighted trajectories that promote efficient and thorough exploration. Extensive experiments on both public and self-collected datasets demonstrate that Dream-SLAM outperforms state-of-the-art methods in localization accuracy, mapping quality, and exploration efficiency. Source code will be publicly available upon paper acceptance.

Figures & tables

Fig. 1: Dream-SLAM overview. Our pipeline consists of two main modules: localization and mapping, and exploration planning. (a) For localization, we propose to dream cross-spatio-temporal images, and use these images to construct additional 3D-2D foreground constraints that can effectively compensate for noise. For mapping, we propose a feedforward network to reconstruct per-pixel Gaussians of both static background and dynamic foreground. We further refine Gaussians based on multi-view constraints provided by cross-spatio-temporal and real images. (b) Our planning module dreams semantically plausible structures of unobserved areas. By integrating the dreamed and observed information, we plan a farsighted path, enabling an efficient and thorough exploration.
Fig. 2: Cross-spatio-temporal images for camera localization. (a) Traditional localization methods rely solely on the static background to estimate the camera pose. (b) In contrast, our method leverages both the dynamic foreground and static background by aligning the Gaussians’ rendering at time tt with the dreamed cross-spatio-temporal image It+1tI^{t}_{t+1} , which represents the scene at time tt from the viewpoint of the (t+1)(t+1) -th camera.
Fig. 3: Dreaming a cross-spatio-temporal image. Given the image It+1I_{t+1} , we segment the foreground and dilate the foreground mask to obtain the inpainting mask M\mathbf{M} . Then we feed the images ItI_{t} and It+1I_{t+1} , together with the mask M\mathbf{M} , into the diffusion model, which dreams the cross-spatio-temporal image It+1tI^{t}_{t+1} .
Fig. 4: 3D Gaussian prediction and refinement. (a) Given images It+1I_{t+1} and ItI_{t} , we design a feedforward network to predict dynamic Gaussians at both time t+1t+1 and time tt . Here, we only visualize the predicted Gaussians at time t+1t+1 . (b) We refine Gaussians at time t+1t+1 based on the photometric loss regarding both dreamed cross-spatio-temporal images It−1t+1,Itt+1I_{t-1}^{t+1},I_{t}^{t+1} and real image It+1I_{t+1} . These images depict the same scene content at time t+1t+1 from the (t−1)(t-1) -th, tt -th, and (t+1)(t+1) -th views, respectively.
Fig. 5: Dreaming semantically plausible structures of unexplored areas. At an unvisited waypoint, we place virtual cameras to render images from different views and select the suitable images. Then we inpaint the selected images, and use them to predict Gaussians. By integrating the dreamed Gaussians into the existing Gaussians, we obtain more complete structures of the environment.
Fig. 6: Comparison between global ordering strategies without and with the capability of structure dreaming. In this schematic diagram, the number below each image represents the accumulated length of the traveled path. They do not have a specific unit, but show relative magnitudes of values. (a) Strategies without dreaming capability result in suboptimal trajectories with unnecessary detours. (b) Our method leverages both observed and dreamed structures to plan a farsighted path.
TUM dataset [ 61 ] Sequences f3/wk_xyz f3/wk_hf f3/wk_st f3/st_hf f3/st_rpy f3/st_st f3/st_xyz Average RMSE ↓\downarrow SD ↓\downarrow RMSE ↓\downarrow SD ↓\downarrow RMSE ↓\downarrow SD ↓\downarrow RMSE ↓\downarrow SD ↓\downarrow RMSE ↓\downarrow SD ↓\downarrow RMSE ↓\downarrow SD ↓\downarrow RMSE ↓\downarrow SD ↓\downarrow RMSE ↓\downarrow SD ↓\downarrow ORB-SLAM3 [ 62 ] 28.1 12.2 30.5 9.0 2.0 1.1 2.6 1.6 6.4 2.5 0.98 0.46 1.6 0.7 10.31 3.94 MonST3R [ 35 ] 26.7 13.2 44.6 23.9 1.7 0.8 38.7 20.2 5.5 2.6 2.2 1.2 30.2 13.2 21.37 10.73 RoDyn-SLAM [ 17 ] 8.3 5.5 5.6 2.8 1.7 0.9 4.4 2.2 11.4 4.6 0.76 0.43 5.0 1.0 5.31 2.49 PG-SLAM [ 18 ] 6.8 2.9 11.7 4.4 1.4 0.6 4.0 1.5 5.4 2.4 0.72 0.39 1.5 0.5 4.50 1.81 WildGS-SLAM [ 21 ] 1.3 0.6 1.6 0.8 0.4 0.2 2.0 0.9 2.4 1.4 0.5 0.3 0.8 0.4 1.28 0.65 Dream-SLAM (ours) 1.7 0.7 1.6 0.7 0.3 0.1 1.9 0.9 2.3 0.7 0.3 0.1 0.6 0.2 1.27 0.48
Bonn dataset [ 30 ] Sequences balloon balloon2 ps_track ps_track2 mv_box mv_box2 Average RMSE ↓\downarrow SD ↓\downarrow RMSE ↓\downarrow SD ↓\downarrow RMSE ↓\downarrow SD ↓\downarrow RMSE ↓\downarrow SD ↓\downarrow RMSE ↓\downarrow SD ↓\downarrow RMSE ↓\downarrow SD ↓\downarrow RMSE ↓\downarrow SD ↓\downarrow ORB-SLAM3 [ 62 ] 6.5 2.9 17.7 8.6 70.7 32.6 77.9 43.8 28.0 8.4 3.5 1.5 34.05 16.30 MonST3R [ 35 ] 11.1 8.5 14.3 6.3 25.4 15.8 22.5 9.4 7.6 3.5 13.1 5.3 15.67 8.13 RoDyn-SLAM [ 17 ] 7.9 2.7 11.5 6.1 14.5 4.6 13.8 3.5 7.2 2.4 12.6 4.7 11.25 4.00 PG-SLAM [ 18 ] 6.4 2.2 7.3 3.4 5.0 1.9 8.5 2.8 4.6 1.3 7.0 2.0 6.47 2.27 WildGS-SLAM [ 21 ] 2.8 1.2 2.4 1.1 3.1 2.0 3.0 1.3 1.6 0.8 2.2 1.2 2.52 1.27 Dream-SLAM (ours) 1.9 0.7 1.9 0.4 1.5 0.5 2.7 0.8 0.6 0.2 1.4 0.5 1.67 0.52
TABLE I: Camera localization comparisons on the TUM [ 61 ] and Bonn [ 30 ] datasets.
Fig. 7: Qualitative comparison of dynamic scene mapping methods on Sequence wk_xyz in the TUM dataset [ 61 ] . We render the mapped scenes from novel views. (a) MonST3R [ 35 ] does not support a photo-realistic rendering due to the point cloud representation. (b) WildGS-SLAM [ 21 ] fails to map the dynamic foreground. (c) PG-SLAM [ 18 ] can reconstruct both static background and dynamic foreground, but some rendering regions remain low-quality. (d) Our Dream-SLAM can map both background and foreground, and also achieves the highest mapping quality.
  Bonn dataset [ 30 ]TUM dataset [ 61 ]
  Seq. ps_track2Seq. ps_trackSeq. mv_box2Seq. wk_stSeq. st_hfSeq. wk_xyz
  w/oww/oww/oww/oww/oww/ow
RoDyn-SLAM [ 17 ]PSNR ↑\uparrow18.4616.1218.5316.1318.3017.4811.4811.4011.4711.1111.9311.91
SSIM ↑\uparrow0.7450.6560.7420.6590.7310.6950.6840.5220.5140.6740.5670.384
LPIPS ↓\downarrow0.5450.6490.5060.5970.5710.6120.5020.6630.4720.5110.5700.745
WildGS-SLAM [ 21 ]PSNR ↑\uparrow21.8718.1222,1318.5521.4020.1321.6016.6119.4917.9417.1114.44
TABLE II: Rendering quality comparison on the Bonn [ 30 ] and TUM [ 61 ] datasets. “w/o” and “w” represent rendering evaluations without and with considering the dynamic foregrounds, respectively.
 Time Cost Per Frame
WildGS-SLAM [ 21 ]2.79 s
PG-SLAM [ 18 ]1.93 s
Rodyn-SLAM [ 17 ]1.42 s
Dream-SLAM (ours)0.65 s
TABLE III: Efficiency comparison in terms of localization and mapping on Sequence mv_box in the Bonn dataset [ 30 ] .
Gibson dataset [ 67 ] Sequences CanWell Eastville Swormville Aloha Nicut Quantico Average CR ∗ ↑ CR↑ PL↓ CR ∗ ↑ CR↑ PL↓ CR ∗ ↑ CR↑ PL↓ CR ∗ ↑ CR↑ PL↓ CR ∗ ↑ CR↑ PL↓ CR ∗ ↑ CR↑ PL↓ CR ∗ ↑ CR↑ PL↓ ANM [ 7 ] - - - 47.7 85.0 102.8 44.5 60.5 70.9 32.8 82.5 90.8 46.7 77.4 85.7 53.0 82.1 90.6 44.9 77.5 88.2 ANM-S [ 60 ] 70.3 97.0 91.8 64.4 91.5 70.6 74.8 93.6 95.2 63.6 95.9 75.8 72.4 97.3 83.0 63.5 95.4 79.2 68.1 95.1 82.6 ActiveSplat [ 6 ] 64.8 97.1 116.2 71.9 94.7 84.8 74.7 95.3 91.9 71.3 91.8 82.5 88.9 97.8 94.1 43.6 95.1 93.7 69.2 95.3 93.9 Dream-SLAM (ours) 98.0 98.1 87.6 95.4 95.3 57.1 95.9 95.9 51.6 98.2 98.1 63.1 98.0 98.1 64.4 95.6 95.6 62.0 96.8 96.9 64.3
HM3D dataset [ 68 ] Sequences CETmJJqkhcK 7dmR22gwQpH 7UdY7HiDnUi 4h4JxvG3cip 6HMiy15cxis T7nCRmufFNR Average CR ∗ ↑ CR↑ PL↓ CR ∗ ↑ CR↑ PL↓ CR ∗ ↑ CR↑ PL↓ CR ∗ ↑ CR↑ PL↓ CR ∗ ↑ CR↑ PL↓ CR ∗ ↑ CR↑ PL↓ CR ∗ ↑ CR↑ PL↓ ANM [ 7 ] 28.3 64.9 112.7 - - - 40.3 42.4 92.6 36.8 56.0 142.3 39.5 61.6 144.6 49.5 68.9 107.9 38.9 58.7 120.0 ANM-S [ 60 ] 33.0 92.1 97.4 36.7 90.2 159.2 24.1 91.9 132.1 49.3 93.1 99.5 58.2 95.1 108.6 86.9 95.4 132.2 48.0 92.9 121.5 ActiveSplat [ 6 ] 30.6 92.9 102.9 81.3 94.1 180.0 63.7 95.2 99.8 54.3 92.2 118.8 76.5 95.4 114.6 82.9 94.8 105.37 64.9 94.2 125.8 Dream-SLAM (ours) 93.2 93.2 84.7 95.0 95.2 154.7 95.9 96.0 75.8 95.3 95.1 89.6 95.7 95.7 85.9 96.9 96.8 75.8 95.3 95.3 99.4
TABLE IV: Exploration planning comparison on Gibson [ 67 ] and HM3D [ 68 ] datasets. “-” represents the failure of the full exploration.
Fig. 8: Comparison between ActiveSplat [ 6 ] and our Dream-SLAM in terms of exploration progress on Sequence T7nCRmufFNR of the HM3D dataset [ 68 ] . We provide a top-view visualization of the mapped scenes at some representative timestamps. A triplet of numbers below each image indicates CR, the length of the traversed path, and the proportion of the traversed path to the total path. The trajectory color reflects the time cost of exploration.
Fig. 9: Comparison between ActiveSplat [ 6 ] and our Dream-SLAM in terms of exploration progress on Sequence Cantwell of the Gibson dataset [ 67 ] . We provide a top-view visualization of the mapped scenes at some representative timestamps. A triplet of numbers below each image indicates CR, the length of the traversed path, and the proportion of the traversed path to the total path. The trajectory color reflects the time cost of exploration.
SequenceWithout DreamingDreaming (ours)
 RMSE ↓\downarrowSD ↓\downarrowRMSE ↓\downarrowSD ↓\downarrow
wk_hf of TUM [ 61 ]1.80.71.60.7
ps_track of Bonn [ 30 ]1.70.71.50.5
TABLE V: Ablation study of localization using the dreamed cross-spatio-temporal images.
Fig. 10: Ablation study of mapping using the dreamed cross-spatio-temporal images. We compare the rendering results of Gaussian maps obtained without and with the dreamed images on (a) Sequence ps_track of the Bonn dataset [ 30 ] , and (b) Sequence wk_hf of the TUM dataset [ 61 ] .
SequenceWithout DreamingDreaming (ours)
 CR↑PL↓CR↑PL↓
Cantwell of Gibson [ 67 ]97.5101.398.188.4
T7nCRmufFNR of HM3D [ 68 ]95.585.196.875.8
TABLE VI: Ablation study of planning using the dreamed structures of unexplored areas.
Fig. 11: Setup of experiments on our self-collected data. (a) Mobile robot to collect data. It is built on a four-wheeled Mecanum chassis and equipped with a front-facing camera and a LiDAR. (b) Sample images of our self-collected data in a home environment containing dynamic humans.
Fig. 12: Comparison between ActiveSplat* [ 21 , 6 ] and our Dream-SLAM in terms of exploration progress on our self-collected data. We provide a top-view visualization of the mapped scenes at some representative timestamps. A pair of numbers below each image indicates the length of the traversed path, and the proportion of the traversed path to the total path. The trajectory color reflects the time cost of exploration.
Fig. 13: Dynamic scene mapping of our Dream-SLAM on our self-collected data. We render the reconstructed map composed of both static background and dynamic foreground from a novel view. The red and green dotted lines denote the trajectories of human and camera, respectively.
WildGS-SLAM [ 21 ]Dream-SLAM (ours)
RMSE ↓\downarrowSD ↓\downarrowRMSE ↓\downarrowSD ↓\downarrow
16.67.310.85.6
TABLE VII: Camera localization comparison between WildGS-SLAM [ 21 ] and our Dream-SLAM on our self-collected data.
More in this topic

Simultaneous Localization and Mapping

Simultaneous localization and mapping estimates a robot’s motion while building a map of an unknown environment.

See all 197 papers

Explore similar work

CardsList

  1. CAOS-SLAM: Class-Agnostic Object-Aware Real-Time vSLAM for Dynamic Environments

    Oct 19, 2026Gaia Borghi, João Carlos Virgolino Soares, Angelo Bratta +2Visual SLAM

    Dynamic Legged Systems Lab (DLS), Istituto Italiano di Tecnologia, Genoa, Italy · Section of Automation & Control, Department of Electronic Systems, Aalborg University, Denmark

  2. DROID-SLAM in the Wild

    Mar 19, 2026Moyang Li, Zihan Zhu, Marc Pollefeys +1Dynamic SLAMVisual SLAM

    ETH Zurich · Microsoft

  3. RoSe-SLAM: Robust Semantic-Aware Gaussian Splatting SLAM from Dynamic Monocular Videos

    Aug 29, 2026Wenting Wang, Jiaxin Guo, Wenzhen Dong +3Dynamic SLAMVisual SLAM

    Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong SAR. · Department of Mechanical, Aerospace and Civil Engineering, The University of Manchester, Manchester, UK. · Centre for Perceptual and Interactive Intelligence (CPII) Limited, Hong Kong SAR.