Map2SLAM: From Streaming Map Prediction to Real-Time SLAM
Organizations: Nanjing University · Robbyant, Ant Group · HKUST · Wuhan University ·
Abstract
Streaming 3D foundation models (3DFMs) estimate camera poses and dense geometry online, but they fall short of SLAM: their predictions are fixed once emitted, so loop closures cannot correct accumulated drift. With such models already providing strong geometric capability, a central question for SLAM becomes how to schedule their inference and how to keep its results as revisable state. We present Map2SLAM, which keeps LingBot-Map frozen, trains only a lightweight retrieval head, and builds SLAM on the model's inference interface, turning its predictions into observations of an external persistent SLAM state. The front-end tracks every incoming frame with the model's causal streaming inference and decides which frames are written to the persistent state as keyframes. The back-end asynchronously decides when and which past frames to re-observe, re-running the model's joint inference over them with fresh context; verified loop observations correct the trajectory online, and segment observations refine dense geometry after the sequence. All invocations share a single frame encoder pass per image, whose cached features also drive loop retrieval, so frequent re-invocation incurs no additional encoding cost. Map2SLAM achieves state-of-the-art performance on KITTI, VBR, and Oxford Spires, reducing mean ATE by over 40% compared with the best uncalibrated baselines, while maintaining real-time tracking at over 28 FPS at a resolution of on an NVIDIA RTX 5090D GPU. Please visit our project page at https://robbyant-research.github.io/Map2SLAM/.
Figures & tables
| Method | LC | Dense Geo. | Frame- wise | 00 [0.45ex] 4541 f [0.10ex] 3.70 km | 01 [0.45ex] 1101 f [0.10ex] 2.50 km | 02 [0.45ex] 4661 f [0.10ex] 5.10 km | 03 [0.45ex] 801 f [0.10ex] 0.60 km | 04 [0.45ex] 271 f [0.10ex] 0.40 km | 05 [0.45ex] 2761 f [0.10ex] 2.20 km | 06 [0.45ex] 1101 f [0.10ex] 1.20 km | 07 [0.45ex] 1101 f [0.10ex] 0.70 km | 08 [0.45ex] 4071 f [0.10ex] 3.20 km | 09 [0.45ex] 1591 f [0.10ex] 1.70 km | 10 [0.45ex] 1201 f [0.10ex] 0.90 km | Avg. |
| Calibrated | |||||||||||||||
| ORB-SLAM3 ( Campos et al., 2021 ) | 7.77 | – | 25.18 | 0.85 | 0.72 | 6.03 | 16.93 | 2.57 | 64.55 | 6.76 | 8.48 | – | |||
| DROID-SLAM ( Teed & Deng, 2021 ) | 110.19 | 90.94 | 138.72 | 3.20 | 1.47 | 73.64 | 61.50 | 16.09 | 103.85 | 92.01 | 22.21 | 64.89 | |||
| DPV-SLAM++ ( Lipson et al., 2024 ) | 7.87 | 27.48 | 71.75 | 2.56 | 1.03 | 5.92 | 12.49 | 18.00 | 113.94 | 74.54 | 13.65 | 31.75 | |||
| Uncalibrated | |||||||||||||||
| DROID-Calib ( Hagemann et al., 2023 ) | 47.65 | 34.21 | 93.95 | 5.78 | 2.39 | 39.74 | 11.29 | 18.62 | 76.32 | 68.26 | 8.79 | 37.00 | |||
| Method | LC | Dense Geo. | Frame- wise | Ag Colosseo-0 [0.45ex] 8815 f [0.10ex] 1.45 km | Ag Campus-0 [0.45ex] 12042 f [0.10ex] 2.73 km | Ag Campus-1 [0.45ex] 11671 f [0.10ex] 2.95 km | Ag Pincio-0 [0.45ex] 11142 f [0.10ex] 1.27 km | Ag Spagna-0 [0.45ex] 14141 f [0.10ex] 1.56 km | Ag Diag-0 [0.45ex] 10021 f [0.10ex] 1.02 km | Ag Ciampino-1 [0.45ex] 18846 f [0.10ex] 5.20 km | Avg. |
| Calibrated | |||||||||||
| DROID-SLAM ( Teed & Deng, 2021 ) | 76.66 | 125.77 | 83.87 | 49.85 | 59.61 | 35.81 | 145.28 | 82.41 | |||
| DPV-SLAM++ ( Lipson et al., 2024 ) | 18.59 | 21.69 | 22.22 | 39.37 | 37.31 | 33.51 | 45.56 | 31.18 | |||
| Uncalibrated | |||||||||||
| DROID-Calib ( Hagemann et al., 2023 ) | 27.36 | 76.05 | 68.52 | 14.09 | 25.38 | – | 124.98 | – | |||
| VGGT-Long ( Deng et al., 2026 ) | 44.13 | 132.70 | 115.62 | 64.29 | 59.05 | 33.66 | 179.41 | 89.84 | |||
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | LC | Dense Geo. | Frame- wise | KC-02 [0.45ex] 5969 f [0.10ex] 0.29 km | KC-03 [0.45ex] 5682 f [0.10ex] 0.28 km | KC-04 [0.45ex] 13518 f [0.10ex] 0.78 km | KC-05 [0.45ex] 11544 f [0.10ex] 0.71 km | OQ-01 [0.45ex] 5746 f [0.10ex] 0.40 km | OQ-02 [0.45ex] 5469 f [0.10ex] 0.39 km | CC-02 [0.45ex] 11775 f [0.10ex] 0.64 km | CC-03 [0.45ex] 6204 f [0.10ex] 0.27 km | CC-05 [0.45ex] 3961 f [0.10ex] 0.81 km | BL-02 [0.45ex] 4966 f [0.10ex] 0.69 km | Avg. |
| Calibrated | ||||||||||||||
| DROID-SLAM ( Teed & Deng, 2021 ) | 7.89 | 31.50 | 3.93 | 36.44 | 0.43 | 0.29 | 4.09 | 0.86 | 8.96 | 1.20 | 9.56 | |||
| DPV-SLAM++ ( Lipson et al., 2024 ) | 2.62 | 0.41 | 1.29 | 0.53 | 0.74 | 0.50 | 1.77 | 0.08 | 11.93 | 2.91 | 2.28 | |||
| Uncalibrated | ||||||||||||||
| DROID-Calib ( Hagemann et al., 2023 ) | 11.89 | 4.82 | 1.83 | 37.02 | 1.01 | 3.24 | 9.92 | 0.63 | 2.63 | 1.36 | 7.43 | |||
| VGGT-Long ( Deng et al., 2026 ) | 1.76 | 14.74 | 32.53 | 4.59 | 3.70 | 5.66 | 11.65 | 4.00 | 13.51 | 16.16 | 10.83 | |||
| Method | Dim. | Params (M) | Latency (ms) | MSLS-val | Pitts250k-test | SPED |
| SFRS ( Ge et al., 2020 ) | 4096 | 149.0 | 2.78 | 64.2/74.6/77.8 | 89.5/95.3/96.5 | 73.0/85.7/87.8 |
| GeM ( Radenović et al., 2018 ) | 2048 | 23.5 | 3.65 | 78.9/87.6/90.1 | 86.5/94.6/96.4 | 66.9/79.4/85.2 |
| NetVLAD ( Arandjelovic et al., 2016 ) | 32768 | 24.6 | 4.03 | 83.1/90.7/92.7 | 92.6/97.5/98.3 | 79.2/89.3/91.9 |
| Conv-AP ( Ali-Bey et al., 2022 ) | 4096 | 25.6 | 2.92 | 83.0/90.1/92.6 | 92.6/97.7/98.6 | 81.1/89.6/93.3 |
| CosPlace ( Berton et al., 2022 ) | 2048 | 27.7 | 3.08 | 87.6/92.7/94.7 | 92.4/97.5/98.4 | 79.6/89.0/92.6 |
| MixVPR ( Ali-Bey et al., 2023 ) | 4096 | 10.9 | 2.48 | 88.1/93.2/94.5 | 94.3/98.2/98.9 | 84.7/92.3/94.4 |
Simultaneous Localization and Mapping
Simultaneous localization and mapping estimates a robot’s motion while building a map of an unknown environment.
See all 210 papers