World SLAM Model: Joint World Modeling for SLAM and Navigation

Minghui Qin; Yijun Yuan; Weicheng Zheng; Kenan Li; Weibang Wang; Chang Sun; Junhao Huang; Anmin Liu

cs.ROSep 26, 2026

World SLAM Model: Joint World Modeling for SLAM and Navigation

Authors: Minghui Qin, Yijun Yuan, Weicheng Zheng, Kenan Li, Weibang Wang, Chang Sun, Junhao Huang, Anmin Liu,

Organizations: Shanghai Qi Zhi Institute · Shanghai Jiao Tong University · IIIS, Tsinghua University

Abstract

We introduce World SLAM Model (WSM), a unified framework that brings the SLAM paradigm directly into downstream navigation. Rather than treating SLAM merely as an upstream module that provides poses, maps or tokens, WSM adopts its core mechanisms, including incremental state updates with persistent memory and backend refinement of accumulated errors, to maintain a consistent world state during interaction. Given the current observation and a navigation goal, WSM predicts future visual states and jointly estimates their camera motion and dense geometry, grounding visual prediction in an evolving spatial world state. This spatial state is continuously updated as new observations arrive and provides the basis for action generation and closed-loop navigation. WSM is trained end-to-end with a joint navigation--SLAM objective, enabling downstream navigation to benefit directly from SLAM-style state maintenance and refinement while preserving accurate geometric estimation. Experiments demonstrate improved navigation performance together with strong SLAM accuracy, highlighting the potential of SLAM as an intrinsic mechanism for long-horizon world modeling and embodied interaction.

Figures & tables

Figure 1: Overview of the World SLAM Model. WSM integrates a SLAM expert and a generation expert into a unified model that autoregressively estimates world states (i.e., depth and camera pose), dreams future observations, and infers future world states, supporting navigation, localization, and mapping.

Figure 2: Framework overview. Our model unifies visual dreaming and SLAM within an autoregressive Mixture-of-Transformers (MoT) for start–goal navigation. Conditioned on the start–goal and the initial observation $\mathbf{o}_{0}$ , the generation expert first dreams future RGB observations $\hat{\mathbf{o}}_{1:K}$ , from which the SLAM expert recovers future world states $\hat{\mathbf{s}}_{1:K}$ as an inverse dynamics model . After execution, the SLAM expert estimates real world states $\mathbf{s}_{1:K}$ from the received observations $\mathbf{o}_{1:K}$ , which are cached to condition the next dream $\hat{\mathbf{o}}_{K+1:2K}$ .

Figure 3: Training sequence and attention mask. (a) Each chunk

n

consists of clean tokens

\mathbf{O}_{n},\mathbf{S}_{n}

and prediction tokens

\mathbf{O}^{\tau}_{n},\bar{\mathbf{S}}_{n}

, and all chunks of an episode are processed by the Mixture-of-Transformers in a single forward pass. (b) Attention mask. Temporal order is preserved across chunks, and observation and state tokens in the history context are invisible to each other. No prediction token can attend to its own target.

Method	Input		Home		Commercial
Method	RGB	Depth	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$
DD-PPO ( Wijmans et al., 2019 )	$\checkmark$	$\checkmark$	0.4	0.4	5.3	5.2
iPlanner ( Yang et al., 2023 )	–	$\checkmark$	43.0	40.6	54.6	52.8
ViPlanner ( Roth et al., 2024 )	$\checkmark$	$\checkmark$	45.0	43.2	63.7	61.9
LoGoPlanner ( Peng et al., 2025 )	$\checkmark$	$\checkmark$	57.3	52.4	67.1	63.9
LoGoPlanner † ( Peng et al., 2025 )	$\checkmark$	$\checkmark$	59.6	53.6	65.8	62.0

Table 1: Navigation results on the InternVLA-N1 start–goal navigation benchmark. SR and SPL are reported in percent. Checkmarks indicate the input modalities. Baseline results are taken from LoGoPlanner. † means our re-evaluation with its latest official code.

Figure 4: Visualization of navigation results in simulation. (a) Navigation result; Q1–Q3 mark the queries shown in (b) and (d). (b) RGB observation at time $t$ and the frames dreamed by WSM for $t{+}1$ to $t{+}4$ at each query. (c) Camera trajectory estimated by the SLAM expert vs. ground truth. (d) Ground-truth and predicted depth at time $t$ for each query, colored from 0.2 to 5 m.

Method	ATE RMSE (m) $\downarrow$	Acc. RMSE (m) $\downarrow$
SLAM-Former † ( Yuan et al., 2026b )	0.051	0.057
SLAM-Former ‡ ( Yuan et al., 2026b )	0.041	0.041
World SLAM Model	0.021	0.062

Table 2: SLAM evaluation on the 5% InternVLN-N1 test split. † denotes the released checkpoint; ‡ denotes the model fine-tuned on the 95% InternVLN-N1 training split.

Method	Home		Commercial
Method	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$
IDM (w/o SLAM)	28.1	27.1	32.1	31.2
World SLAM Model (w/o post-training)	64.6	61.5	68.5	66.9

Table 3: SLAM IDM ablation. Both models are evaluated after pretraining, without post-training. SR and SPL are reported in percent.

Method	Home		Commercial
Method	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$
w/o local backend	68.3	65.2	72.6	70.5
w/ local backend	70.3	66.3	73.9	71.8

Table 4: Local backend ablation. Both variants are post-trained from the same pretrained model and differ only in whether the local backend is used in history rollout, during both post-training and closed-loop evaluation. SR and SPL are reported in percent.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Initialization	Wan2.2-5B
Input resolution	$320\times 192$
Chunk size (latent frames)	uniform in $\{1,\dots,4\}$
Timestep shift $\gamma$	5
$\lambda_{d}$ / $\lambda_{p}$ / $\delta$	1 / 0.5 / 0.1
Optimizer	AdamW ( $\beta_{1}=0.9$ , $\beta_{2}=0.95$ , $\epsilon=10^{-8}$ )

Appendix

Table 5: Pretraining hyperparameters.

Hyperparameter	Value
Initialization	pretrained model
Trainable parameters	self-attention LoRA of the generation expert
LoRA rank / $\alpha$	32 / 64
Group size $G$	16
Dreamed latent frames $K$	4
Sampling steps	8

Appendix

Table 6: Post-training hyperparameters.

Figure 5: Two-stage ablation. SR and SPL are reported in percent.

Figure 6: Navigation result in Home scene 15, episode 27.

Figure 7: Navigation result in Home scene 15, episode 79.

Figure 8: Navigation result in Home scene 16, episode 94.

Figure 9: Navigation result in Home scene 16, episode 96.

World Models for Robotics

World models for robotics predict how scenes and robot states change under actions to support planning, control, and learning.

See all 386 papers

J-LAW: Joint Localization and Action-Conditioned World Modeling via Coupled Latent Factor Graphs
Jun 27, 2026Guanqun Cao, Liang ChenSimultaneous Localization and Mapping Factor Graph Optimization
Geely Technology Europe, Pumpgatan 1, 417 55 Gothenburg Sweden · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, China
Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigation
Jun 29, 2026Hong Chen, Daqi Liu, Zehan Zhang +10Visual Navigation World Model-Based Planning
Huazhong University of Science and Technology, Wuhan, China · Xiaomi EV, Beijing, China · Zhejiang University, Hangzhou, China
G $^2$ WAM: World Action Models for Efficient and Geometry Grounded Image-Goal Navigation
Oct 12, 2026Ke Gui, Chenghao Shi, Ziyue Wang +3Scene Coordinate Regression Image-Goal Navigation
College of Intelligence Science and Technology, National University of Defense Technology, China · National Key Laboratory of Equipment State Sensing and Smart Support, National University of Defense Technology, China · School of Future Technology, National University of Defense Technology, China

Jun 27, 2026J/K move · Enter open · S save

J-LAW: Joint Localization and Action-Conditioned World Modeling via Coupled Latent Factor Graphs

Guanqun Cao, Liang Chen

Geely Technology Europe, Pumpgatan 1, 417 55 Gothenburg Sweden · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, China

Simultaneous Localization and Mapping Factor Graph Optimization World Model Learning World Models Action-Conditioned World Models

Open paper AskPDF arXiv

Classical simultaneous localization and mapping (SLAM) estimates metric poses and a geometric map but does not provide an action-conditioned predictive state. Action-conditioned world models learn compact latent dynamics but ignore global metric consistency and accumulate drift under open-loop rollout. We introduce J-LAW (Joint Localization and Action-Conditioned World Modeling), a unified factor-graph formulation that connects metric pose variables, predictive latent states, and persistent latent landmarks in this letter.J-LAW represents each image as a compact predictive state and combines it with pose or motion measurements through a separately learned mapping. Its maximum a posteriori (MAP) factor graph enforces consistency between these complementary sources of information over time. Experiments on PushT and WildGS show that J-LAW's factor-graph representation can improve long-horizon latent consistency and recover more reliable predictive states under partial observations, forming a foundation for future integrated localization and planning systems.

Abstract

World Models for Robotics

Explore similar work

J-LAW: Joint Localization and Action-Conditioned World Modeling via Coupled Latent Factor Graphs

Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigation

G2^22WAM: World Action Models for Efficient and Geometry Grounded Image-Goal Navigation

G $^2$ WAM: World Action Models for Efficient and Geometry Grounded Image-Goal Navigation