SkipWorld: Latent Skip Planning with Distilled World Models for Efficient Visual Navigation

Yifei Yang; Zehua Fan; Zike Yan; Zhongyu Li; Yan Wang

cs.ROOct 13, 2026

SkipWorld: Latent Skip Planning with Distilled World Models for Efficient Visual Navigation

Authors: Yifei Yang, Zehua Fan, Zike Yan, Zhongyu Li, Yan Wang

Organizations: Institute for AI Industry Research (AIR), Tsinghua University · Hong Kong Embodied AI Lab · The Chinese University of Hong Kong · Shanghai Jiao Tong University

Abstract

World-model-based visual navigation enables action planning by predicting future states under candidate trajectories. However, long-horizon rollout remains challenging: models are often trained with ground-truth visual histories but deployed in closed-loop prediction, and evaluating many candidates with step-by-step rollout is computationally expensive. We propose SkipWorld, a latent skip planning framework for efficient visual navigation. SkipWorld introduces three components: (i) Latent AutoRoll, which trains a world model to recursively roll out future states in latent space, reducing the mismatch between training and closed-loop inference; (ii) SkipDistill, which distills the latent autoregressive model into a skip world model that directly predicts long-stride future latent states; and (iii) Skip-then-Refine Planning, a dual-model coarse-to-fine planner that first filters candidate trajectories with the skip model and then reranks selected candidates with the stronger autoregressive model. This design preserves accurate trajectory evaluation while reducing redundant rollout computation. Experiments on RECON, HuRoN, SCAND, and TartanDrive show that, compared with NWM, SkipWorld reduces the average ATE by 4.2% and RPE by 2.7%, while decreasing planning time by 62.4% with a 2.7-fold speedup.

Figures & tables

Figure 1: Overview and performance of SkipWorld. Top: SkipWorld performs coarse-to-fine planning by first using a skip student to filter

N

candidate trajectories into a compact Top-

P

subset, followed by latent autoregressive teacher refinement. Bottom-left: Example trajectory prediction on RECON, where SkipWorld more closely follows the ground-truth trajectory than NWM. Bottom-right: Compared with NWM, SkipWorld reduces average ATE by 4.2% and RPE by 2.7%, while achieving a

2.7\times

planning speedup.

Figure 2: Overview of SkipWorld. First, Latent AutoRoll trains an autoregressive world model that rolls out directly in latent space. Second, SkipDistill trains a skip student to predict longer-horizon latent transitions through teacher endpoint distillation, where the latent alignment error (Section A.5 ) visualizes the spatial discrepancy between the student skip endpoint and the autoregressive teacher endpoint. Third, Skip-then-Refine Planning uses the skip student to cheaply filter candidate trajectories and then uses the autoregressive teacher to accurately rerank only the selected candidates.

Model	RECON		HuRoN		TartanDrive		SCAND		Avg.
Model	ATE	RPE	ATE	RPE	ATE	RPE	ATE	RPE	ATE	RPE
GNM	1.87	0.73	3.71	1.00	6.65	1.62	2.12	0.61	3.59	0.99
NoMaD	1.95	0.53	3.73	0.96	6.32	1.31	2.24	0.49	3.56	0.82
NWM + NoMaD ( $\times 16$ )	1.88	0.51	3.73	0.95	6.26	1.30	2.18	0.48	3.51	0.82
NWM + NoMaD ( $\times 32$ )	1.79	0.49	3.68	0.95	6.25	1.29	2.19	0.47	3.47	0.80
NWM (original)	1.13	0.35	4.12	0.96	5.63	1.18	1.28	0.33	3.04	0.71

Table 1: Main comparison on four visual navigation benchmarks. Lower is better. The best result is shown in bold and the second best is underlined. NWM (original) is from its original paper. NWM (pretrained) denotes our four‑dataset retrained baseline. Due to privacy restrictions, only a low‑resolution HuRoN subset is available. For fair comparison, We thus fine‑tune it from the 100K‑step checkpoint for another 50K steps with identical settings.

Variant	AutoRoll	SkipDistill	Refine	ATE	RPE	Time
NWM	–	–	–	3.33	0.73	2220.5
Latent AutoRoll	✓	–	–	3.26	0.72	3807.9
AutoRoll + SkipDistill	✓	✓	–	3.26	0.73	465.3
AutoRoll + Refine	✓	–	✓	3.20	0.71	991.7
SkipWorld	✓	✓	✓	3.19	0.71	834.5

Table 2: Component ablation averaged over four datasets. Time is seconds per query. Lower is better. The best result is shown in bold and the second best is underlined.

Stride	RECON		HuRoN		TartanDrive		SCAND		Avg.		Time
Stride	ATE	RPE	ATE	RPE	ATE	RPE	ATE	RPE	ATE	RPE	Time
2	1.55	0.39	4.97	1.06	5.90	1.23	1.06	0.29	3.37	0.74	1257.7
4	1.30	0.36	4.94	1.05	5.72	1.19	1.02	0.28	3.24	0.72	973.7
8	1.19	0.34	4.86	1.04	5.74	1.20	0.98	0.28	3.19	0.71	834.5

Table 3: Ablation on skip stride with candidate budget fixed to

P=20

. Time is seconds per query. Lower is better. The best result is shown in bold and the second best is underlined.

Budget	RECON		HuRoN		TartanDrive		SCAND		Avg.		Time
Budget	ATE	RPE	ATE	RPE	ATE	RPE	ATE	RPE	ATE	RPE	Time
$P=5$	1.32	0.36	4.92	1.06	5.67	1.19	1.07	0.29	3.25	0.73	460.5
$P=10$	1.33	0.36	4.89	1.05	5.67	1.18	0.96	0.28	3.21	0.72	610.1
$P=20$	1.19	0.34	4.86	1.04	5.74	1.20	0.98	0.28	3.19	0.71	834.5
$P=40$	1.15	0.33	4.88	1.05	5.83	1.21	0.98	0.27	3.21	0.72	1436.1

Table 4: Ablation on coarse candidate budget with skip stride fixed to

s=8

. Time is seconds per query. Lower is better. The best result is shown in bold and the second best is underlined.

Figure 3: Predicted trajectories on the four evaluation datasets. SkipWorld produces trajectories that better follow the ground-truth path across diverse indoor, outdoor, and driving scenes.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Candidate samples $N$	120
Elite candidates $K$	5
CEM update steps	1
Batch size	1
Number of workers	12
Dense repeat evaluations $R$	3

Appendix

Table 5: Default planning hyperparameters used in our experiments.

Method	RECON		HuRoN		TartanDrive		SCAND		Avg.		Time
Method	ATE	RPE	ATE	RPE	ATE	RPE	ATE	RPE	ATE	RPE	Time
NWM	1.381	0.351	5.076	1.083	5.791	1.209	1.081	0.290	3.332	0.733	2220.5
Latent AutoRoll	1.245	0.336	4.909	1.048	5.846	1.216	1.020	0.277	3.255	0.720	3807.9
AutoRoll + SkipDistill	1.487	0.380	4.846	1.043	5.638	1.178	1.067	0.297	3.260	0.725	465.3
AutoRoll + Refine	1.191	0.336	4.839	1.038	5.798	1.208	0.987	0.271	3.204	0.713	991.7
SkipWorld	1.191	0.344	4.859	1.039	5.736	1.196	0.980	0.277	3.192	0.714	834.5

Appendix

Table 6: Full dataset-wise component ablation. Time is average planning time per query. Lower is better. The best result is shown in bold and the second best is underlined.

Method	0.5s		1.0s		1.5s		2.0s
Method	ATE	RPE	ATE	RPE	ATE	RPE	ATE	RPE
NWM	1.134	0.708	1.884	0.706	2.607	0.716	3.332	0.733
Latent AutoRoll	1.119	0.694	1.854	0.693	2.551	0.700	3.255	0.720
AutoRoll+SkipDistill	1.092	0.682	1.821	0.687	2.532	0.701	3.260	0.725
AutoRoll+Refine	1.098	0.681	1.816	0.682	2.505	0.693	3.204	0.713
SkipWorld	1.084	0.673	1.797	0.677	2.487	0.691	3.192	0.714

Appendix

Table 7: Horizon-wise error analysis averaged over four datasets. Lower is better. The best result is shown in bold and the second best is underlined.

Training Loss	RECON		HuRoN		TartanDrive		SCAND		Avg.		Time
Training Loss	ATE	RPE	ATE	RPE	ATE	RPE	ATE	RPE	ATE	RPE	Time
$\mathcal{L}_{\mathrm{direct}}$	1.235	0.345	4.945	1.056	5.807	1.208	0.996	0.275	3.246	0.721	824.5
$\mathcal{L}_{\mathrm{direct}}+\mathcal{L}_{\mathrm{distill}}$	1.168	0.331	4.896	1.048	5.779	1.201	0.999	0.276	3.210	0.714	839.9
$\mathcal{L}_{\mathrm{distill}}$	1.191	0.344	4.859	1.039	5.736	1.196	0.980	0.277	3.192	0.714	834.5

Appendix

Table 8: Loss ablation of SkipDistill under the same Skip-then-Refine planning setting. Time is average planning time per query. Lower is better. The best result is shown in bold and the second best is underlined.

Figure 4: Additional scene-aware qualitative results on RECON.

Figure 5: Additional scene-aware qualitative results on SCAND.

Figure 6: Additional scene-aware qualitative results on HuRoN.

Figure 7: Additional scene-aware qualitative results on TartanDrive.

Visual Navigation

Visual navigation enables robots and embodied agents to reach goals by interpreting camera images and acting in unfamiliar or changing environments.

See all 71 papers

LiteNWM: Efficient Latent World Models for Onboard Visual Navigation in the Wild
Oct 8, 2026Linkai Liu, Yuntian Zhang, Zhenshan Bing +5Robot Navigation Latent World Models
Nanjing University · Imperial College London · Sony AI +1
Scope-WM: Scoped Computation for Efficient Visual World Models
Sep 27, 2026Chunzheng Li, Zesheng Jia, Hongda Zhang +4World Models for Robotics World Models
Soochow University · Fudan University
Fast LeWorldModel
Jun 24, 2026Yuntian Gao, Xiangyu XuLatent Dynamics Modeling World Model-Based Planning
Xi’an Jiaotong University

Oct 8, 2026J/K move · Enter open · S save

LiteNWM: Efficient Latent World Models for Onboard Visual Navigation in the Wild

Linkai Liu, Yuntian Zhang, Zhenshan Bing, Chen Chen+4

Nanjing University · Imperial College London · Sony AI+1

Robot Navigation Latent World Models World Model-Based Planning Latent-Space Planning Efficient Inference

Open paper AskPDF arXiv

Direct visual navigation policies generate trajectories efficiently but do not explicitly evaluate their future consequences. Generative navigation world models provide this foresight through visual rollouts, which are costly when evaluating multiple candidates. We present LiteNWM, a latent navigation world model that shares visual encoding across candidates and jointly predicts their action-conditioned future representations at multiple horizons, while a learned scorer uses these predictions to select trajectories. In offline evaluations on RECON, SCAND, and SACSoN, LiteNWM reduces macro-averaged trajectory error by 17.56% relative to NoMaD+NWM-XL and achieves a 128.00-fold end-to-end speedup on an RTX 5090. The same evaluator transfers from NoMaD to MBRA without proposer-specific retraining, reducing MBRA's macro-averaged trajectory error by 16.2%. In real-robot experiments in unseen indoor and outdoor environments, LiteNWM improves navigation success from 43.3% to 83.3% relative to NoMaD. These results demonstrate that LiteNWM can be deployed for future-aware planning and closed-loop navigation on a physical robot.

Abstract

Visual Navigation

Explore similar work

LiteNWM: Efficient Latent World Models for Onboard Visual Navigation in the Wild

Scope-WM: Scoped Computation for Efficient Visual World Models

Fast LeWorldModel