RIWANav: Recursive World-Action Models with Self-Improvement for Urban Navigation

Jing Xie; Shouwei Ruan; Yubin Wang; Yuxiang Zhang; Haitao Yang; Songchang Jin; Dianxi Shi

cs.ROOct 6, 2026

RIWANav: Recursive World-Action Models with Self-Improvement for Urban Navigation

Authors: Jing Xie, Shouwei Ruan, Yubin Wang, Yuxiang Zhang, Haitao Yang, Songchang Jin, Dianxi Shi

Organizations: MoE Key Lab of Artificial Intelligence, Institute of AI, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Beihang University, Beijing, China · The Hong Kong University of Science and Technology, Hong Kong, China · Tsinghua University, Beijing, China · The University of Texas at Austin, Austin, USA · Intelligent Game and Decision Lab (IGDL), Beijing, China

Abstract

Long-horizon urban navigation requires sequential local decisions whose errors can compound over time. Imitation learning (IL) rarely learns from failures, while physical trial-and-error reinforcement learning (RL) is costly. Action-conditioned world models can provide imagined feedback by predicting visual consequences for candidate actions. However, a frozen world model may become less reliable as the policy evolves. In this paper, we introduce RIWANAV, a post-training framework that casts the coupled adaptation of a world model and an action model (policy) as task-specific recursive self-improvement (RSI). Each cycle alternates two updates. The world model evaluates policy actions through imagined outcomes, providing comparative feedback for group-relative policy optimization (GRPO). The improved policy then constructs a grounded self-curriculum, selecting expert-consistent action-video pairs by behavioral novelty and prediction error. The refined world model supplies feedback for the next policy update, closing the recursive self-improvement loop. Experiments show that RIWANAV outperforms training baselines and prior methods, validating the proposed recursive self-improvement loop between the policy and world model. Real-world trials further demonstrate its practical applicability.

Figures & tables

Fig. 1: From imitation learning to recursive world–action models self-improvement. (a) Imitation learning trains the policy from expert trajectories but provides limited supervision for policy-induced failures. (b) A frozen world model enables learning from imagined outcomes, but its predictions become misaligned with the evolving policy. (c) RIWANav closes the training loop: imagined outcomes improve the policy, while grounded policy-induced samples refine the world model for subsequent policy learning.

Fig. 2: Recursive self-improvement loop in RIWANav . (1) The current world model imagines action consequences that improve the policy. (2) The improved policy constructs a grounded self-curriculum that improves the next world model. Repeating these updates recursively changes both components and the feedback exchanged between them. The flame marks the component optimized in each phase.

Test Seen
Training framework	SR (%) $\uparrow$	SPL (%) $\uparrow$	NE $\downarrow$	nDTW (%) $\uparrow$	SDTW (%) $\uparrow$	OSR (%) $\uparrow$
Imitation learning	70.91	70.55	3.8427	83.81	69.16	71.18
PPO [ 35 ]	$72.04\pm 0.06$	$71.40\pm 0.09$	$3.8634\pm 0.0136$	$84.02\pm 0.08$	$70.29\pm 0.06$	$72.36\pm 0.07$
GRPO [ 8 ]	$75.11\pm 0.75$	$74.45\pm 0.75$	$3.7012\pm 0.1803$	$86.40\pm 0.47$	$73.37\pm 0.76$	$75.32\pm 0.81$
PPO-WM	$78.90\pm 0.18$	$78.25\pm 0.19$	$2.9530\pm 0.0647$	$88.06\pm 0.13$	$77.36\pm 0.18$	$79.39\pm 0.20$
GRPO-WM	$83.35\pm 1.25$	$81.27\pm 1.38$	$2.7059\pm 0.1858$	$90.55\pm 0.73$	$81.66\pm 1.29$	$84.31\pm 1.17$

TABLE I: Navigation performance on the UrbanNav test-seen and test-unseen splits. Values are reported as mean

\pm

standard deviation.

Test Seen
Category	Method	SR (%) $\uparrow$	SPL (%) $\uparrow$	NE $\downarrow$	nDTW (%) $\uparrow$	SDTW (%) $\uparrow$	OSR (%) $\uparrow$
Imitation Learning	UrbanNav [ 2 ]	$60.36\pm 5.39$	$59.75\pm 5.34$	$5.3958\pm 0.6689$	$76.86\pm 3.50$	$58.22\pm 5.34$	$60.48\pm 5.36$
	NoMaD [ 11 ]	$\underline{82.35\pm 0.78}$	$\underline{77.59\pm 0.55}$	$\underline{2.5532\pm 0.0608}$	$90.15\pm 0.19$	$\underline{77.97\pm 0.79}$	$\underline{82.47\pm 0.76}$
	OmniVLA [ 12 ]	$77.49\pm 1.66$	$77.25\pm 1.66$	$3.1856\pm 0.1415$	$\underline{90.25\pm 0.47}$	$73.63\pm 1.72$	$77.66\pm 1.62$
WM-based	Fast-WAM [ 6 ]	$58.48\pm 0.75$	$39.57\pm 0.18$	$8.4899\pm 0.0723$	$64.77\pm 0.20$	$50.62\pm 0.56$	$62.07\pm 0.31$
WM-based	NWM [ 4 ]	$71.26\pm 0.21$	$70.67\pm 0.21$	$3.7117\pm 0.0200$	$83.88\pm 0.05$	$69.45\pm 0.20$	$71.63\pm 0.20$

TABLE II: Comparison with representative prior approaches on the UrbanNav test-seen and test-unseen splits. Values are reported as mean

\pm

standard deviation.

WM	Policy-induced	Grounded	SR (%) $\uparrow$	SPL (%) $\uparrow$
update	replay	self-curriculum	SR (%) $\uparrow$	SPL (%) $\uparrow$
$\times$	$\times$	$\times$	$80.31\pm 0.94$	$76.09\pm 1.05$
$\checkmark$	$\times$	$\times$	$78.23\pm 1.91$	$74.72\pm 1.92$
$\checkmark$	$\checkmark$	$\times$	$84.83\pm 0.33$	$80.11\pm 0.32$
$\checkmark$	$\checkmark$	$\checkmark$	$\mathbf{88.25\pm 0.43}$	$\mathbf{82.59\pm 0.25}$

TABLE III: Ablation study on the UrbanNav test-unseen split. Values are reported as mean

\pm

standard deviation.

World model	Evaluation	$L_{1}\downarrow$	LPIPS $\downarrow$
World model	actions	$L_{1}\downarrow$	LPIPS $\downarrow$
Initial $\phi_{0}$	Expert	$0.1396\pm 0.0032$	$0.4532\pm 0.0065$
Initial $\phi_{0}$	Policy-induced	$0.1521\pm 0.0042$	$0.4539\pm 0.0059$
Static expert	Expert	$0.1352\pm 0.0034$	$0.4265\pm 0.0079$
Static expert	Policy-induced	$0.1505\pm 0.0044$	$0.4314\pm 0.0067$
RIWANav $\phi_{K}$	Expert	$\mathbf{0.1329\pm 0.0034}$	$\mathbf{0.4203\pm 0.0079}$

TABLE IV: Endpoint world-model prediction quality on expert actions and actions generated by the improved policy.

Fig. 3: World-model prediction quality across recursive rounds. The left and right panels show

L_{1}

and LPIPS on two fixed evaluation sets: expert actions and actions generated by the improved policy. Points are means over 128 samples and error bars denote standard error. Round 0 uses the initial world model, whereas the final round

K=19

uses the final RIWANav world model. Lower is better.

Fig. 4: Qualitative evidence for recursive world-model improvement on actions generated by the improved policy. All three models receive the same observation, language goal, and action; the aligned recorded future is shown as reference. The rows depict night traffic, a bicycle-lined sidewalk, and a rainy sidewalk. In these cases, the RIWANav world model after recursive updates better preserves traversable corridors and scene layout, with fewer long-horizon artifacts.

Fig. 5: Real-world urban navigation with RIWANav . Orange dashed arrows indicate schematic paths, and green boxes highlight the targets.

Explore similar work

NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation
Jun 11, 2026Daichi Azuma, Taiki Miyanishi, Koya Sakamoto +6Efficient World-Action Model Object Goal Navigation
The University of Tokyo · National Institute of Informatics · AIRoA +1
NavOL: Navigation Policy with Online Imitation Learning
May 12, 2026Xiaofei Wei, Chun Gu, Li ZhangNavigation
School of Data Science, Fudan University · 2Shanghai Innovation Institute.
SC $^{2}$ -WM: A Self-Correcting World Model with Closed-Loop Feedback for Vision-and-Language Navigation in Continuous Environments
Aug 1, 2026Xuan Yao, Yuze Zhu, Junyu Gao +2Vision-Language Navigation Object Goal Navigation
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA) · School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS) · HiThink Research, China +1

Jun 11, 2026, cs.ROJ/K move · Enter open · S save

NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation

Daichi Azuma, Taiki Miyanishi, Koya Sakamoto, Shuhei Kurita+5

The University of Tokyo · National Institute of Informatics · AIRoA+1

Efficient World-Action Model Object Goal Navigation Navigation Foresight Diffusion Policies

Open paper AskPDF arXiv

Goal-conditioned visual navigation requires a robot to act under partial observability by anticipating how its motion will change the future egocentric view and whether that change brings it closer to the goal. Navigation world models provide such visual foresight, but they remain prediction modules that require an external planner to convert predicted futures into closed-loop control. We propose Navigation World Action Model (NavWAM), a diffusion-transformer policy that turns navigation world-model prediction into executable action by representing future observations, goal-progress values, and action chunks in a shared latent sequence. By learning future prediction jointly with the action and value targets that determine closed-loop behavior, NavWAM makes visual foresight directly usable for robot control. We build NavWAM through simulation pretraining and real-robot adaptation, and evaluate it on image-goal navigation against planning-based world models and a representative direct navigation policy. Across offline benchmarks and closed-loop real-robot deployment, NavWAM improves over planning-based world-model baselines in our evaluations while using the default policy mode without CEM-style action search. Project page: https://dachii-azm.github.io/navwam/

Abstract

Explore similar work

NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation

NavOL: Navigation Policy with Online Imitation Learning

SC2^{2}2-WM: A Self-Correcting World Model with Closed-Loop Feedback for Vision-and-Language Navigation in Continuous Environments

SC $^{2}$ -WM: A Self-Correcting World Model with Closed-Loop Feedback for Vision-and-Language Navigation in Continuous Environments