Trajectory-Diversity-Driven Robust Vision-and-Language Navigation

Jiangyang Li; Cong Wan; SongLin Dong; Chenhao Ding; Qiang Wang; Zhiheng Ma; Yihong Gong

cs.CVMar 16, 2026

Trajectory-Diversity-Driven Robust Vision-and-Language Navigation

Authors: Jiangyang Li, Cong Wan, SongLin Dong, Chenhao Ding, Qiang Wang, Zhiheng Ma, Yihong Gong

Organizations: Xi’an Jiaotong University · Faculty of Computility Microelectronics, Shenzhen University of Advanced Technology

Abstract

Vision-and-Language Navigation (VLN) requires agents to navigate photo-realistic environments following natural language instructions. Current methods predominantly rely on imitation learning, which suffers from limited generalization and poor robustness to execution perturbations. We present NavGRPO, a reinforcement learning framework that learns goal-directed navigation policies through Group Relative Policy Optimization. By exploring diverse trajectories and optimizing via within-group performance comparisons, our method enables agents to distinguish effective strategies beyond expert paths without requiring additional value networks. Built on ScaleVLN, NavGRPO achieves superior robustness on R2R and REVERIE benchmarks with +3.0% and +1.71% SPL improvements in unseen environments. Under extreme early-stage perturbations, we demonstrate +14.89% SPL gain over the baseline, confirming that goal-directed RL training builds substantially more robust navigation policies. Code and models will be released.

Figures & tables

Figure 1: Navigation behavior under early-stage perturbations in unfamiliar environments. The baseline IL agent struggles to recover from errors due to limited exposure to failed trajectories, often detouring or failing to reach the goal. Our NavGRPO agent learns from diverse rollouts, enabling robust error correction and successful navigation despite perturbations.

Figure 2: Overview of our NavGRPO training framework for vision-language navigation. For each instruction, we sample K diverse trajectories through policy rollout, compute rewards using trajectory-level and step-level signals, estimate group relative advantages by comparing within instruction groups, and optimize the policy through debiased advantage estimation without value networks.

Method	Global Perturbation						Early Perturbation
Method	prob	OSR $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$	$\Delta$ SPL	Steps	OSR $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$	$\Delta$ SPL
ScaleVLN	0.0	87.48	2.34	79.40	69.97	-0.00	0	87.48	2.34	79.40	69.97	-0.00
NavGRPO	0.0	89.18	2.19	81.88	72.65	-0.00	0	89.18	2.19	81.88	72.65	-0.00
ScaleVLN	0.2	87.14	2.43	78.97	67.80	-2.17	1	83.06	2.94	74.76	59.06	-10.91
NavGRPO	0.2	88.75	2.21	81.27	72.11	-0.54	1	85.05	2.67	77.44	67.04	-5.61
ScaleVLN	0.4	87.19	2.42	78.67	65.90	-4.07	2	83.61	2.86	75.69	54.25	-15.72

Table 1: Robustness analysis under stochastic perturbations on R2R Val Unseen. Left: Global perturbation samples from policy distribution with probability

p

. Right: Early perturbation selects the least probable action for the first

N

steps.

\Delta

SPL shows SPL degradation from the unperturbed setting (prob=0 or Steps=0) for each method. NavGRPO demonstrates superior robustness across all perturbation levels.

Methods	R2R Val Unseen			R2R Test Unseen			REVERIE Val Unseen			REVERIE Test Unseen
Methods	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$
RCM † [ 91 ]	5.88	43	-	6.12	43	38	14.23	9.29	6.97	11.68	7.84	6.67
SERL † [ 84 ]	4.74	56	48	5.63	53	49	-	-	-	-	-	-
VLN $\circlearrowright$ BERT [ 29 ]	3.93	63	57	4.09	63	57	35.02	30.67	24.90	32.91	29.61	23.99
HAMT † [ 12 ]	3.65	66	61	3.93	65	60	36.84	32.95	30.20	33.41	30.40	26.67
SEvol † [ 9 ]	3.99	62	57	4.13	62	57	-	-	-	-	-	-

Table 2: Comparison with SoTA methods on R2R and REVERIE datasets.

\dagger

: Methods using reinforcement learning for policy optimization.

Methods	Val Unseen			Test Unseen
Methods	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$
LAW [ 77 ]	6.83	35	31	-	-	-
Sim2Sim [ 41 ]	6.07	43	36	6.17	44	37
MGMap [ 11 ]	6.28	39	34	7.11	35	28
CMA $\dagger$ [ 28 ]	6.20	41	36	6.30	38	33
VLN $\circlearrowright$ BERT $\dagger$ [ 28 ]	5.74	44	39	5.89	42	36

Table 3: Navigation performance on the R2R-CE dataset.

\dagger

: Methods that apply candidate waypoint predictor to support high-level action space.

Method #	Reward Components			R2R Val Unseen
Method #	R ${}_{\text{nav}}$	R ${}_{\text{path}}$	R ${}_{\text{step}}$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$
1	✓			2.31	80.14	71.07
2	✓	✓		2.26	81.02	71.88
3	✓		✓	2.25	81.29	71.93
4	✓	✓	✓	2.19	81.88	72.65

Table 4: Ablation study on reward function design. Experiments are conducted on R2R val unseen split.

Method	OSR $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$
w/o Group	87.48	2.34	79.40	69.97
GRPO [ 22 ]	88.15	2.26	80.15	71.23
GSPO [ 103 ]	88.62	2.23	80.52	71.68
GMPO [ 102 ]	88.56	2.24	80.47	71.62
Dr.GRPO [ 63 ]	89.02	2.19	81.88	72.65

Table 5: Comparison of group-based policy optimization variants on R2R Val Unseen split. All methods use the same reward function and sampling strategy.

Trajectories	OSR $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$
$K=2$	88.28	2.31	79.98	71.02
$K=4$	88.62	2.28	80.37	71.98
$K=8$	89.02	2.19	81.88	72.65
$K=16$	89.12	2.15	81.96	72.87

Table 6: Analysis of sampling trajectory number on R2R Val Unseen split.

Method	OSR $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$
SFT only	87.48	2.34	79.40	69.97
SFT+REINFORCE	87.35	2.36	79.28	69.85
SFT+A2C	87.72	2.32	79.65	70.25
SFT+PPO	88.15	2.29	79.95	70.68
SFT+GRPO	89.02	2.19	81.88	72.65

Table 7: Comparison with alternative RL methods on R2R Val Unseen split.

Training Strategy	OSR $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$
SFT only	87.48	2.34	79.40	69.97
RL only	74.52	4.30	64.30	59.82
Sequential SFT-RL	88.95	2.22	81.25	72.08
Ours	89.02	2.19	81.88	72.65

Table 8: Analysis of different training strategies on R2R Val Unseen split.

Figure 3: Qualitative comparison on challenging instructions under normal conditions (top) and initial perturbations (bottom). ScaleVLN fails to recover from errors in both scenarios. Our GRPO-trained agent successfully completes tasks and demonstrates robust error correction under perturbations.

Standard (No Perturbation)
Method	OSR↑	NE↓	SR↑	SPL↑	$\Delta$ SPL
ScaleVLN + HAMT	83.07	2.65	74.98	66.12	-0.00
ScaleVLN + NavGRPO	89.18	2.19	81.88	72.65	-0.00
Global Perturbation ( $p=0.8$ )
ScaleVLN + HAMT	82.14	2.83	72.01	61.92	-4.20
ScaleVLN + NavGRPO	88.49	2.27	79.95	69.98	-2.67

Table 9: Comparison with value-based RL method (HAMT) under standard and perturbed settings on R2R Val Unseen. Both methods build upon ScaleVLN baseline.

$\alpha$	OSR $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$
0.0	88.64	2.26	81.92	71.48
0.25	89.18	2.19	81.88	72.65
0.5	88.71	2.23	81.34	71.92
1.0	88.23	2.31	80.76	70.85

Table 10: Ablation study on path efficiency weight

\alpha

R_{\text{path}}

on R2R Val Unseen split.

$\beta$	OSR $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$
0.0	88.52	2.27	81.64	72.28
0.01	89.18	2.19	81.88	72.65
0.05	88.73	2.24	81.52	71.84
0.1	88.21	2.32	80.92	70.56

Table 11: Ablation study on entropy regularization coefficient

\beta

on R2R Val Unseen split.

Training Strategy	OSR $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$
Sequential SFT-RL	88.95	2.22	81.25	72.08
Ours (M=100)	88.89	2.23	81.64	72.41
Ours (M=200)	89.18	2.19	81.88	72.65
Ours (M=400)	89.02	2.20	81.77	72.53

Table 12: Impact of hard case buffer size M on R2R Val Unseen split. All variants substantially outperform Sequential SFT-RL baseline.

	ScaleVLN	Ours(K=4)	Ours(K=8)
NE $\downarrow$	2.34	2.28	2.19
SR $\uparrow$	79.40	80.37	81.88
SPL $\uparrow$	69.97	71.98	72.65
FLOPs(G)	4.95	4.95	4.95
Batchsize	16	8	8
Rollouts per Sample	2	4	8

Table 13: Computational cost comparison between ScaleVLN and NavGRPO on R2R training.

Figure 4: Qualitative comparison of navigation trajectories. Purple point indicates the starting location. NavGRPO’s trajectory (green) successfully completes the instruction by navigating through the archway and reaching the target doorway, while ScaleVLN’s trajectory (yellow) fails to execute the complete route, demonstrating NavGRPO’s superior robustness against environmental complexity.

Figure 5: Failure case analysis. NavGRPO incorrectly interprets the spatial reference ”second bedroom to your left” in an instruction with multiple ambiguous directional cues, highlighting challenges in compositional spatial reasoning.

Vision-Language Navigation

Vision-Language Navigation studies how embodied agents use natural-language instructions and visual observations to plan and execute routes.

See all 310 papers

SeeNav-Agent: Enhancing Vision-Language Navigation with Visual Prompt and Step-Level Policy Optimization
Dec 2, 2025Zhengcheng Wang, Zichuan Lin, Yijun Yang +2Vision-Language Navigation
Tencent AI Lab
Joint On-and-Off Policy Learning for Vision-and-Language Navigation
Jul 15, 2026Qingrong He, Lin Zhao, Kevin Zheng +1Vision-Language Navigation Interactive Imitation Learning
All authors are with Joy Future Academy.
What Limits Vision-and-Language Navigation ?
May 13, 2026Yunheng Wang, Yuetong Fang, Taowen Wang +9Sim-to-Real Transfer Vision-Language Navigation
1HKUST(GZ) · 2JD Explore Academy · 1HKUST(GZ

Dec 2, 2025J/K move · Enter open · S save

SeeNav-Agent: Enhancing Vision-Language Navigation with Visual Prompt and Step-Level Policy Optimization

Zhengcheng Wang, Zichuan Lin, Yijun Yang, Haobo Fu+1

Tencent AI Lab

Vision-Language Navigation

Open paper AskPDF arXiv

Existing Vision-Language Navigation (VLN) agents based on Large Vision-Language Models (LVLMs) often suffer from errors of perception, reasoning, and planning, which significantly hinder their navigation performance. To address these limitations, a novel VLN agent framework, named SeeNav-Agent, is proposed. First, to reduce hallucinations of the visual module of the VLN agent, a dual-view Visual Prompt (VP) technique is introduced, which can also improve the agent's understanding of current spatial states. Subsequently, a novel step-level Reinforcement Fine-Tuning (RFT) method, Step Reward Group Policy Optimization (SRGPO), is designed for post-training. In SRGPO, we first define verifiable process rewards for navigation, and then perform efficient step-level advantage estimation by randomly grouping navigation steps. SRGPO provides dense reward signals for the RFT of the VLN agent and enhances its planning capability. Experiments on both the EmbodiedBench Navigation (Embench-Nav) and VLN-CE validate the effectiveness of the proposed method. By introducing the zero-shot VP module, the GPT-4.1 achieves a navigation success rate of 86.7%, surpassing the current best LVLM by approximately 20 percentage points (pp) on Embench-Nav and achieves an 11.1 pp improvement on VLN-CE compared to the base model. With VP+SRGPO, the Qwen2.5-VL-3B model reaches a navigation success rate of 72.3%, outperforming the best existing LVLM model by 5.6 pp on Embench-Nav and achieves 8.9 pp improvement on VLN-CE compared to the baseline method. Moreover, compared to GRPO and GiGPO, the proposed SRGPO demonstrates significant improvements in training stability, convergence efficiency, and generalization capability.

Abstract

Vision-Language Navigation

Explore similar work

SeeNav-Agent: Enhancing Vision-Language Navigation with Visual Prompt and Step-Level Policy Optimization

Joint On-and-Off Policy Learning for Vision-and-Language Navigation

What Limits Vision-and-Language Navigation ?