Vision-and-Language Navigation (VLN) requires agents to navigate photo-realistic environments following natural language instructions. Current methods predominantly rely on imitation learning, which suffers from limited generalization and poor robustness to execution perturbations. We present NavGRPO, a reinforcement learning framework that learns goal-directed navigation policies through Group Relative Policy Optimization. By exploring diverse trajectories and optimizing via within-group performance comparisons, our method enables agents to distinguish effective strategies beyond expert paths without requiring additional value networks. Built on ScaleVLN, NavGRPO achieves superior robustness on R2R and REVERIE benchmarks with +3.0% and +1.71% SPL improvements in unseen environments. Under extreme early-stage perturbations, we demonstrate +14.89% SPL gain over the baseline, confirming that goal-directed RL training builds substantially more robust navigation policies. Code and models will be released.
Figures & tables
Figure 1: Navigation behavior under early-stage perturbations in unfamiliar environments. The baseline IL agent struggles to recover from errors due to limited exposure to failed trajectories, often detouring or failing to reach the goal. Our NavGRPO agent learns from diverse rollouts, enabling robust error correction and successful navigation despite perturbations.
Figure 2: Overview of our NavGRPO training framework for vision-language navigation. For each instruction, we sample K diverse trajectories through policy rollout, compute rewards using trajectory-level and step-level signals, estimate group relative advantages by comparing within instruction groups, and optimize the policy through debiased advantage estimation without value networks.
Method
Global Perturbation
Early Perturbation
prob
OSR ↑
NE ↓
SR ↑
SPL ↑
Δ SPL
Steps
OSR ↑
NE ↓
SR ↑
SPL ↑
Δ SPL
ScaleVLN
0.0
87.48
2.34
79.40
69.97
-0.00
0
87.48
2.34
79.40
69.97
-0.00
NavGRPO
0.0
89.18
2.19
81.88
72.65
-0.00
0
89.18
2.19
81.88
72.65
-0.00
ScaleVLN
0.2
87.14
2.43
78.97
67.80
-2.17
1
83.06
2.94
74.76
59.06
-10.91
NavGRPO
0.2
88.75
2.21
81.27
72.11
-0.54
1
85.05
2.67
77.44
67.04
-5.61
ScaleVLN
0.4
87.19
2.42
78.67
65.90
-4.07
2
83.61
2.86
75.69
54.25
-15.72
Table 1: Robustness analysis under stochastic perturbations on R2R Val Unseen. Left: Global perturbation samples from policy distribution with probability p . Right: Early perturbation selects the least probable action for the first N steps. Δ SPL shows SPL degradation from the unperturbed setting (prob=0 or Steps=0) for each method. NavGRPO demonstrates superior robustness across all perturbation levels.
Methods
R2R Val Unseen
R2R Test Unseen
REVERIE Val Unseen
REVERIE Test Unseen
NE ↓
SR ↑
SPL ↑
NE ↓
SR ↑
SPL ↑
OSR ↑
SR ↑
SPL ↑
OSR ↑
SR ↑
SPL ↑
RCM † [ 91 ]
5.88
43
-
6.12
43
38
14.23
9.29
6.97
11.68
7.84
6.67
SERL † [ 84 ]
4.74
56
48
5.63
53
49
-
-
-
-
-
-
VLN ↻ BERT [ 29 ]
3.93
63
57
4.09
63
57
35.02
30.67
24.90
32.91
29.61
23.99
HAMT † [ 12 ]
3.65
66
61
3.93
65
60
36.84
32.95
30.20
33.41
30.40
26.67
SEvol † [ 9 ]
3.99
62
57
4.13
62
57
-
-
-
-
-
-
Table 2: Comparison with SoTA methods on R2R and REVERIE datasets. † : Methods using reinforcement learning for policy optimization.
Methods
Val Unseen
Test Unseen
NE ↓
SR ↑
SPL ↑
NE ↓
SR ↑
SPL ↑
LAW [ 77 ]
6.83
35
31
-
-
-
Sim2Sim [ 41 ]
6.07
43
36
6.17
44
37
MGMap [ 11 ]
6.28
39
34
7.11
35
28
CMA † [ 28 ]
6.20
41
36
6.30
38
33
VLN ↻ BERT † [ 28 ]
5.74
44
39
5.89
42
36
Table 3: Navigation performance on the R2R-CE dataset. † : Methods that apply candidate waypoint predictor to support high-level action space.
Method #
Reward Components
R2R Val Unseen
R nav
R path
R step
NE ↓
SR ↑
SPL ↑
1
✓
2.31
80.14
71.07
2
✓
✓
2.26
81.02
71.88
3
✓
✓
2.25
81.29
71.93
4
✓
✓
✓
2.19
81.88
72.65
Table 4: Ablation study on reward function design. Experiments are conducted on R2R val unseen split.
Method
OSR ↑
NE ↓
SR ↑
SPL ↑
w/o Group
87.48
2.34
79.40
69.97
GRPO [ 22 ]
88.15
2.26
80.15
71.23
GSPO [ 103 ]
88.62
2.23
80.52
71.68
GMPO [ 102 ]
88.56
2.24
80.47
71.62
Dr.GRPO [ 63 ]
89.02
2.19
81.88
72.65
Table 5: Comparison of group-based policy optimization variants on R2R Val Unseen split. All methods use the same reward function and sampling strategy.
Trajectories
OSR ↑
NE ↓
SR ↑
SPL ↑
K=2
88.28
2.31
79.98
71.02
K=4
88.62
2.28
80.37
71.98
K=8
89.02
2.19
81.88
72.65
K=16
89.12
2.15
81.96
72.87
Table 6: Analysis of sampling trajectory number on R2R Val Unseen split.
Method
OSR ↑
NE ↓
SR ↑
SPL ↑
SFT only
87.48
2.34
79.40
69.97
SFT+REINFORCE
87.35
2.36
79.28
69.85
SFT+A2C
87.72
2.32
79.65
70.25
SFT+PPO
88.15
2.29
79.95
70.68
SFT+GRPO
89.02
2.19
81.88
72.65
Table 7: Comparison with alternative RL methods on R2R Val Unseen split.
Training Strategy
OSR ↑
NE ↓
SR ↑
SPL ↑
SFT only
87.48
2.34
79.40
69.97
RL only
74.52
4.30
64.30
59.82
Sequential SFT-RL
88.95
2.22
81.25
72.08
Ours
89.02
2.19
81.88
72.65
Table 8: Analysis of different training strategies on R2R Val Unseen split.
Figure 3: Qualitative comparison on challenging instructions under normal conditions (top) and initial perturbations (bottom). ScaleVLN fails to recover from errors in both scenarios. Our GRPO-trained agent successfully completes tasks and demonstrates robust error correction under perturbations.
Method
OSR↑
NE↓
SR↑
SPL↑
Δ SPL
Standard (No Perturbation)
ScaleVLN + HAMT
83.07
2.65
74.98
66.12
-0.00
ScaleVLN + NavGRPO
89.18
2.19
81.88
72.65
-0.00
Global Perturbation ( p=0.8 )
ScaleVLN + HAMT
82.14
2.83
72.01
61.92
-4.20
ScaleVLN + NavGRPO
88.49
2.27
79.95
69.98
-2.67
Table 9: Comparison with value-based RL method (HAMT) under standard and perturbed settings on R2R Val Unseen. Both methods build upon ScaleVLN baseline.
α
OSR ↑
NE ↓
SR ↑
SPL ↑
0.0
88.64
2.26
81.92
71.48
0.25
89.18
2.19
81.88
72.65
0.5
88.71
2.23
81.34
71.92
1.0
88.23
2.31
80.76
70.85
Table 10: Ablation study on path efficiency weight α in Rpath on R2R Val Unseen split.
β
OSR ↑
NE ↓
SR ↑
SPL ↑
0.0
88.52
2.27
81.64
72.28
0.01
89.18
2.19
81.88
72.65
0.05
88.73
2.24
81.52
71.84
0.1
88.21
2.32
80.92
70.56
Table 11: Ablation study on entropy regularization coefficient β on R2R Val Unseen split.
Training Strategy
OSR ↑
NE ↓
SR ↑
SPL ↑
Sequential SFT-RL
88.95
2.22
81.25
72.08
Ours (M=100)
88.89
2.23
81.64
72.41
Ours (M=200)
89.18
2.19
81.88
72.65
Ours (M=400)
89.02
2.20
81.77
72.53
Table 12: Impact of hard case buffer size M on R2R Val Unseen split. All variants substantially outperform Sequential SFT-RL baseline.
ScaleVLN
Ours(K=4)
Ours(K=8)
NE ↓
2.34
2.28
2.19
SR ↑
79.40
80.37
81.88
SPL ↑
69.97
71.98
72.65
FLOPs(G)
4.95
4.95
4.95
Batchsize
16
8
8
Rollouts per Sample
2
4
8
Table 13: Computational cost comparison between ScaleVLN and NavGRPO on R2R training.
Figure 4: Qualitative comparison of navigation trajectories. Purple point indicates the starting location. NavGRPO’s trajectory (green) successfully completes the instruction by navigating through the archway and reaching the target doorway, while ScaleVLN’s trajectory (yellow) fails to execute the complete route, demonstrating NavGRPO’s superior robustness against environmental complexity.
Figure 5: Failure case analysis. NavGRPO incorrectly interprets the spatial reference ”second bedroom to your left” in an instruction with multiple ambiguous directional cues, highlighting challenges in compositional spatial reasoning.
Existing Vision-Language Navigation (VLN) agents based on Large Vision-Language Models (LVLMs) often suffer from errors of perception, reasoning, and planning, which significantly hinder their navigation performance. To address these limitations, a novel VLN agent framework, named SeeNav-Agent, is proposed. First, to reduce hallucinations of the visual module of the VLN agent, a dual-view Visual Prompt (VP) technique is introduced, which can also improve the agent's understanding of current spatial states. Subsequently, a novel step-level Reinforcement Fine-Tuning (RFT) method, Step Reward Group Policy Optimization (SRGPO), is designed for post-training. In SRGPO, we first define verifiable process rewards for navigation, and then perform efficient step-level advantage estimation by randomly grouping navigation steps. SRGPO provides dense reward signals for the RFT of the VLN agent and enhances its planning capability. Experiments on both the EmbodiedBench Navigation (Embench-Nav) and VLN-CE validate the effectiveness of the proposed method. By introducing the zero-shot VP module, the GPT-4.1 achieves a navigation success rate of 86.7%, surpassing the current best LVLM by approximately 20 percentage points (pp) on Embench-Nav and achieves an 11.1 pp improvement on VLN-CE compared to the base model. With VP+SRGPO, the Qwen2.5-VL-3B model reaches a navigation success rate of 72.3%, outperforming the best existing LVLM model by 5.6 pp on Embench-Nav and achieves 8.9 pp improvement on VLN-CE compared to the baseline method. Moreover, compared to GRPO and GiGPO, the proposed SRGPO demonstrates significant improvements in training stability, convergence efficiency, and generalization capability.
Vision-and-Language Navigation (VLN) necessitates an embodied agent to navigate in the physical world by adhering to natural language instructions. Recent advancements in Vision-Language Models (VLM) have propelled the development of VLM-based VLN methods with two predominant paradigms: (1) imitation learning (IL) on expert demonstrations, followed by the Dataset Aggregation (DAgger) algorithm to bolster error recovery capabilities; (2) reinforcement learning (RL) driven by verifiable rewards to enhance reasoning and exploration. A notable gap is the absence of integration between these two distinct paradigms. This paper introduces JOP-VLN, a novel VLN framework that synergistically combines off-policy imitation learning and on-policy exploration within a three-stage training pipeline. Initially, IL is employed on expert demonstrations to acquire basic navigation skills. Subsequently, the DAgger algorithm is utilized to generate heuristic exploration trajectories, which are then used for imitation learning to improve error recovery capabilities. Finally, a joint on-and-off policy learning framework is implemented, featuring high-entropy trajectory sampling to enhance RL training efficiency and an error-correction-prioritized trajectory sorting strategy for effective error correction. Extensive experiments demonstrate the efficacy of JOP-VLN, achieving success rates of 69.9% and 68.0% on the VLN-CE R2R and RxR benchmarks, respectively, setting a new state-of-the-art on R2R. Project page: https://qingrongh.github.io/JOP-VLN.
Vision-and-Language Navigation (VLN) is a cornerstone of embodied intelligence. However, current agents often suffer from significant performance degradation when transitioning from simulation to real-world deployment, primarily due to perceptual instability (e.g., lighting variations and motion blur) and under-specified instructions. While existing methods attempt to bridge this gap by scaling up model size and training data, we argue that the bottleneck lies in the lack of robust spatial grounding and cross-domain priors. In this paper, we propose StereoNav, a robust Vision-Language-Action framework designed to enhance real-world navigation consistency. To address the inherent gap between synthetic training and physical execution, we introduce Target-Location Priors as a persistent bridge. These priors provide stable visual guidance that remains invariant across domains, effectively grounding the agent even when instructions are vague. Furthermore, to mitigate visual disturbances like motion blur and illumination shifts, StereoNav leverages stereo vision to construct a unified representation of semantics and geometry, enabling precise action prediction through enhanced depth awareness. Extensive experiments on R2R-CE and RxR-CE demonstrate that StereoNav achieves state-of-the-art egocentric RGB performance, with SR and SPL scores of 81.1% and 68.3%, and 67.5% and 52.0%, respectively, while using significantly fewer parameters and less training data than prior scaling-based approaches. More importantly, real-world robotic deployments confirm that StereoNav substantially improves navigation reliability in complex, unstructured environments. Project page: https://yunheng-wang.github.io/stereonav-public.github.io.