Long-horizon vision-language navigation (VLN) requires agents to track instruction progress and recover from local errors before they propagate across subtasks. However, executors trained on expert trajectories encounter policy-induced state shifts during deployment, and their stop signals do not reliably indicate subtask completion. We introduce Learning and Verifying Progress for VLN (LeaP-VLN), an agentic VLN system that couples progress-aware execution with independent progress verification through plan--execute--verify--recover cycles. LeaP-VLN leverages progress both explicitly, through a progress verifier that verifies subtask completion before the agent advances and triggers dual-phase recovery upon route deviation, and implicitly, by co-training the executor on progress reasoning and action prediction to enhance the executor's progress awareness and corrective control. To adapt the executor to this closed-loop interface, we propose a two-stage progress-enhanced curriculum: behavior cloning on expert demonstrations, followed by joint training on demonstrations, expert action corrections, and progress-reasoning supervision. Policy-generated trajectories expose the executor to both intra-subtask drift and inter-subtask state shifts, aligning training with the deviations encountered during closed-loop execution. Extensive experiments on R2R-CE and RxR-CE demonstrate state-of-the-art success rates among monocular methods trained without additional navigation datasets, while real-world experiments further validate the effectiveness of LeaP-VLN in long-horizon navigation.
Figures & tables
Fig. 1: Comparison of long-horizon VLN execution paradigms. End-to-end policies map the complete route instruction and visual observations directly to actions, with route progress implicitly inferred within the action policy. Hierarchical planner–executor pipelines without outcome verification dispatch local subtasks and advance based on executor termination. Our LeaP-VLN couples progress-aware execution with progress verification and dual-phase recovery to ground subtask transitions in observed outcomes and limit error propagation.
Fig. 2: Overview of the proposed closed-loop agentic framework with test-time verification and recovery. Two-stage progress-enhanced curriculum learning, which clones expert actions and then co-trains intra- and inter-subtask on-policy action corrections with progress-reasoning supervision, produces the low-level VLN executor, while the high-level multimodal agent maintains landmark memory and accepted anchors and orchestrates subtask planning, progress verification, and dual-phase failure recovery.
Fig. 3: Low-level VLN executor training. (a) Boundary-aligned local-instruction data generation cleans and merges segment annotations, aligns frame–action targets, and adds explicit STOP supervision at subtask boundaries. (b) The two-stage progress-enhanced curriculum first initializes the executor by behavior cloning, then collects intra- and inter-subtask on-policy rollouts for expert action correction and progress reasoning, and finally jointly trains on all supervision.
Method
Observation
R2R-CE Val-Unseen
RxR-CE Val-Unseen
S.RGB
Pano.
Depth
Odom.
NE ↓
OSR ↑
SR ↑
SPL ↑
NE ↓
SR ↑
SPL ↑
CMA [ 2 ]
✓
✓
✓
6.20
52.0
41.0
36.0
8.76
26.5
22.1
InstructNav [ 8 ]
✓
✓
✓
6.89
–
31.0
24.0
–
–
–
ETPNav [ 24 ]
✓
✓
✓
4.71
65.0
57.0
49.0
5.64
54.7
44.8
HNR [ 25 ]
✓
✓
✓
4.42
67.0
61.0
51.0
5.50
56.3
46.7
NavFoM [ 15 ]
✓
4.61
72.1
61.7
55.3
4.74
64.4
56.2
TABLE I: Comparison on the R2R-CE and RxR-CE val-unseen splits.
Fig. 4: Comparison with Uni-NaVid and StreamVLN on hard episodes (ground-truth action sequences exceed 50 steps) in VLN-CE.
Fig. 5: Qualitative visualization of long-horizon instruction following. The complete navigation instruction is shown at the top of each example. For every subtask, the egocentric observation is paired with the corresponding top-down map, where the colored trajectory indicates the agent’s stage-wise navigation progress. In the top example, the agent sequentially executes all subtasks and successfully reaches the target location. In the bottom example, the verifier detects a mid-route deviation and triggers a subtask retry, enabling successful completion.
Components
R2R-CE Val-Unseen
RxR-CE Val-Unseen
A
B
C
D
NE ↓
OSR ↑
SR ↑
SPL ↑
NE ↓
SR ↑
SPL ↑
✓
4.96
64.8
55.5
49.7
5.78
58.6
42.3
✓
✓
4.46
69.3
61.7
52.4
4.33
65.2
51.5
✓
✓
3.78
73.0
67.0
59.0
4.86
62.5
50.1
✓
✓
✓
3.91
71.7
66.7
57.6
4.10
67.9
54.4
✓
✓
✓
✓
3.71
73.9
67.5
59.1
3.89
69.6
52.8
TABLE II: Ablation of the proposed framework components on the R2R-CE and RxR-CE val-unseen splits.
Fig. 6: Qualitative effects of progress co-training on action prediction. Without co-training, the executor either continues after task completion (top) or stops prematurely (bottom). Co-training improves action-progress consistency, producing STOP for completed subtasks and continued execution for unfinished ones.
Fig. 7: Real-world long-horizon navigation on the AgiBot Genie 02 robot. LeaP-VLN decomposes the route instruction into three local subtasks: approaching the divider, passing the sink, and stopping before the toilet. For each subtask, the top image shows the robot’s third-person start state and the bottom row shows the corresponding egocentric image.
Existing vision-language navigation methods often couple a VLM with waypoint decoders to produce multi-step action plans, but they typically lack an explicit closed-loop mechanism for tracking semantic progress, diagnosing execution failures, and recovering from error accumulation in long-horizon navigation. To address this gap, we propose ReflectVLN, an agentic VLN framework that organizes decision-making through bidirectionally interactive intention and execution agents. The intention agent performs subtask decomposition and reflection, generating executable subtask descriptions as corrective plans. Conditioned on these descriptions, the execution agent grounds them into short-horizon actions under current observations while monitoring sub-goal progress and detecting off-track behavior. Crucially, ReflectVLN enables closed-loop bidirectional communication: the execution agent emits progress and deviation signals to trigger reflection and subtask updates on demand, and the intention agent returns structured guidance that reconditions subsequent actions for recovery. To encourage temporally coherent decisions with interpretable intermediate rationales, we introduce Action Chain-of-Thought (Action-CoT), a path-conditioned dual-query training scheme for action generation. Experiments on standard VLN benchmarks show that ReflectVLN improves success rates and path efficiency under a constrained data budget, with favorable training cost and fewer high-level intention calls at inference time, while providing interpretable intermediate decisions for analysis and collaboration. Code is available at: https://github.com/AIprogrammer/ReflectVLN
In Vision-Language Navigation (VLN) tasks involving multi-step instructions as guidance, an agent must associate its observations with the relevant instruction content at each decision point. Most graph-based methods commonly match candidate trajectories with the entire instruction, allowing already executed content to interfere with subsequent decisions. Although instruction-execution progress provides a direct cue for separating executed and pending content, existing progress estimation approaches often rely on explicit progress supervision, sophisticated progress-learning stages, or costly online reasoning with LLMs. In this paper, we propose a robust VLN framework with Awareness of Instruction-Execution Progress (VLN-AIEP), which couples progress estimation with navigation decision-making. VLN-AIEP associates the visited trajectory with the instruction to suppress embeddings corresponding to executed content, and then matches candidate locations with the pending content. This coupling makes correct navigation decisions depend on accurate progress estimation, allowing the progress estimator to learn directly from navigation decision labels without explicit progress annotations. To support progress estimation, we construct a landmark-anchored trajectory representation that encodes the semantic labels and viewpoint-relative geometry of instruction-referred landmarks. These landmarks serve as spatial references for associating the visited trajectory with executed instruction content. Extensive experiments on R2R, RxR, CVDN, and R2R-CE demonstrate that the proposed method achieves favorable performance across multiple metrics.
Vision-Language Navigation (VLN) requires embodied agents to generate actions based on instructions and observations. General-purpose multimodal agents offer a promising basis for this task, but selecting plausible local actions does not ensure that execution remains consistent with the intended route, particularly in long-horizon tasks. Moreover, the accumulated interaction history increases the input required for subsequent decisions, resulting in a significant inference overhead. To this end, we introduce \method, an Agentic VLN framework that includes a Goal Agent that sets adaptive goals for local actions, a Verify Agent that dynamically verifies whether a goal has been completed, a Memory Agent for multimodal context compression, and a Visuomotor Agent to execute adaptive goals. Specifically, the Goal Agent formulates adaptive goals based on the instruction, current observation, and execution history. Then the Visuomotor Agent executes navigation actions to achieve each goal, while the Verify Agent uses a goal-specific verification question to dynamically assess whether the observed outcomes satisfy the intended completion condition. Verified goal completion then marks a boundary for the Memory Agent to compress the corresponding multimodal interaction history while preserving information needed for subsequent navigation. We evaluate navigation on R2R-CE and RxR-CE, examine framework variants across three model backbones, and study context evolution during execution. For Real-World evaluation, \method achieves 83.3% success and 1.51,m navigation error across eight challenging routes evaluated three times each.
Haoxiang Shi, Zaijing Li, Muhe Ding +3
Harbin Institute of Technology (Shenzhen) · Pengcheng Laboratory