MATCH: Execution-Aware Hierarchical Reinforcement Learning for Vision-and-Language Navigation in Continuous Environment

Shuhao Ye; Yufei Wei; Demao Wang; Shichao Zhai; Rong Xiong; Yue Wang

cs.ROOct 28, 2026

MATCH: Execution-Aware Hierarchical Reinforcement Learning for Vision-and-Language Navigation in Continuous Environment

Authors: Shuhao Ye, Yufei Wei, Demao Wang, Shichao Zhai, Rong Xiong, Yue Wang

Organizations: Zhejiang University, Hangzhou, China

Abstract

Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires instruction-consistent planning and reliable execution under collision constraints. Yet many hierarchical approaches optimize the planner while keeping execution fixed. We introduce MATCH, a framework that jointly learns a language-conditioned planner and a goal-conditioned primitive-action executor. An execution-aware topological graph represents start-dependent execution options and integrates executor value estimates with realized outcome memory, enabling the planner to select execution requests and refine their targets through bounded continuous offsets. A transition-aware dense macro reward balances ordered progress along the reference route against execution residual, encouraging executable requests without substituting local success for instruction following. A staged curriculum culminates in synchronized on-policy updates, with the executor learning on planner-generated requests and the planner adapting its requests to the executor's current behavior. With 0.5B parameters, MATCH achieves state-of-the-art performance on the R2R-CE and RxR-CE benchmarks. Preliminary real-robot deployments further demonstrate its practical potential.

Figures & tables

Fig. 1: Comparison of three Hierarchical paradigms in VLN-CE.

Fig. 2: Overview of MATCH. The language-conditioned planner issues execution requests from an execution-aware graph, while the goal-conditioned executor realizes them with primitive actions. Macro- and micro-level experiences from the same rollouts are used for synchronized updates, enabling mutual adaptation between planning and execution.

Method	Params	Training	Planner output	Executor	Observation	R2R-CE Val-Unseen				RxR-CE Val-Unseen
Method	Params	Training	Planner output	Executor	Observation	NE $\downarrow$	OS $\uparrow$	SR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	SR $\uparrow$	SPL $\uparrow$	nDTW $\uparrow$
ETPNav [ 2 ]	0.3B	IL	frontier	rotate-then-fwd.	pano. RGB-D+odo.	4.71	65.0	57.0	49.0	5.64	54.8	44.9	61.9
HNR [ 8 ]	0.3B	IL	frontier	rotate-then-fwd.	pano. RGB-D+odo.	4.42	67.0	61.0	51.0	5.51	56.4	46.7	63.6
Revisiting [ 5 ]	0.5B	IL+RL	frontier	rotate-then-fwd.	pano. RGB-D+odo.	3.89	72.6	68.1	57.3	5.07	62.3	50.2	66.8
VLN-R1 [ 25 ]	7B	IL+RL	primitives	–	mono RGB	7.00	41.2	30.2	21.8	10.4	22.3	17.5	–
StreamVLN [ 26 ]	7B	IL	primitives	–	mono RGB	4.98	64.2	56.9	51.9	6.22	52.9	46.0	61.9

TABLE I: Comparison on the val-unseen splits of R2R-CE and RxR-CE.

\ddagger

: reproduced with the released checkpoint the rotate-then-forward executor, since the original evaluation follows the predicted waypoint trajectory by teleportation.

Variant	SR $\uparrow$	nDTW $\uparrow$	$\Delta$ SR	$\Delta$ nDTW	Time
MATCH	67.7	70.0	–	–	$\sim$ 120
Training (Sec. III-B , IV )
w/o joint HRL	61.8	65.1	$-5.9$	$-4.9$	–
heuristic executor	64.3	66.9	$-3.4$	$-3.1$	$\sim$ 50
learned executor, frozen	65.6	68.0	$-2.1$	$-2.0$	$\sim$ 110
alternating updates	67.0	69.8	$-0.7$	$-0.2$	$\sim$ 200

TABLE II: Ablations on RxR-CE val-unseen.

\Delta

is relative to MATCH; time is the wall-clock of the joint stage in hours.

Fig. 3: Training dynamics during Stage 3 joint HRL. The x-axis in all panels denotes the training step. (a) Planner and executor returns, together with the change in success rate ( $\Delta\mathrm{SR}$ ) on RxR-CE val-unseen. (b) Planner-proposed target distance $\rho_{0}$ and the execution residual $\mathrm{res}_{t}$ after each macro action. (c) The two dense components of the macro reward: normalized ordered-route coverage increment reward, $\sum_{t}\Delta_{t}/L$ , and normalized execution-residual penalty, $-\sum_{t}\mathrm{res}_{t}/L$ . Together, these curves illustrate how the planner gradually balances target ambition with executability as the planner and executor co-adapt.

Fig. 4: Comparative case study on RxR-CE val-unseen.

Fig. 5: Real robot deployment. We deployed MATCH on a robot modified from TurtleBot 4 and tested it across four episodes.

Module	Size	Init.	S1	S2	S3
Shared encoders
RGB encoder	SigLIP2 ViT-B/16	pretrained	$\circ$	$\circ$	$\circ$
Depth encoder	ResNet-50, 512-d	DD-PPO	$\circ$	$\circ$	$\circ$
Planner
Instruction encoder	InfoXLM-base	pretrained	$\checkmark$	$\checkmark$	$\checkmark$
Panorama encoder	2 layers, 768	random	$\checkmark$	$\checkmark$	$\checkmark$

TABLE I: Modules of MATCH and their status in the three training stages:

\checkmark

trained,

\circ

frozen, – not used.

Vision-Language Navigation

Vision-Language Navigation studies how embodied agents use natural-language instructions and visual observations to plan and execute routes.

See all 313 papers

Robust Vision-Language Navigation with Awareness of Instruction-Execution Progress
Oct 9, 2026Jiagan Huang, Zikun Zhou, Hongpeng Wang +3Vision-Language Navigation
Harbin Institute of Technology, Shenzhen
Revisiting Topological Graphs for Macro Action based Closed-loop Reinforcement Learning of Vision Language Navigation in Continuous Environment
Sep 3, 2026Shuhao Ye, Sitong Mao, Yuxiang Cui +7Robot Navigation Hierarchical RL
Zhejiang University, Hangzhou, China · Huawei Technologies Co., Ltd · Zhejiang Humanoid Robot Innovation Center, Ningbo, China
NavHarness: Adaptive Goals for Agentic Vision-Language Navigation
Sep 30, 2026Haoxiang Shi, Zaijing Li, Muhe Ding +3Vision-Language Navigation Multimodal Agents
Harbin Institute of Technology (Shenzhen) · Pengcheng Laboratory

Oct 9, 2026J/K move · Enter open · S save

Robust Vision-Language Navigation with Awareness of Instruction-Execution Progress

Jiagan Huang, Zikun Zhou, Hongpeng Wang, Guangming Lu+2

Harbin Institute of Technology, Shenzhen

Vision-Language Navigation

Open paper AskPDF arXiv

In Vision-Language Navigation (VLN) tasks involving multi-step instructions as guidance, an agent must associate its observations with the relevant instruction content at each decision point. Most graph-based methods commonly match candidate trajectories with the entire instruction, allowing already executed content to interfere with subsequent decisions. Although instruction-execution progress provides a direct cue for separating executed and pending content, existing progress estimation approaches often rely on explicit progress supervision, sophisticated progress-learning stages, or costly online reasoning with LLMs. In this paper, we propose a robust VLN framework with Awareness of Instruction-Execution Progress (VLN-AIEP), which couples progress estimation with navigation decision-making. VLN-AIEP associates the visited trajectory with the instruction to suppress embeddings corresponding to executed content, and then matches candidate locations with the pending content. This coupling makes correct navigation decisions depend on accurate progress estimation, allowing the progress estimator to learn directly from navigation decision labels without explicit progress annotations. To support progress estimation, we construct a landmark-anchored trajectory representation that encodes the semantic labels and viewpoint-relative geometry of instruction-referred landmarks. These landmarks serve as spatial references for associating the visited trajectory with executed instruction content. Extensive experiments on R2R, RxR, CVDN, and R2R-CE demonstrate that the proposed method achieves favorable performance across multiple metrics.

Abstract

Vision-Language Navigation

Explore similar work

Robust Vision-Language Navigation with Awareness of Instruction-Execution Progress

Revisiting Topological Graphs for Macro Action based Closed-loop Reinforcement Learning of Vision Language Navigation in Continuous Environment

NavHarness: Adaptive Goals for Agentic Vision-Language Navigation