NavBind: Diagnosing Instruction-Action Binding at Route Forks in Navigation VLAs

Josh Qixuan Sun; Xiaoying Xing; Yurun Chen; Huaiyuan Weng; Wentao Zhu; Mark Crowley

cs.ROOct 21, 2026

NavBind: Diagnosing Instruction-Action Binding at Route Forks in Navigation VLAs

Authors: Josh Qixuan Sun, Xiaoying Xing, Yurun Chen, Huaiyuan Weng, Wentao Zhu, Mark Crowley

Organizations: University of Waterloo · Northwest University · Shanghai Jiao Tong University · Eastern Institute of Technology, Ningbo

Abstract

Vision-language-action (VLA) navigation models translate visual observations and instructions into actions. Navigation success assesses whether an agent reaches the goal. However, reaching the goal successfully does not establish whether the model genuinely follows the instructions or relies on visual cues or navigation history. We introduce NavBind to diagnose instruction-action binding: whether changing the instruction influences the probabilistic distribution of the actions, with the navigation context fixed. We construct 3,252 route-fork contexts by recombining route-aligned instruction segments from R2R-CE and RxR-CE. Each context pairs full instructions that share the description of the route already traveled but specify different branches. Both are evaluated under the same agent state and navigation history. We compare generated local action sequences, the likelihoods of fixed reference action sequences under both instructions, and changes in visual attention and attribution. Across the evaluated models, higher reported navigation success does not imply that the model more faithfully follows the instructions.

Figures & tables

Figure 1: Does the action follow the instruction? NavBind pairs one fixed navigation history $H$ with two full instructions $P+S_{A}$ and $P+S_{B}$ . The shared prefix $P$ describes the completed route, and the remaining suffixes $S_{A}$ and $S_{B}$ specify different executable branches. This example contrasts turning left with going up the stairs. We jointly examine generated action-sequence changes, fixed-target action likelihood, visual attention, and attribution for each instruction’s corresponding reference actions.

Figure 2: Route-aligned pair construction. (a) Original routes A/B (blue/orange) share a directed segment from

u

to fork

v

, then diverge. Dashed paths are traversed, and solid paths remain. Each full instruction

I_{h,h}=P_{h}+S_{h}

splits at

v

into completed prefix

P_{h}

and remaining suffix

S_{h}

. (b) Horizontal comparisons change only the suffix. Vertical comparisons change history and prefix together with the suffix fixed.

Figure 3: Action likelihood and visual attribution. (a) Conditional action-token probabilities for each fixed reference under the two suffixes. Arrows denote left, forward, and right actions. (b) Full attention $\times$ gradient heatmaps for each suffix’s corresponding reference. Warmer colors indicate higher attribution on a shared scale. IoU uses the original top-25% masks.

	RxR-CE	Behavior	Fixed-target likelihood			Visual overlap (IoU)
Model	SR (%) $\uparrow$	Same seq. (%) $\downarrow$	$\bar{d}\uparrow$	Both+ (%) $\uparrow$	KL $\uparrow$	Attn. $\downarrow$	Grad. $\downarrow$	Attn. $\times$ Grad. $\downarrow$
Uni-NaVid	56.1	85.79	0.0127	29.15	0.0124	0.860	0.454	0.773
StreamVLN	54.4	32.41	0.8360	80.38	0.8233	0.602	0.420	0.456
JanusVLN ∗	51.4	30.75	0.3643	72.88	0.4537	0.727	0.760	0.698
FutureNav ∗	54.5	25.49	0.4411	70.60	0.6208	0.728	0.786	0.702

Table 1: Reported navigation success and NavBind diagnostics. SR is the officially reported RxR-CE val-unseen success rate for each released checkpoint used in our experiments. Lower Same seq. and IoU indicate greater output and spatial change. Higher

\bar{d}

and Both+ indicate stronger or more frequent matching-suffix likelihood gains. Higher KL indicates greater action-distribution change. IoU uses top-25% masks. ∗ marks single-step models.

Output	N	$\bar{d}\uparrow$	Attn. IoU $\downarrow$	Grad. IoU $\downarrow$	Attn. $\times$ Grad. IoU $\downarrow$
Unchanged	1,054	0.3766	0.6754	0.4831	0.5128
Changed	2,198	1.0563	0.5673	0.3895	0.4292

Table 2: StreamVLN results for unchanged and changed action sequences under the two instructions.

Prefix tokens	Same seq. (%) $\downarrow$	$\bar{d}\uparrow$	KL $\uparrow$
$\leq 47$	30.80	0.8904	0.8964
$48$ - $76$	30.90	0.8492	0.8497
$\geq 77$	35.62	0.7660	0.7205

Table 3: StreamVLN results by prefix token count, with roughly equal group sizes.

Action pair	$N$	Same seq. (%) $\downarrow$	$\bar{d}\uparrow$	KL $\uparrow$
F/L	486	37.24	0.7426	0.6905
F/R	508	33.07	0.8404	0.8248
L/R	1,504	26.66	1.0135	0.9458

Table 4: StreamVLN results by action-pair group. F/L/R denote forward/left/right.

First differing GT position	$N$	Action position
First differing GT position	$N$	1	2	3	4
1	2,498	0.5911	0.5829	0.5687	0.5757
2	409	0.6103	0.5693	0.5736	0.5765
3	219	0.6569	0.6183	0.5777	0.6021
4	126	0.6792	0.6642	0.6373	0.6273

Table 5: StreamVLN attention IoU at each action position. Bold marks the lowest mean IoU in each row.

Source split	$N$	Original $\downarrow$	Recombined $\downarrow$	Gap
Train	2,367	0.4666	0.6007	+0.1341
Val-unseen	385	0.5763	0.6374	+0.0611

Table 6: StreamVLN mean reference-action NLL under original and recombined pairings. Gap is recombined minus original NLL.

Model	Same seq. (%) $\uparrow$	KL $\downarrow$	Attn. IoU $\uparrow$	Grad. IoU $\uparrow$	Attn. $\times$ Grad. IoU $\uparrow$
StreamVLN	51.42	0.3131	0.4295	0.3032	0.4088
JanusVLN ∗	25.18	0.3225	0.2675	0.2278	0.2543
FutureNav ∗	27.67	0.3150	0.3083	0.2548	0.2894

Table 7: Cross-history stability. With the suffix and reference actions fixed, higher Same seq. and IoU and lower KL indicate greater stability, opposite to Table 1 . ∗ marks single-step models.

Vision-Language-Action Models

Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.

See all 1,190 papers

GuidedNav: Shaping Vision-Language Navigation Representations through Pre-action Attention and Spatial Guidance
Oct 9, 2026Jingyu Guo, Jiaxin Huang, Chengxing Lin +6Vision-Language Navigation Visual Navigation
University of Melbourne · AgiBot · Mohamed bin Zayed University of Artificial Intelligence +3
Do VLAs Really Follow Instructions? Diagnosing How-Level Instruction Following in Vision-Language-Action Models
Oct 15, 2026Sangwu Park, Wonjoong Kim, Yeonjun In +1Vision-Language-Action Models VLA Model Training
KAIST
From Routes to Steps: Separating Semantic Progress from Local Execution in Vision-and-Language Navigation
Aug 4, 2026Xiangyun Huang, Xiangchen Wang, Runfeng Lin +5Vision-Language Navigation VLMs for Robotics
1Beihang University · 2Southern University of Science and Technology · 3Central South University +2

Oct 9, 2026J/K move · Enter open · S save

GuidedNav: Shaping Vision-Language Navigation Representations through Pre-action Attention and Spatial Guidance

Jingyu Guo, Jiaxin Huang, Chengxing Lin, Xiaoyu Luo+5

University of Melbourne · AgiBot · Mohamed bin Zayed University of Artificial Intelligence+3

Vision-Language Navigation Visual Navigation

Open paper AskPDF arXiv

Vision-and-language navigation (VLN) requires an agent to track progress through an extended instruction, identify landmarks relevant to its current objective, and understand the surrounding spatial structure to act. End-to-end navigation policies typically rely on action supervision to implicitly learn these capabilities. Without explicit guidance on the evidence underlying each decision, however, policies can overfit to correlations predictive of training actions rather than learn the task-relevant grounding and geometry needed for generalization. Our insight is to guide the representations that connect instructions to actions: identifying what remains to be done determines which visual evidence matters, while understanding the geometry around that evidence informs how to move. We introduce GuidedNav, a unified framework that explicitly shapes these representations through training-time guidance. Subtask and landmark guidance steer native pre-action attention toward the active instruction segment and relevant visual regions, grounding action prediction in the current objective. Spatial guidance aligns the policy's internal representations with geometric features that capture long-horizon context and local spatial structure. Together, these objectives strengthen navigation capabilities while avoiding the inference overhead of intermediate textual reasoning and online geometric encoding. We also introduce Guided-R2R and Guided-RxR, two annotation-enriched navigation datasets that support more interpretable VLN research. GuidedNav establishes state-of-the-art performance among the compared monocular RGB policies, improving success rates by 8.1% and 13.1% relative to the strongest prior results on R2R-CE and RxR-CE val-unseen, respectively, while achieving faster inference. Extensive experiments further demonstrate the effectiveness and real-world transferability of our approach.

Abstract

Vision-Language-Action Models

Explore similar work

GuidedNav: Shaping Vision-Language Navigation Representations through Pre-action Attention and Spatial Guidance

Do VLAs Really Follow Instructions? Diagnosing How-Level Instruction Following in Vision-Language-Action Models

From Routes to Steps: Separating Semantic Progress from Local Execution in Vision-and-Language Navigation