Structure over Reasoning: Unified Zero-Shot Navigation with a Small Vision-Language Model

Zeqi Li; Xunyi Zhao; Zerui Li; Sihao Lin; Qi Wu; ShiJie Li

cs.ROOct 18, 2026

Structure over Reasoning: Unified Zero-Shot Navigation with a Small Vision-Language Model

Authors: Zeqi Li, Xunyi Zhao, Zerui Li, Sihao Lin, Qi Wu, ShiJie Li

Organizations: Technical University of Munich, Germany. · Adelaide University, Responsible AI Research Centre, Australian Institute for Machine Learning, Australia. · Institute for Infocomm Research (I2R), A*STAR, Singapore.

Abstract

Vision-language navigation is moving into photorealistic, physically simulated scenes, where the leading zero-shot agents put a vision-language model in the control loop to choose every move and stop. Robots favour small models, whose abilities are uneven: Qwen3-VL-4B can judge whether a single view looks like the goal, but its route reasoning is at chance, its distance estimates nearly constant, its stop votes unreliable. Our agent therefore asks it two perceptual questions per step, which direction looks like the goal and whether the view justifies stopping, plus one gated frontier query on route-bearing instructions, and treats the answers as evidence, not actions: a route-blind semantic field over a bird's-eye map accumulates the directional scores through a tempered update, a skeleton graph supplies frontiers and geodesic distances, and a two-stage terminator turns stop votes into termini through a trail-consistency gate and a budget-forced return. With every baseline re-run in one installation at one budget, the agent exceeds all four zero-shot baselines of the VLNVerse subset in mean success on both granularities, though the paired interval against the strongest coarse-grained baseline includes zero. Termination is the dominant lever in ablation, thirteen attempts to give the model more of the navigation establish no reliable gain, and the agent runs on a Unitree Go2 quadruped with off-board inference. Code, per-episode logs and a robot video will be released at https://anonymous.4open.science/r/structure-over-reasoning-PAGE.

Figures & tables

Fig. 1: One navigation step in four lanes. Inputs (grey): the instruction, twelve RGB views, the matching depth views and the pose. The model (orange): a five-point sector score per view, a stop vote on the best-scoring view and, on route-bearing instructions, a gated chain-of-thought query over the frontier list. Structure (blue): depth becomes a range scan and an occupancy map, the map a skeleton graph with frontier nodes, and the sector scores a route-blind belief field; the proposal enters the field through a gate closed on coarse-grained instructions, and a utility over the waypoint candidates picks the move. The terminator (green): a vote is accepted only within a tolerance $\delta$ of the best trail score; otherwise the agent walks back to that position when the budget runs short. The model, structure and terminator panels and the strip below them are real data from one fine-grained episode.

	Fine-grained instructions						Coarse-grained instructions
Method	TL	NE $\downarrow$	SR $\uparrow$	OSR $\uparrow$	SPL $\uparrow$	nDTW $\uparrow$	TL	NE $\downarrow$	SR $\uparrow$	OSR $\uparrow$	SPL $\uparrow$	nDTW $\uparrow$
Text-history [ 1 , 2 ]	45.57	9.30	15.96	56.91	3.95	30.34	46.64	6.49	31.03	71.45	5.24	39.81
Map-history [ 1 , 3 ]	48.27	8.86	10.90	58.24	2.06	31.35	47.99	4.98	35.64	79.08	4.48	43.78
Text-history + CoT [ 1 , 2 , 7 ]	30.91	6.52	26.33	53.99	10.11	39.26	27.22	4.71	45.39	67.02	14.90	46.72
Map-history + CoT [ 1 , 3 , 7 ]	26.83	5.93	27.13	53.01	10.62	41.89	19.66	4.63	40.07	56.03	16.18	46.74
Ours	28.13	4.88	36.20	55.73	14.04	42.15	31.18	3.80	51.61	70.97	15.07	45.13

TABLE I: Zero-shot results on the official VLNVerse subset at a thirty-step budget; best value per column in bold.

Terminator	SR $\uparrow$	SPL $\uparrow$	Steps	Vote	Return	$\Delta$ SR
Fine-grained instructions
Full agent	42.11	13.63	21.0	70%	30%	–
No stopping	20.97	2.45	40.0	0%	0%	$-$ 21.1
No trail gate	32.80	14.18	14.8	41%	13%	$-$ 9.3
No budget return	35.48	11.70	22.7	65%	0%	$-$ 6.6
Conservative prompt	41.94	11.48	26.2	54%	46%	$-0.2$

TABLE II: Terminator ablation; fine-grained rows at forty steps, coarse-grained at thirty.

Guidance	SR $\uparrow$	SPL $\uparrow$	OSR $\uparrow$	$\Delta$ SR	SR $\uparrow$	SPL $\uparrow$	OSR $\uparrow$	$\Delta$ SR
	Fine-grained				Coarse-grained
Field + route channel	42.11	13.63	58.96	–	51.61	15.07	70.97	–
Field only	36.56	11.68	53.23	$-$ 5.6	51.61	14.39	72.04	$0.0$
Route channel only	31.18	8.61	46.24	$-$ 10.9	45.16	10.43	70.43	$-6.5$
Neither	25.81	6.39	45.70	$-16.3$	44.62	9.22	69.89	$-7.0$

TABLE III: Guidance ablation; fine-grained at forty steps, coarse-grained at thirty.

Loses beyond the run-to-run spread
Attempt	SR	$\Delta$	Attempt	SR	$\Delta$
Twice reasoning budget	31.91	$-$ 10.2	Sub-goal decomposition	29.84	$-$ 12.3
Inside the run-to-run spread: no reliable gain observed
Frontier when silent	37.63	$-$ 4.5	Context-landmark grounding	37.63	$-$ 4.5
Route answer first	37.81	$-$ 4.3	Terminus by grounding	38.71	$-$ 3.4
Per-view final approach	39.25	$-$ 2.9	Room before object	40.86	$-$ 1.3

TABLE IV: Thirteen attempts to give the model more of the navigation, against 42.11 at forty steps.

Setting	SR $\uparrow$	SPL $\uparrow$	Vote
$\delta=0.02$	43.01	14.15	61%
$\delta=0.05$ , main	42.11	13.63	70%
$\delta=0.10$	37.63	12.98	76%
votes ignored for 3 steps, main	42.11	13.63	70%
votes ignored for 5 steps	37.10	10.83	70%
votes ignored for 10 steps	43.01	8.46	59%

TABLE V: Sensitivity of the gate tolerance

\delta

and the vote delay, fine-grained, forty steps.

Query	Calls per step	Tokens	Latency
Sector score, one view	12	3	62 ms
Stop vote	1	3	72 ms
Route proposal, text only	1	43	691 ms
Frontier caption, cached	$\leq 2$	7	124 ms
Ours, one step, captions cached	14	82	1.51 s
Ours, one step, two new captions	16	96	1.76 s

TABLE VI: Cost of one navigation step, model calls only, median of twelve runs on an idle RTX 5090; the baseline is timed with the same weights, decoding and image resolution.

	Fine-grained		Coarse-grained
Agent	SR $\uparrow$	NE $\downarrow$	SR $\uparrow$	NE $\downarrow$
Text-history + CoT	20	4.01	32	3.82
Map-history + CoT	24	3.83	32	3.80
Ours	36	3.77	44	3.06

TABLE VII: Real-robot results on a Unitree Go2, twenty-five episodes per granularity.

Visual Robot Navigation

See all 46 papers

LightZeroNav: Zero-Shot Vision Language Navigation in Continuous Environments Based on Lightweight VLMs
Mar 16, 2026Kun Luo, Xiangyu Dong, Xiaoguang Ma +2Vision-Language Navigation Vision-Language Models
Foshan Graduate School of Innovation, Northeastern University, Foshan 528311, China · Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China · QingniaoAI, China +1
C $^2$ Nav: Compare Before You Commit for Zero-Shot Vision-and-Language Navigation
Sep 14, 2026Runtian Zheng, Congpeng Zhang, Ying LiuVision-Language Navigation VLM Reasoning
North China University of Technology, Beijing, China
Three-Step Nav: A Hierarchical Global-Local Planner for Zero-Shot Vision-and-Language Navigation
Apr 29, 2026Wanrong Zheng, Yunhao Ge, Laurent IttiVision-Language Navigation
University of Southern California · 2NVIDIA Research

Mar 16, 2026J/K move · Enter open · S save

LightZeroNav: Zero-Shot Vision Language Navigation in Continuous Environments Based on Lightweight VLMs

Kun Luo, Xiangyu Dong, Xiaoguang Ma, Haoran Zhao+1

Foshan Graduate School of Innovation, Northeastern University, Foshan 528311, China · Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China · QingniaoAI, China+1

Vision-Language Navigation Vision-Language Models Memory-Augmented VLMs

Open paper AskPDF arXiv

Although vision-language navigation (VLN) has progressed rapidly, zero-shot VLN in continuous environments (VLN-CE) remains highly challenging when using lightweight vision-language models (VLMs), whose limited reasoning capacity makes long-horizon navigation unreliable. In this paper, we propose LightZeroNav to tackle the three major bottlenecks when using lightweight VLMs in zero-shot VLN-CE,i.e.,information redundancy from multi-source inputs, inaccurate progress estimation caused by noisy textual memory, and task entanglement between action execution and stage transition. Using only RGB observations and a lightweight open-source Qwen3-VL-8B backbone, LightZeroNav achieves competitive performance with GPT-4o (~200B) without task-specific training, graph search, or waypoint predictors, demonstrating its effectiveness in zero-shot VLN-CE.

Abstract

Visual Robot Navigation

Explore similar work

LightZeroNav: Zero-Shot Vision Language Navigation in Continuous Environments Based on Lightweight VLMs

C2^22Nav: Compare Before You Commit for Zero-Shot Vision-and-Language Navigation

Three-Step Nav: A Hierarchical Global-Local Planner for Zero-Shot Vision-and-Language Navigation

C $^2$ Nav: Compare Before You Commit for Zero-Shot Vision-and-Language Navigation