SFCo-Nav: Efficient Zero-Shot Visual Language Navigation via Collaboration of Slow LLM and Fast Attributed Graph Alignment

Chaoran Xiong; Litao Wei; Xinhao Hu; Kehui Ma; Ziyi Xia; Zixin Jiang; Zhen Sun; Ling Pei

cs.ROMar 2, 2026

SFCo-Nav: Efficient Zero-Shot Visual Language Navigation via Collaboration of Slow LLM and Fast Attributed Graph Alignment

Authors: Chaoran Xiong, Litao Wei, Xinhao Hu, Kehui Ma, Ziyi Xia, Zixin Jiang, Zhen Sun, Ling Pei

Organizations: The authors are with the Shanghai Key Laboratory of Navigation and Location Based Services, Shanghai Jiao Tong University (SJTU), Shanghai 200240, China. · Litao Wei is also with Zhiyuan College, SJTU. · Ling Pei is also with the State Key Laboratory of Submarine Geoscience, SJTU.

Abstract

Recent advances in large vision-language models (VLMs) and large language models (LLMs) have enabled zero-shot approaches to visual language navigation (VLN), where an agent follows natural language instructions using only ego perception and reasoning. However, existing zero-shot methods typically construct a naive observation graph and perform per-step VLM-LLM inference on it, resulting in high latency and computation costs that limit real-time deployment. To address this, we present SFCo-Nav, an efficient zero-shot VLN framework inspired by the principle of slow-fast cognitive collaboration. SFCo-Nav integrates three key modules: 1) a slow LLM-based planner that produces a strategic chain of subgoals, each linked to an imagined object graph; 2) a fast reactive navigator for real-time object graph construction and subgoal execution; and 3) a lightweight asynchronous slow-fast bridge aligns advanced structured, attributed imagined and perceived graphs to estimate navigation confidence, triggering the slow LLM planner only when necessary. To the best of our knowledge, SFCo-Nav is the first slow-fast collaboration zero-shot VLN system supporting asynchronous LLM triggering according to the internal confidence. Evaluated on the public R2R and REVERIE benchmarks, SFCo-Nav matches or exceeds prior state-of-the-art zero-shot VLN success rates while cutting total token consumption per trajectory by over 50% and running more than 3.5 times faster. Finally, we demonstrate SFCo-Nav on a legged robot in a hotel suite, showcasing its efficiency and practicality in indoor environments.

Figures & tables

Fig. 1: SFCo-Nav is an efficient zero-shot VLN framework inspired by slow–fast cognitive collaboration. It comprises three modules: 1) a slow brain LLM-based planner; 2) a fast brain reactive navigator; and 3) a lightweight asynchronous slow–fast bridge that aligns the imagined and perceived graphs, estimates navigation confidence, and triggers LLM only when necessary. This design minimizes costly LLM calls while preserving high navigation success.

Fig. 2: System overview of SFCo‑Nav, a slow–fast collaborative framework for efficient zero-shot visual language navigation. The Slow LLM Planner ( $\Pi_{\mathrm{slow}}$ ) decomposes the navigation instruction into subgoals, each paired with an imagined object graph $G^{i}_{t}$ . The Fast Reactive Navigator ( $\pi_{\mathrm{fast}}$ ) builds a perceived object graph $G^{p}_{t}$ in real time and executes low-level actions to align with $G^{i}_{t}$ . The Slow–Fast Bridge computes the graph-alignment confidence $C_{t}$ ; high confidence ( $C_{t}>\tau_{C}$ ) keeps control with $\pi_{\mathrm{fast}}$ , while low confidence ( $C_{t}\leq\tau_{C}$ ) triggers replanning by $\Pi_{\mathrm{slow}}$ .

Fig. 3: Slow LLM Planner prompt structure and operation process.

Fig. 4: Observed and imagined attributed graph structure.

Method	Task Success Metric					Token Efficiency Metric			Time Efficiency Metric[s]
Method	TL	NE	OSR	SR	SPL	V-Tok	L-Tok	U-Tok	V-Time	L-Time	T-Time
NavGPT (GPT-4)	11.45	6.46	42.0	34.0	29.0	108k	64.62k	496.62k	1440	48.19	1488.19
MapGPT (GPT-4)	–	6.29	57.6	38.8	25.8	4.95k	9.46k	29.26k	66	41.01	107.01
NavCoT (Tuned ∗ Llama)	9.83	6.67	44.0	36.4	33.17	4.50k	1.42k	19.42k	60	16.42	76.42
SF-Nav (GPT-4o)	10.89	5.70	53.8	41.3	36.35	0.9k	13.10k	16.10k	3.36	36.0	39.36
SFCo-Nav (GPT-4o)	10.80	6.04	50.5	38.2	32.54	0.9k	9.94k	13.54k	3.36	22.8	26.16

TABLE I: Comparison on the R2R Dataset.

Method	Task Success Metric					Token Efficiency Metric			Time Efficiency Metric[s]
Method	TL	NE	OSR	SR	SPL	V-Tok	L-Tok	U-Tok	V-Time	L-Time	T-Time
NavGPT (GPT-4)	–	–	28.3	19.2	14.6	108.00k	57.35k	489.35k	1440	49.74	1489.74
MapGPT (GPT-4)	–	–	42.6	28.4	14.5	4.95k	7.56k	27.36k	66	26.57	92.57
NavCoT (Tuned ∗ Llama)	12.36	–	14.20	9.20	7.18	4.50k	1.44k	19.44k	60	11.40	71.40
SF-Nav (GPT-4o)	10.05	7.87	46.63	35.87	30.62	0.9k	14.04k	17.64k	3.36	31.4	34.76
SFCo-Nav (GPT-4o)	10.32	7.16	42.36	31.33	27.01	0.9k	9.44k	13.04k	3.36	21.6	24.96

TABLE II: Comparison on the REVERIE Dataset.

Fig. 5: Real-world hotel suite deployment of SFCo-Nav. Early in navigation, sparse observations yield low match probability, triggering the slow planner. As observations grow, confidence exceeds the threshold, enabling fast, LLM-free execution. This slow-fast collaboration preserves success while improving time efficiency and reducing token usage.

SFCo-Nav- Confidence Threshold	Task Success Metric			Efficiency Metric
SFCo-Nav- Confidence Threshold	SR	OSR	SPL	U-Tok	T-Time[s]
SFCo-Nav-1.0	39.67	49	37.08	12.58k	28.08
SFCo-Nav-0.95	36.67	50.33	34.10	10.55k	22.5
SFCo-Nav-0.85	36.67	50.33	34.03	10.42k	22.2
SFCo-Nav-0.6	33.33	48.33	31.22	9.77k	20.34
SFCo-Nav-0.4	29.67	45	27.39	9.51k	19.64

TABLE III: Comparison of Various Confidence level on R2R Subset.

LLMs	Chain Decision	Object-Skill Pair	SR	U-Tok	T-Time[s]
GPT-4o	$\times$	$\times$	32	13.29k	32.90
	✓	$\times$	31.33	10.13k	28.73
	$\times$	✓	45.67	13.06k	31.05
	✓	✓	37	9.9k	27.47
Deepseek-V3.1	$\times$	$\times$	32.67	15.07k	99
	✓	$\times$	31	12.87k	96.84

TABLE IV: Ablation Study of SFCo-Nav on R2R Subsets.

Vision-Language Navigation

Vision-Language Navigation studies how embodied agents use natural-language instructions and visual observations to plan and execute routes.

See all 290 papers

One MLLM, One Call: Efficient Zero-Shot Vision-and-Language Navigation via Spatial-Aware Waypoints
Sep 6, 2026Shiqi Pan, Qi Zheng, Hanqin Sun +3Vision-Language Navigation VLMs for Robotics
College of Electronics and Information Engineering, Shenzhen University, Shenzhen 518060, China · School of Computer Science, The University of Sydney, Darlington 2008, Australia · College of Computer Science and Software Engineering, Shenzhen University, Shenzhen 518060, China
LightZeroNav: Zero-Shot Vision Language Navigation in Continuous Environments Based on Lightweight VLMs
Mar 16, 2026Kun Luo, Xiangyu Dong, Xiaoguang Ma +2Vision-Language Navigation Vision-Language Models
Foshan Graduate School of Innovation, Northeastern University, Foshan 528311, China · Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China · QingniaoAI, China +1
AgenticNav: Zero-Shot Vision-and-Language Navigation as a Tool-Calling Harness
Jun 9, 2026Yijian Li, Changze Li, Hantian Shi +4Vision-Language Navigation Tool-Using Vision-Language Agents
Shanghai Jiao Tong University · Huawei Technologies Ltd.

Sep 6, 2026J/K move · Enter open · S save

One MLLM, One Call: Efficient Zero-Shot Vision-and-Language Navigation via Spatial-Aware Waypoints

Shiqi Pan, Qi Zheng, Hanqin Sun, Youjian Zhang+2

College of Electronics and Information Engineering, Shenzhen University, Shenzhen 518060, China · School of Computer Science, The University of Sydney, Darlington 2008, Australia · College of Computer Science and Software Engineering, Shenzhen University, Shenzhen 518060, China

Vision-Language Navigation VLMs for Robotics Efficient VLM Inference

Open paper AskPDF arXiv

Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires an embodied agent to navigate unseen environments by following natural language instructions. Current zero-shot VLN-CE methods either rely on pre-trained waypoint predictors or require multiple queries to large models per step. To address prohibitive inference latency and computational overhead, we propose O2C-Nav, an efficient zero-shot navigation framework that calls only a single large model once per decision step. Our approach introduces a training-free structured waypoint generator and a novel abstract representation that projects sparse, history-aware candidate waypoints directly onto RGB images as visual markers. The MLLM selects a waypoint or generates a fallback target bounding box at each step, while a low-level Fast Marching Method (FMM) planner converts the selected target into an executable collision-free path. This paradigm provides the model with concrete spatial perception and explicit memory while significantly reducing the visual processing load. Extensive evaluations on the R2R-CE and RxR-CE benchmarks demonstrate that O2C-Nav outperforms current state-of-the-art zero-shot methods, highlighting its great potential for real-time robotic deployment. Code is available at https://github.com/kkpsq/O2C-Nav-Code.

Abstract

Vision-Language Navigation

Explore similar work

One MLLM, One Call: Efficient Zero-Shot Vision-and-Language Navigation via Spatial-Aware Waypoints

LightZeroNav: Zero-Shot Vision Language Navigation in Continuous Environments Based on Lightweight VLMs

AgenticNav: Zero-Shot Vision-and-Language Navigation as a Tool-Calling Harness