Global Commander and Local Operative: A Dual-Agent Framework for Scene Navigation

Kaiming Jin; Yuefan Wu; Shengqiong Wu; Bobo Li; Shuicheng Yan; Tat-Seng Chua

cs.CVFeb 21, 2026

Global Commander and Local Operative: A Dual-Agent Framework for Scene Navigation

Authors: Kaiming Jin, Yuefan Wu, Shengqiong Wu, Bobo Li, Shuicheng Yan, Tat-Seng Chua

Organizations: National University of Singapore · Simon Fraser University · University of Oxford

Abstract

Vision-and-Language Scene navigation is a fundamental capability for embodied human-AI collaboration, requiring agents to follow natural language instructions to execute coherent action sequences in complex environments. Existing approaches either rely on multiple agents, incurring high coordination and resource costs, or adopt a single-agent paradigm, which overloads the agent with both global planning and local perception, often leading to degraded reasoning and instruction drift in long-horizon settings. To address these issues, we introduce DACo, a planning-grounding decoupled architecture that disentangles global deliberation from local grounding. Concretely, it employs a Global Commander for high-level strategic planning and a Local Operative for egocentric observing and fine-grained execution. By disentangling global reasoning from local action, DACo alleviates cognitive overload and improves long-horizon stability. The framework further integrates dynamic subgoal planning and adaptive replanning to enable structured and resilient navigation. Extensive evaluations on R2R, REVERIE, and R4R demonstrate that DACo achieves 4.9%, 6.5%, 5.4% absolute improvements over the best-performing baselines in zero-shot settings, and generalizes effectively across both closed-source (e.g., GPT-4o) and open-source (e.g., Qwen-VL Series) backbones. DACo provides a principled and extensible paradigm for robust long-horizon navigation. Project page: https://github.com/ChocoWu/DACo

Figures & tables

Figure 1: Comparison of different agentic systems for scene navigation. Multi-agent systems rely on multiple experts, resulting in high coordination overhead and resource costs. Single-agent systems must handle both global planning and local perception, overloading the decision process. In contrast, our dual-agent framework assigns clear and complementary roles to global and local agents, simplifying system design and enabling more robust navigation reasoning.

Figure 2: Overview of the DACo framework. Our system comprises two collaborative components: (1) Global Agent : Acting as a high-level strategic planner, the Global Agent maintains a panoramic perspective by integrating current location descriptions, historical trajectories, and Top-down maps. It iteratively generates a dynamic global plan to guide the navigation. (2) Local Agent : Serving as the low-level executor, the Local Agent initiates each time step by synthesizing local observations into a concise environment description and issuing a planning or re-planning request. Upon receiving the global guidance, it grounds the high-level plan into primitive navigation actions to interact with the environment.

Figure 3: The illustration of the self-correction capability inherent in DACo system. Our framework achieves robust self-correction through two complementary processes: (1) Dynamic Planning : The Global Agent monitors the Local Agent’s trajectory via a top-down view; upon detecting a deviation, it adaptively refines the global plan to rectify the path. (2) Replan Mechanism : The Local Agent cross-checks the global plan’s validity against its immediate visual observations. If an inconsistency is detected (e.g., a missing landmark), a re-planning cycle is proactively triggered to resolve the discrepancy.

Methods	Backbone	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$
NavGPT [ 36 ]	GPT-3.5	8.02	26.4	16.7	13.0
DiscussNav [ 37 ]	GPT-4	6.30	51.0	37.5	33.3
MapGPT [ 41 ]	GPT-4o	5.45	55.1	44.6	35.4
MSNav [ 42 ]	Qwen-SP	5.02	63.9	50.9	42.6
DACo (Ours)	GPT-4o	5.86	63.7	50.5	39.7

Table 1: Results on 72 various scenes of the R2R dataset. MapGPT is reproduced using GPT-4o for fair comparison. Bold denotes the best result in the training-free method. Light color denotes the method with the fine-tuning stage

Settings	Methods	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$
Train	Seq2Seq [ 12 ]	7.81	28	21	-
	Speaker [ 19 ]	6.62	45	35	-
	EnvDrop [ 45 ]	5.22	-	52	48
Pretrain	PREVALENT [ 66 ]	4.71	-	58	53
	RecBERT [ 26 ]	3.93	69	63	57
	HAMT [ 24 ]	2.29	73	66	61

Table 2: Results on R2R validation unseen set. Bold denotes the best result in the training-free method. * denotes the zero-shot version provided by the original paper. Light color denotes the method with the fine-tuning stage.

Settings	Methods	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$
Train	Seq2Seq [ 12 ]	-	8.1	4.2	2.8
	RCM [ 13 ]	-	14.2	9.3	7.0
	FAST-MATTN [ 11 ]	-	28.2	14.4	7.2
Pretrain	RecBERT [ 26 ]	-	35.0	30.7	24.9
	HAMT [ 24 ]	-	35.4	31.6	29.6
	DUET [ 25 ]	-	51.0	46.9	33.7

Table 3: Results on REVERIE validation unseen set. Bold denotes the best result in the training-free method. Methods in “Train", “Pretrain", and “LLM-Ft" settings are evaluated on the full set. NavGPT and MapGPT are reproduced using GPT-4o.

Methods	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$
NavGPT [ 36 ]	9.90	44.0	15.0	14.1
MapGPT [ 41 ]	8.94	54.8	15.1	13.0
DACo (Ours)	8.91	55.0	20.5	17.4

Table 4: Comparison with zero-shot state-of-the-art methods on R4R validation unseen subset. All methods are implemented with Qwen3-VL-8B for fair comparison.

Plan Style	Replan	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$
$\times$	$\times$	6.44	63.5	45.7	35.1
static	$\checkmark$	5.79	64.3	46.9	36.4
dynamic	$\times$	6.2	61.6	47.5	38.2
dynamic	$\checkmark$	5.86	63.7	50.5	39.7

Table 5: Ablation on 72 various scenes of the R2R dataset.

Figure 4: Impact of Agent Backbone. All methods are implemented using Qwen2.5-VL-32B, Qwen3-VL-8B, and GPT-4o. The zero success rate of NavGPT on Qwen2.5-VL is primarily due to catastrophic format parsing failures.

Method	Time $\downarrow$	Prompt Token $\downarrow$	Completion Token $\downarrow$	Latency $\downarrow$
NavGPT [ 36 ]	106.14	$\mathbf{3.56\times 10^{4}}$	$2.79\times 10^{3}$	7.0
MapGPT [ 41 ]	63.24	$5.33\times 10^{4}$	$\mathbf{1.76\times 10^{3}}$	6.1
DACo w/ static plan	95.48	$4.72\times 10^{4}$	$1.99\times 10^{3}$	6.8

Table 6: Analysis of efficiency and API cost. All experiments are carried out with Qwen3-VL-8B on the REVERIE subset. Time and Prompt/Completion Token denote the average per task, while Latency refers to the average per call.

Figure 5: Results corresponding to samples of different steps. MapGPT is reproduced with GPT-4o.

Method	SR				OSR
Method	Mean $\uparrow$	Range $\downarrow$	SD $\downarrow$	CV $\downarrow$	Mean $\uparrow$	Range $\downarrow$	SD $\downarrow$	CV $\downarrow$
NavGPT	25.5	4.00	2.11	8.22%	35.83	4.00	2.08	5.81%
MapGPT	30.50	5.00	2.65	8.67%	43.83	6.00	3.33	7.60%
DACo	32.50	3.00	1.50	4.62%	49.00	5.00	2.65	5.40%

Table 7: Analysis of stochasticity. All experiments are conducted on the REVERIE subset using Qwen3-VL-8B.

Figure 6: A case demonstrating the effectiveness of the replanning mechanism. Following a replan, DACo successfully recovers from an off-track trajectory. Blue indicates the historical trajectory, red represents the deviated trajectory, and green denotes the corrected, target trajectory.

Figure 7: A comparison case between baseline and our method showcasing the long-horizon navigation capability. While baseline often suffers from orientation loss and trajectory drift, our approach maintains higher coherence and robustness. Blue indicates the historical trajectory, red represents the deviated trajectory, and green denotes the corrected, target trajectory.

Figure 8: A comparison case between baseline and our method showcasing the strong spatial layout understanding.

Figure 9: A comparison case between baseline and our method showcasing the multi-level navigation capability.

Vision-Language Navigation

Vision-Language Navigation studies how embodied agents use natural-language instructions and visual observations to plan and execute routes.

See all 310 papers

NavHarness: Adaptive Goals for Agentic Vision-Language Navigation
Sep 30, 2026Haoxiang Shi, Zaijing Li, Muhe Ding +3Vision-Language Navigation Multimodal Agents
Harbin Institute of Technology (Shenzhen) · Pengcheng Laboratory
AgentVLN: Towards Agentic Vision-and-Language Navigation
Mar 18, 2026Zihao Xin, Wentong Li, Yixuan Jiang +6Vision-Language Navigation Robot Navigation
Nanjing University of Aeronautics and Astronautics · Shandong University · Zhejiang University
DA-Nav: Direction-Aware City-Scale Vision-Language Navigation
Jul 13, 2026Ye Yuan, Kehan Chen, Xinqiang Yu +7Embodied Navigation Vision-Language Navigation
School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China · National Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences, Beijing, China +3

Sep 30, 2026J/K move · Enter open · S save

NavHarness: Adaptive Goals for Agentic Vision-Language Navigation

Haoxiang Shi, Zaijing Li, Muhe Ding, Xiang Deng+2

Harbin Institute of Technology (Shenzhen) · Pengcheng Laboratory

Vision-Language Navigation Multimodal Agents

Open paper AskPDF arXiv

Vision-Language Navigation (VLN) requires embodied agents to generate actions based on instructions and observations. General-purpose multimodal agents offer a promising basis for this task, but selecting plausible local actions does not ensure that execution remains consistent with the intended route, particularly in long-horizon tasks. Moreover, the accumulated interaction history increases the input required for subsequent decisions, resulting in a significant inference overhead. To this end, we introduce \method, an Agentic VLN framework that includes a Goal Agent that sets adaptive goals for local actions, a Verify Agent that dynamically verifies whether a goal has been completed, a Memory Agent for multimodal context compression, and a Visuomotor Agent to execute adaptive goals. Specifically, the Goal Agent formulates adaptive goals based on the instruction, current observation, and execution history. Then the Visuomotor Agent executes navigation actions to achieve each goal, while the Verify Agent uses a goal-specific verification question to dynamically assess whether the observed outcomes satisfy the intended completion condition. Verified goal completion then marks a boundary for the Memory Agent to compress the corresponding multimodal interaction history while preserving information needed for subsequent navigation. We evaluate navigation on R2R-CE and RxR-CE, examine framework variants across three model backbones, and study context evolution during execution. For Real-World evaluation, \method achieves 83.3% success and 1.51,m navigation error across eight challenging routes evaluated three times each.

Abstract

Vision-Language Navigation

Explore similar work

NavHarness: Adaptive Goals for Agentic Vision-Language Navigation

AgentVLN: Towards Agentic Vision-and-Language Navigation

DA-Nav: Direction-Aware City-Scale Vision-Language Navigation