DeCoNav: Dialog enhanced Long-Horizon Collaborative Vision-Language Navigation

Sunyao Zhou; Yunzi Wu; Tianhang Wang; Xinhai Li; Guang Chen; Lizheng Liu; Chenjia Bai; Xuelong Li

cs.ROApr 14, 2026

DeCoNav: Dialog enhanced Long-Horizon Collaborative Vision-Language Navigation

Authors: Sunyao Zhou, Yunzi Wu, Tianhang Wang, Xinhai Li, Guang Chen, Lizheng Liu, Chenjia Bai, Xuelong Li

Organizations: Institute of Artificial Intelligence (TeleAI), China Telecom · Fudan University · Tongji University

Abstract

Long-horizon collaborative vision-language navigation (VLN) is critical for multi-robot systems to accomplish complex tasks beyond the capability of a single agent. CoNavBench takes a first step by introducing the first collaborative long-horizon VLN benchmark with relay-style multi-robot tasks, a collaboration taxonomy, along with graph-grounded generation and evaluation to model handoffs and rendezvous in shared environments. However, existing benchmarks and evaluations often do not enforce strictly synchronized dual-robot rollout on a shared world timeline, and they typically rely on static coordination policies that cannot adapt when new cross-agent evidence emerges. We present Dialog enhanced Long-Horizon Collaborative Vision-Language Navigation (DeCoNav), a decentralized framework that couples event-triggered dialogue with dynamic task allocation and replanning for real-time, adaptive coordination. In DeCoNav, robots exchange compact semantic states via dialogue without a central controller. When informative events such as new evidence, uncertainty, or conflicts arise, dialogue is triggered to dynamically reassign subgoals and replan under synchronized execution. Implemented in DeCoNavBench with 1,213 tasks across 176 HM3D scenes, DeCoNav improves the both-success rate (BSR) by 69.2%, demonstrating the effectiveness of dialogue-driven, dynamically reallocated planning for multi-robot collaboration.

Figures & tables

Fig. 1: Overview of DeCoNav and DeCoNavBench. Module 1: The ROVE pipeline constructs verified episodes through rule-based inference, VLM classification, human adjudication (RTSA), and triple-gate target verification (TriGate). Module 2: DeCoNav coordinates dual-robot execution via three coupled components: Semantic Visual Bus (SVB) for compact state exchange, Event-driven Dialogue Replanning (EDR) for online subtask reassignment, and Synchronous Parallel Execution (SPE) for shared-clock inference. Module 3: Two robots execute a relay task concurrently, where Robot 1 follows a start $\rightarrow$ pickup $\rightarrow$ handoff trajectory and Robot 2 follows a start $\rightarrow$ handoff $\rightarrow$ delivery trajectory, communicating via event-triggered dialogue.

Fig. 2: Overview of the TriGate target verification pipeline. Each candidate waypoint must pass three gates: GT semantic visibility, room consistency, and CLIP-based recognizability confirmation.

Fig. 3: Illustration of dynamic subtask reassignment in DeCoNav. Left: Initial task allocation, where Robot 1 navigates to the object and proceeds to the handoff point, and Robot 2 moves from the handoff point to the goal. Right: After event-triggered dialogue replanning, subtasks are reassigned based on updated semantic evidence, resulting in shorter overall paths for both robots.

Metric	CoNavBench [ 1 ]	DeCoNavBench
Tasks	992	1,213
HM3D scenes	128	176
Mean steps/robot	$61.2\pm 36.5$	$139.9\pm 62.3$
Mean path/robot (m)	$9.7\pm 6.7$	$20.6\pm 10.4$
Combined path (m)	$19.4\pm 11.0$	$41.2\pm 17.8$

TABLE I: CoNavBench vs. DeCoNavBench;

\pm

denotes standard deviation.

Metric	LHPR-VLN [ 13 ]	CoNavBench [ 1 ]	DeCoNavBench (ours)
Labelled rooms	1,803	1,582	2,469
Unlabelled rooms	666	168	0
Total rooms	2,469	1,750	2,469
Coverage	73%	90%	100%
Correctness	61%	68%	100%

TABLE II: Room-type label quality across three datasets (181 HM3D scenes, 2,469 rooms).

Setting	SR $\uparrow$	BSR $\uparrow$	ISR $\uparrow$	SPL $\uparrow$	NE $\downarrow$
CoNavBench baseline	0.28	0.13	0.35	0.18	5.19
DeCoNav (Ours)	0.39	0.22	0.47	0.32	4.75

TABLE III: Comparison between CoNavBench and DeCoNav on the same test split.

Setting	SR $\uparrow$	BSR $\uparrow$	ISR $\uparrow$	SPL $\uparrow$	NE $\downarrow$
w/o DeCoNav	0.32	0.15	0.41	0.24	4.93
w/ DeCoNav	0.39	0.22	0.47	0.32	4.75

TABLE IV: Ablation of DeCoNav on the same test split.

Fig. 4: Real-robot deployment on a collaborative object transport task. Robot 1 and Robot 2 operate as equal peers and execute decomposed subtasks concurrently under mapless exploration. When Robot 1 discovers a previously passable corridor is locked, the EDR module triggers online replanning. Through ROS2 communication and shared semantic memory, both robots negotiate a subtask swap that yields shorter overall paths, and complete the task.

Explore similar work

Mar 3, 2026

MA-CoNav: A Master-Slave Multi-Agent Framework with Hierarchical Collaboration and Dual-Level Reflection for Long-Horizon Embodied VLN

Vision-Language Navigation (VLN) aims to empower robots with the ability to perform long-horizon navigation in unfamiliar environments based on complex linguistic instructions. Its success critically hinges on establishing an efficient

language-understanding -- visual-perception -- embodied-execution'' closed loop. Existing methods often suffer from perceptual distortion and decision drift in complex, long-distance tasks due to the cognitive overload of a single agent. Inspired by distributed cognition theory, this paper proposes MA-CoNav, a Multi-Agent Collaborative Navigation framework. This framework adopts a

Master-Slave'' hierarchical agent collaboration architecture, decoupling and distributing the perception, planning, execution, and memory functions required for navigation tasks to specialized agents. Specifically, the Master Agent is responsible for global orchestration, while the Subordinate Agent group collaborates through a clear division of labor: an Observation Agent generates environment descriptions, a Planning Agent performs task decomposition and dynamic verification, an Execution Agent handles simultaneous mapping and action, and a Memory Agent manages structured experiences. Furthermore, the framework introduces a ``Local-Global'' dual-stage reflection mechanism to dynamically optimize the entire navigation pipeline. Empirical experiments were conducted using a real-world indoor dataset collected by a Limo Pro robot, with no scene-specific fine-tuning performed on the models throughout the process. The results demonstrate that MA-CoNav comprehensively outperforms existing mainstream VLN methods across multiple metrics.

Ling Luo, Qianqian Bai

Southwestern University of Finance and Economics, Chengdu, 611130, China

Vision-Language Navigation Multi-Agent Coordination Multi-Agent Planning

Ask PDF

Sep 28, 2026

Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Method

Vision-and-Language Navigation (VLN) has largely focused on a single agent following a single instruction, yet many real-world applications require teams of robots to tackle tasks beyond the capabilities of any individual agent. We present Systematic Multi-Agent Vision-and-Language Navigation, providing, to our knowledge, the first systematic formalization of multi-agent VLN as a constrained coordination problem: each mission consists of subtasks carrying dependency and resource constraints (presence locks and holding chains). A verified four-stage crafting pipeline instantiates the task as MAVLN, comprising 11,724 episodes across 145 scenes with teams of up to four agents under three instruction regimes, accompanied by tailored constraint-aware metrics. We further present TRISS, a coordination-ready navigation system coupling an LLM-based subtask scheduler, a shared topological memory that turns each agent's exploration into team knowledge, and a conflict-aware execution mechanism that realizes simultaneous intentions as collision-free routes. Extensive experiments establish TRISS as a comprehensive baseline and reveal substantial room for improvement across scheduling, planning, and execution, highlighting the challenges of coordinating under MAVLN task constraints. Project page: https://xyz9911.github.io/mavln.

Yunzhe Xu, Zhe Liu

School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai 200240, China

Multi-Agent Coordination Vision-Language Navigation

Ask PDF

Apr 23, 2026

DM $^3$ -Nav: Decentralized Multi-Agent Multimodal Multi-Object Semantic Navigation

We present DM

^3

-Nav, a fully decentralized multi-agent semantic navigation system supporting multimodal open-vocabulary goal specification and multi-object missions. In our setting, decentralization implies operation without a central coordinator, global map aggregation, or shared global state at runtime. Robots operate autonomously and coordinate through ad-hoc pairwise communication, exchanging local maps, goal status, and navigation intent without synchronization. An implicit task allocation mechanism combining intent broadcasting and distance-weighted frontier selection reduces redundant exploration while preserving decentralized operation. Evaluations on HM3DSem scenes using the HM3Dv0.2 and GOAT-Bench datasets demonstrate that DM

^3

-Nav matches or exceeds centralized and shared-map baselines while eliminating single points of failure inherent in centralized architectures. Finally, we validate our approach in a real-world office environment using two mobile robots, demonstrating successful deployment relying entirely on onboard sensing and computation. A video of our real-world experiments is available online: https://drive.google.com/file/d/1QiUSCn5rIvtuTUqtuXLPgmt6S8x9-MCZ/view?usp=drive_link

Amin Kashiri, Atharva Jamsandekar, Yasin Yazıcıoğlu

Northeastern University, Boston, USA · Department of Electrical and Computer Engineering at Northeastern University, Boston, MA

Vision-Language Navigation Multi-Robot Coordination Multi-Robot Task Allocation Multi-Agent Communication

Ask PDF

Abstract

Explore similar work

MA-CoNav: A Master-Slave Multi-Agent Framework with Hierarchical Collaboration and Dual-Level Reflection for Long-Horizon Embodied VLN

Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Method

DM3^33-Nav: Decentralized Multi-Agent Multimodal Multi-Object Semantic Navigation

DM $^3$ -Nav: Decentralized Multi-Agent Multimodal Multi-Object Semantic Navigation