Long-horizon collaborative vision-language navigation (VLN) is critical for multi-robot systems to accomplish complex tasks beyond the capability of a single agent. CoNavBench takes a first step by introducing the first collaborative long-horizon VLN benchmark with relay-style multi-robot tasks, a collaboration taxonomy, along with graph-grounded generation and evaluation to model handoffs and rendezvous in shared environments. However, existing benchmarks and evaluations often do not enforce strictly synchronized dual-robot rollout on a shared world timeline, and they typically rely on static coordination policies that cannot adapt when new cross-agent evidence emerges. We present Dialog enhanced Long-Horizon Collaborative Vision-Language Navigation (DeCoNav), a decentralized framework that couples event-triggered dialogue with dynamic task allocation and replanning for real-time, adaptive coordination. In DeCoNav, robots exchange compact semantic states via dialogue without a central controller. When informative events such as new evidence, uncertainty, or conflicts arise, dialogue is triggered to dynamically reassign subgoals and replan under synchronized execution. Implemented in DeCoNavBench with 1,213 tasks across 176 HM3D scenes, DeCoNav improves the both-success rate (BSR) by 69.2%, demonstrating the effectiveness of dialogue-driven, dynamically reallocated planning for multi-robot collaboration.
Figures & tables
Fig. 1: Overview of DeCoNav and DeCoNavBench. Module 1: The ROVE pipeline constructs verified episodes through rule-based inference, VLM classification, human adjudication (RTSA), and triple-gate target verification (TriGate). Module 2: DeCoNav coordinates dual-robot execution via three coupled components: Semantic Visual Bus (SVB) for compact state exchange, Event-driven Dialogue Replanning (EDR) for online subtask reassignment, and Synchronous Parallel Execution (SPE) for shared-clock inference. Module 3: Two robots execute a relay task concurrently, where Robot 1 follows a start → pickup → handoff trajectory and Robot 2 follows a start → handoff → delivery trajectory, communicating via event-triggered dialogue.
Fig. 2: Overview of the TriGate target verification pipeline. Each candidate waypoint must pass three gates: GT semantic visibility, room consistency, and CLIP-based recognizability confirmation.
Fig. 3: Illustration of dynamic subtask reassignment in DeCoNav. Left: Initial task allocation, where Robot 1 navigates to the object and proceeds to the handoff point, and Robot 2 moves from the handoff point to the goal. Right: After event-triggered dialogue replanning, subtasks are reassigned based on updated semantic evidence, resulting in shorter overall paths for both robots.
Metric
CoNavBench [ 1 ]
DeCoNavBench
Tasks
992
1,213
HM3D scenes
128
176
Mean steps/robot
61.2±36.5
139.9±62.3
Mean path/robot (m)
9.7±6.7
20.6±10.4
Combined path (m)
19.4±11.0
41.2±17.8
TABLE I: CoNavBench vs. DeCoNavBench; ± denotes standard deviation.
Metric
LHPR-VLN [ 13 ]
CoNavBench [ 1 ]
DeCoNavBench (ours)
Labelled rooms
1,803
1,582
2,469
Unlabelled rooms
666
168
0
Total rooms
2,469
1,750
2,469
Coverage
73%
90%
100%
Correctness
61%
68%
100%
TABLE II: Room-type label quality across three datasets (181 HM3D scenes, 2,469 rooms).
Setting
SR ↑
BSR ↑
ISR ↑
SPL ↑
NE ↓
CoNavBench baseline
0.28
0.13
0.35
0.18
5.19
DeCoNav (Ours)
0.39
0.22
0.47
0.32
4.75
TABLE III: Comparison between CoNavBench and DeCoNav on the same test split.
Setting
SR ↑
BSR ↑
ISR ↑
SPL ↑
NE ↓
w/o DeCoNav
0.32
0.15
0.41
0.24
4.93
w/ DeCoNav
0.39
0.22
0.47
0.32
4.75
TABLE IV: Ablation of DeCoNav on the same test split.
Fig. 4: Real-robot deployment on a collaborative object transport task. Robot 1 and Robot 2 operate as equal peers and execute decomposed subtasks concurrently under mapless exploration. When Robot 1 discovers a previously passable corridor is locked, the EDR module triggers online replanning. Through ROS2 communication and shared semantic memory, both robots negotiate a subtask swap that yields shorter overall paths, and complete the task.