Does Peer Observation Help? Vision-Sharing Collaboration for Vision-Language Navigation
Authors: Qunchao Jin, Yiliao Song, Qi Wu
Organizations: Australian Institute for Machine Learning, Adelaide University · Responsible AI Research Centre, Australian Institute for Machine Learning
Vision-Language Navigation (VLN) systems are fundamentally constrained by partial observability, as an agent can only accumulate knowledge from locations it has personally visited. As multiple robots increasingly coexist in shared environments, a natural question arises: can agents navigating the same space benefit from each other's observations? In this work, we introduce Co-VLN, a minimalist, model-agnostic framework for systematically investigating whether and how peer observations from concurrently navigating agents can benefit VLN. When independently navigating agents identify common traversed locations, they exchange structured perceptual memory, effectively expanding each agent's receptive field at no additional exploration cost. We validate our framework on the R2R benchmark under two representative paradigms (the learning-based DUET and the zero-shot MapGPT), and conduct extensive analytical experiments to systematically reveal the underlying dynamics of peer observation sharing in VLN. Results demonstrate that vision-sharing enabled model yields substantial performance improvements across both paradigms, establishing a strong foundation for future research in collaborative embodied navigation.
Figures & tables
Figure 1: Motivation of peer observation sharing in VLN. (a) A single agent navigates independently, with its perception limited to personally visited regions while large portions of the environment remain unobserved, resulting in only 35% spatial coverage. (b) When a peer agent independently navigates the same environment, their trajectories may overlap at a spatial section (yellow star), enabling knowledge exchange that expands perceptual coverage from 35% to 65% without additional exploration.
Figure 2: Overview of the Co-VLN framework. Given multiple agents navigating in a shared environment, Co-VLN operates in three stages: (1) each agent independently explores and builds its own topological graph; (2) spatial overlap between the two graphs is detected via embedding-based matching or viewpoint ID matching; (3) the graphs are merged through anchor nodes identified in Stage 2, producing an enriched navigation graph that expands each agent’s knowledge beyond its own exploration.
Method Paradigm
Methods
TL
NE ↓
OSR ↑
SR ↑
SPL ↑
PREVALENT [ 18 ]
10.19
4.71
–
58
53
VLN ↻ BERT [ 20 ]
12.01
3.93
69
63
57
HAMT [ 10 ]
11.46
3.62
73
66
61
Supervised
HOP+ [ 36 ]
11.76
3.49
–
67
61
Learning
TD-STP [ 59 ]
–
3.22
76
70
63
BEVBert [ 1 ]
14.55
2.81
84
75
64
Table 1: Comparison with state-of-the-art methods on the R2R val unseen. ↑ : higher is better, ↓ : lower is better. MapGPT ‡ uses Qwen3-VL-32B as the MLLM. † denotes the use of oracle viewpoint IDs for spatial overlap detection. ∗ denotes applying CLIP ViT-B16 as image features. Best results are bold .
Methods
TL
NE ↓
OSR ↑
SR ↑
SPL ↑
Seq2Seq [ 25 ]
8.64
7.37
40
32
30
HPN [ 23 ]
7.62
6.31
40
36
34
CM2 [ 16 ]
11.54
7.02
42
34
28
WS-MGMAP [ 9 ]
10.00
6.28
48
39
34
CWP-CMA [ 19 ]
10.90
6.20
52
41
36
CWP-RecBERT [ 19 ]
12.23
5.74
53
44
39
Table 2: Comparison with state-of-the-art methods on the R2R-CE val unseen. ↑ : higher is better, ↓ : lower is better. Best results are bold .
Figure 5
MLLM
Vision- Sharing
NE ↓
OSR ↑
SR ↑
SPL ↑
InternVL3.5-38B
✗
6.60
50.5
37.0
31.1
✓
6.40 (-0.20)
51.5 (+1.0)
38.0 (+1.0)
31.6 (+0.5)
Gemini-3-Flash †
✗
6.09
40.5
38.0
35.2
✓
5.10 (-0.99)
59.0 (+18.5)
46.0 (+8.0)
40.5 (+5.3)
Qwen3-VL-32B
✗
5.51
59.5
46.5
38.0
✓
4.84 (-0.67)
64.0 (+4.5)
49.5 (+3.0)
41.9 (+3.9)
Table 4: Effect of different MLLMs on Vision-Sharing setting. Best results are bold , second best are underlined . † indicates closed-source models.
Area Group
Vision- Sharing
SR ↑
SPL ↑
OSR ↑
NE ↓
Small ( < 250 m 2 )
✗
56.75
49.95
68.65
3.77
✓
58.73
50.71
72.62
3.62
+ 1.98 ( + 3.49%)
+ 0.76 ( + 1.52%)
+ 3.97 ( + 5.78%)
− 0.15 ( − 3.98%)
Medium (250–450 m 2 )
✗
52.40
43.62
61.74
4.26
✓
55.81
46.75
67.30
3.89
+ 3.41 ( + 6.51%)
+ 3.13 ( + 7.18%)
+ 5.56 ( + 9.01%)
− 0.37 ( − 8.69%)
Table 5: Effect of scene area on Vision-Sharing setting. Shaded rows show absolute improvement with relative improvement (%). Bold indicates the best improvement.
Baseline
Vision- Sharing
Pairing Strategy
NE ↓
OSR ↑
SR ↑
SPL ↑
MapGPT
✗
–
4.80
64.0
52.2
44.7
✓
random pairing
4.66
65.3
53.4
45.7
✓
prior-based pairing
4.36
68.6
55.8
47.3
DUET
✗
–
3.31
80.5
71.5
60.4
✓
random pairing
3.24
81.2
72.6
61.2
✓
prior-based pairing
2.99
82.7
74.5
62.6
Table 6: Effect of trajectory pairing strategies on Vision-Sharing setting. Best results are bold , second best are underlined .
Detection Trigger
Bi- direction
Persistent
SR ↑
SPL ↑
DUET (w/o Vision-Sharing)
71.5
60.4
✓
✓
×
72.4
60.0
✓
×
✓
73.7
61.3
×
✓
✓
74.3
61.5
✓
✓
✓
74.5
62.6
Table 7: Effect of graph fusion strategy. Best results are bold , second best are underlined .
Figure 4: Qualitative comparison . We visualize DUET with and without vision-sharing.
Vision-Language Navigation (VLN) aims to empower robots with the ability to perform long-horizon navigation in unfamiliar environments based on complex linguistic instructions. Its success critically hinges on establishing an efficient language-understanding -- visual-perception -- embodied-execution'' closed loop. Existing methods often suffer from perceptual distortion and decision drift in complex, long-distance tasks due to the cognitive overload of a single agent. Inspired by distributed cognition theory, this paper proposes MA-CoNav, a Multi-Agent Collaborative Navigation framework. This framework adopts a Master-Slave'' hierarchical agent collaboration architecture, decoupling and distributing the perception, planning, execution, and memory functions required for navigation tasks to specialized agents. Specifically, the Master Agent is responsible for global orchestration, while the Subordinate Agent group collaborates through a clear division of labor: an Observation Agent generates environment descriptions, a Planning Agent performs task decomposition and dynamic verification, an Execution Agent handles simultaneous mapping and action, and a Memory Agent manages structured experiences. Furthermore, the framework introduces a ``Local-Global'' dual-stage reflection mechanism to dynamically optimize the entire navigation pipeline. Empirical experiments were conducted using a real-world indoor dataset collected by a Limo Pro robot, with no scene-specific fine-tuning performed on the models throughout the process. The results demonstrate that MA-CoNav comprehensively outperforms existing mainstream VLN methods across multiple metrics.
Ling Luo, Qianqian Bai
Southwestern University of Finance and Economics, Chengdu, 611130, China
Long-horizon collaborative vision-language navigation (VLN) is critical for multi-robot systems to accomplish complex tasks beyond the capability of a single agent. CoNavBench takes a first step by introducing the first collaborative long-horizon VLN benchmark with relay-style multi-robot tasks, a collaboration taxonomy, along with graph-grounded generation and evaluation to model handoffs and rendezvous in shared environments. However, existing benchmarks and evaluations often do not enforce strictly synchronized dual-robot rollout on a shared world timeline, and they typically rely on static coordination policies that cannot adapt when new cross-agent evidence emerges. We present Dialog enhanced Long-Horizon Collaborative Vision-Language Navigation (DeCoNav), a decentralized framework that couples event-triggered dialogue with dynamic task allocation and replanning for real-time, adaptive coordination. In DeCoNav, robots exchange compact semantic states via dialogue without a central controller. When informative events such as new evidence, uncertainty, or conflicts arise, dialogue is triggered to dynamically reassign subgoals and replan under synchronized execution. Implemented in DeCoNavBench with 1,213 tasks across 176 HM3D scenes, DeCoNav improves the both-success rate (BSR) by 69.2%, demonstrating the effectiveness of dialogue-driven, dynamically reallocated planning for multi-robot collaboration.
Sunyao Zhou, Yunzi Wu, Tianhang Wang +5
Institute of Artificial Intelligence (TeleAI), China Telecom · Fudan University · Tongji University
Navigation is a fundamental capability of autonomous systems, yet most existing approaches rely on highly structured models and strong prior assumptions, limiting their robustness in open and uncertain real-world environments. Vision-and-Language Navigation (VLN) offers a promising direction by enabling robots to integrate natural language understanding with visual perception in a data-driven manner. Although VLN has attracted increasing research attention, systematic methodological taxonomy and real-world validation remain limited. This survey presents a comprehensive review of VLN research. Specifically, state-of-the-art methods are organized along two orthogonal dimensions: action paradigms, including hierarchical and monolithic frameworks, and model paradigms, including discriminative and generative approaches. A critical analysis of their respective strengths and limitations is provided. Additionally, we conduct a systematic real-world evaluation of representative VLN system configurations on a physical robotic platform. Experiments across ten diverse real-world scenes show a substantial performance gap between simulation and real-world deployment under the tested configurations: a representative monolithic RGB-only method achieves 61% success in simulation but drops to 22% in real-world deployment, while a hierarchical framework achieves a higher real-world success rate of 51%, suggesting stronger robustness in our evaluation setting. Finally, we highlight key challenges in perception, decision-making, and control that must be addressed in future research.
Liuyi Wang, Kai Sheng, Zongtao He +8
College of Electronic and Information Engineering, Tongji University, Shanghai, China