Organizations: Zhejiang University · Shanghai AI Laboratory · University of Science and Technology of China · Tongji University · The University of Hong Kong
Emerging strong Vision-Language Models (VLMs) have advanced navigation foundation models. However, these models still have unsatisfactory performance on goal-oriented navigation settings. In this paper, we first conduct a motivating study revealing that the key issue lies in the insufficient utilization of streaming observations and imprecise target localization. To tackle these challenges, we propose GG-Nav, a simple yet effective navigation framework that learns a structured chain-of-thought (CoT) mechanism via supervised fine-tuning to explicitly localize goal-relevant instances, accumulate grounding evidence, and support more reliable decision-making. Specifically, the structured CoT is designed with two key stages: (i) Observation-to-Evidence, which reasons over all intermediate frames between adjacent decision steps to retain useful goal-related evidence; and (ii) Evidence-to-Decision, which integrates the current observation with accumulated evidence to ground the target and support the generation of navigation actions or clarification questions. Extensive experiments show that GG-Nav outperforms the state-of-the-art methods in terms of success rates on the HM3D ObjectNav, VL-LN, and CoIN benchmarks by 3.1%, 16.6%, and 5.3%, respectively. Ablation studies further show that accumulating evidence from intermediate observations reduces evidence-missing failures by 11.6%, while explicit grounding reduces stopping errors by 49.4%. We also deploy GG-Nav on a Unitree Go2 robot, demonstrating its zero-shot generalization capability in unseen real-world environments.
Figures & tables
Figure 1: Overview of GG-Nav. Observation-to-Evidence converts intermediate frames between adjacent decision steps into accumulated grounding evidence. Evidence-to-Decision integrates the current observation with this evidence to ground goal-relevant instances and generate navigation actions or clarification questions. GG-Nav supports both ObjectNav and IIGN and supports real-world robot deployment.
Target visibility
Error type
# Failures
Ratio
Not observed 119 (32%)
Exploration error
119
32%
Target appears in observations 254 (68%)
Evidence miss
153
41%
Reaching error
26
7%
Stopping error
75
20%
Total failures
373
100%
Table 1: Failure analysis of adapted InternVLA-N1 on VL-LN Bench.
Figure 2: Inference process of GG-Nav . Given the instruction, compact visual history, dialogue-associated evidence, and intermediate frames since the previous decision step, GG-Nav first converts the intermediate frames into accumulated grounding evidence through Observation-to-Evidence, and then uses Evidence-to-Decision to integrate this evidence with the current observation and generate either a navigation action or a clarification question.
Method
Interaction
HM3Dv2
VL-LN
CoIN (Seen)
CoIN (Seen Synonyms)
CoIN (Unseen)
SR ↑
SPL ↑
SR ↑
SPL ↑
NQ ↓
SR ↑
SPL ↑
NQ ↓
SR ↑
SPL ↑
NQ ↓
SR ↑
SPL ↑
NQ ↓
FBE
✗
31.5
13.69
9.2
5.09
-
4.69
2.75
-
2.35
1.55
-
3.92
2.05
-
VLFM Yokoyama et al. (2024)
✗
63.6
32.5
8.8
5.63
-
0.48
0.29
-
0.28
0.02
-
0.22
0.10
-
MTU3D Zhu et al. (2025)
✗
59.3
19.87
12.8
5.44
-
27.79
11.23
-
26.54
12.95
-
8.71
3.34
-
InternVLA-N1 Team (2025)
✓
69.2
27.61
25.4
16.18
1.98
17.57
9.51
2.79
20.94
11.13
2.54
17.21
8.20
2.61
AIUTA Taioli et al. (2025)
✓
-
-
4.0
2.16
5.42
7.42
2.92
1.67
14.38
7.99
1.36
6.67
2.30
1.13
Table 2: Baseline comparison on HM3D ObjectNav and the VL-LN and CoIN IIGN benchmarks.
Variant
Setting
HM3D
VL-LN
CoIN (Seen Synonyms)
Inter. Frames
O-to-E
E-to-D
SR ↑
SPL ↑
SR ↑
SPL ↑
NQ ↓
SR ↑
SPL ↑
NQ ↓
Action-only
✗
✗
✗
62.5
21.56
20.8
10.56
2.44
18.0
11.99
2.14
+ Intermediate frames
✓
✗
✗
66.7
25.78
36.8
20.39
2.79
28.3
18.13
3.82
+ Observation-to-Evidence
✓
✓
✗
68.9
26.94
36.6
20.08
2.80
27.7
14.68
3.66
+ Evidence-to-Decision
✓
✓
✓
72.3
31.12
42.0
23.22
3.01
31.9
16.62
4.92
Table 3: Progressive ablation study.
Benchmark
Variant
E
G
T
R
S
HM3D
Action-only
111
162
-
-
102
+ Intermediate frames
107
141
-
-
85
+ Observation-to-Evidence
93
154
6
20
38
+ Evidence-to-Decision
92
122
4
27
32
VL-LN
Action-only
163
164
-
54
15
+ Intermediate frames
123
143
-
23
27
Table 4: Stage-wise error analysis across benchmarks.
Strategy
Setting
SR ↑
SPL ↑
NQ ↓
Tokens ↓
Decision keyframe
50
43.4
24.36
3.32
3204
100
41.2
22.62
3.08
3270
200
42.0
23.22
3.01
3381
400
41.0
22.62
2.74
3443
Spatially local keyframe
3
41.6
24.02
3.13
3191
6
40.2
22.83
2.92
3324
Table 5: Ablation on dialog-frame retention strategies.
Figure 3: Case studies on using intermediate observations and disambiguating similar targets a in distractor-rich scene.
Figure 4: Real-world navigation visualization. The left side shows the IIGN (easy) scenario, where the agent is required to search for ‘chair_3’ among three distractors. The right side shows the IIGN (medium) scenario, where the robot needs to reach ‘plant_5’ among seven distractors. The robot-human dialogues are shown on the right, while selected observation frames and the final stopping frames are displayed next to the reconstructed scenes.