Pretrained VLMs bring rich world knowledge and semantic priors to object-goal navigation, and fine-tuning a VLM into a navigation policy has become a common recipe. Despite this progress, success rates remain low, and it is unclear which capability these fine-tuned policies lack. Our diagnosis identifies goal grounding as the dominant failure. These policies explore well, yet often fail to notice that the goal is already in view or to estimate how far away it is. To address this failure, we ground the goal explicitly. Explicit grounding substantially improves success and reaches state-of-the-art results on six benchmarks across three scene collections. We also correct annotation defects in these benchmarks, so that future methods are evaluated more faithfully. Our analysis on the clean benchmarks shows that, with ground-truth grounding, our policy nearly matches the expert that generated its demonstrations, leaving the headroom in perception rather than policy learning. Our work points to goal grounding as a missing piece of object-goal navigation and offers a simple and effective solution to the task.
Figures & tables
Figure 1: Goal grounding is key to object-goal navigation. (a, b) Baselines explore as well as ours yet walk past the goal; ours stops within 0.25 m. (c) Our system has explicit goal grounding, which enables the agent to stop at the right place. (d) Ours has the highest success rate of all compared systems on six cleaned benchmarks. (e) Success rate against the rate of reaching the goal vicinity. All baselines reach the vicinity far more often than they succeed.
Figure 2: Where episodes end. We trace novel-goal episodes of an end-to-end RGB policy (top) and the same policy with ground-truth grounding (bottom).
Figure 3: Design choices. Three decisions turn an end-to-end policy into our system. Each column contrasts a design used in prior work (top) with our choice (bottom).
Figure 4: System overview. The perception model writes grounding text for each frame. The policy reads the frames interleaved with this text and outputs the next action. The two models are trained separately, both on grounding text from the scene annotations.
Figure 5: Two defects of the evaluation. (a) The same object type is annotated under many names, and only the exact annotated name earns success credit. (b) The annotations contain errors of two kinds, and a manual audit filters out 23% of the val_unseen episodes.
HM3D
MP3D
HSSD
OVON val_seen
OVON val_unseen
6 cat.
OV
21 cat.
OV
6 cat.
official
clean
official
clean
official
clean
ours
official
clean
ours
official
clean
VLFM ( Yokoyama et al., 2024a )
40.0
53.6
39.0
52.8
64.7
66.5
30.8
33.6
37.8
36.3
59.0
65.1
Nav-R 2 ( Xiang et al., 2025 )
39.6
51.6
27.4
34.0
58.1
56.7
28.3
36.1
40.4
13.8
35.7
34.9
DAgRL ( Yokoyama et al., 2024b )
44.8
60.7
21.0
26.0
54.0
53.3
23.0
23.2
22.3
5.8
23.6
24.9
Uni-NaVid ( Zhang et al., 2025 )
35.4
46.4
32.1
41.0
62.5
49.0
30.3
32.2
34.5
22.0
51.4
52.5
Table 1: Comparison with baselines. Official columns use each benchmark’s own success criterion; clean columns use our criterion on the audited clean subsets ( Sec. 4.1 ). GT grounding is the same policy reading GT grounding text, shown for reference. All numbers are success rates in percent (%).
Figure 6: Same episode, different decisions. (a) Paths of both systems through the scene. (b) Views at each system’s closest approach to the goal and at its STOP , labeled with the step and the action taken. Our frames also show the predicted grounding. (c) Goal distance of both systems, with our predicted goal visibility, distance bin, and stop probability. Steps 30 to 64 are enlarged.
Disjoint
Semantic grounding
Distance grounding
Seen
Unseen
72.7
63.6
✓
✓
64.4
50.7
✓
✓
76.5
68.7
✓
✓
73.4
60.1
✓
✓
✓
77.6
69.6
Table 2: Ablation study. Success of the design choices on the clean OVON splits. The last row is our complete design and has the best performance.
Figure 7: From our system to the expert. (a) Success on HM3D-OVON val_unseen as predicted grounding fields are replaced with ground truth. Gray marks episodes where the policy stops before ever coming within 0.25 m of the goal. (b) The policy with ground-truth grounding against the expert. Bar height is reach within 0.25 m, and the solid part is success. Clean subsets throughout.
Figure 8: Perception errors of our system. Top, examples of each error type. (a) Final-system failures on OVON val_unseen by cause. (b) Final-system success per goal category against how often the perception model reports the goal in view.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Failures and goal size with VLM grounding. (a) Failed episodes by perception error, as in Fig. 8 a. (b) Success per goal category against its physical size, the median area of its training instances, with an episode-weighted linear fit for each source.
Perception model
Goal recall
Object F1
Success
Step time (s)
Qwen3-VL-4B
83.2
48.6
66.0
1.0
Joint model
70.0
66.4
–
–
GPT-6 Luna
89.7
57.8
72.9
6.4 ∗
GPT-6 Astra
95.9
58.6
81.4
16.6 / 9.3 ∗
Ours
77.9
71.3
78.6
1.0
Ground-truth grounding
–
–
91.1
–
Appendix
Table 3: Comparison on perception models. Goal recall and object F1 on OVON val_unseen frames, and success of the same policy on clean val_unseen with GT goal distance as in the first step of Fig. 7 a. The VLMs are not trained on our task. Among them, success rises with goal recall. ∗ Amazon Bedrock API, with the second GPT-6 Astra value on its ultrafast API.
OVON val_unseen
MP3D-OV
HSSD-OV
SR
SPL
SPL FE
SR
SPL
SPL FE
SR
SPL
SPL FE
VLFM ( Yokoyama et al., 2024a )
52.8
30.2
39.8
30.8
16.7
27.1
36.3
18.5
29.0
Nav-R 2 ( Xiang et al., 2025 )
34.0
14.4
19.9
28.3
11.0
19.6
13.8
6.2
9.8
DAgRL ( Yokoyama et al., 2024b )
26.0
10.7
15.7
23.0
9.6
17.8
5.8
1.7
3.5
Uni-NaVid ( Zhang et al., 2025 )
41.0
16.7
23.0
30.3
12.7
22.2
22.0
10.1
16.9
LightNav-0 ( Wang et al., 2026 )
60.8
28.0
38.0
44.3
21.4
34.9
32.8
16.9
25.7
Appendix
Table 4: Path efficiency on the clean open-vocabulary benchmarks.
Evaluation set
Episodes
Scenes
Goal names
Goal groups
Clean
HM3D-OVON val_seen
3,000
36
79
66
1,937
HM3D-OVON val_unseen
3,000
36
49
40
2,309
HM3D ObjectNav six-class
1,000
36
6
n/a
905
MP3D-OV
3,000
11
62
59
1,193
MP3D ObjectNav 21-class
2,195
11
21
n/a
1,078
HSSD-OV
3,000
43
110
104
1,751
Appendix
Table 5: Evaluation sets. Episodes, scenes, goal names, and goal groups describe the raw sets before exclusions. Goal groups count equivalence groups under our vocabulary. Clean counts give the adopted evaluation subsets.
Parameter
Value
Body height / radius
1.41 m / 0.17 m
Camera height
1.31 m
Rendered RGB / horizontal field of view
640×360 / 120∘
Forward step / turn angle
0.25 m / 30∘
Action budget / sliding
500 / enabled
Appendix
Table 6: Simulation settings for our system. Image dimensions are width by height.
Field
Format
Objects:
Category names separated by |
Goal visible:
true or false
Distance:
<0.5m , 0.5-1m , 1-2m , 2-4m , >4m , or not seen
Appendix
Table 7: Grounding text read by the policy. The action prompt follows these fields.
Parameter
Perception
Policy
Initialization
Qwen3-VL-4B-Instruct
Qwen3-VL-2B-Instruct
Model RGB input
640×352
320×160
Maximum history frames
128
500
Updates
20,000
40,000
Frames per update, cap
8,192
8,192
Peak learning rate
10−5
10−5
Appendix
Table 8: Training hyperparameters. Frame budgets are global caps per update. FSDP is fully sharded data parallelism. Training time sums the running time of all job attempts, including setup and checkpointing. The policy run resumed twice after preemption.
Branch or outcome
RGB-only
Ground-truth grounding
All episodes
3,000
3,000
Reached vicinity
2,288
2,818
Success
1,569
2,679
Wrong stop
183
0
Timeout
536
139
Never reached vicinity
712
182
Appendix
Table 9: Counts behind Figure 2 . The ground-truth-grounding policy is the same single-source policy described above.
Figure 10: Same-episode comparisons of our system and LightNav-0. Each row follows the layout of Fig. 6 . The last row is a failure of ours and adds the ground-truth goal visibility.
Figure 11: Failure flows on clean HM3D-OVON val_unseen, as in Fig. 2 . Percentages use the 2,309 clean episodes.
HM3D-OVON val_unseen 2,309
MP3D-OV 1,193
HSSD-OV 1,751
Outcome
Pred.
GT
Exp.
Pred.
GT
Exp.
Pred.
GT
Exp.
Success
1,606
2,115
2,218
595
883
904
988
1,350
1,426
Reached, wrong stop
82
18
0
46
5
0
31
7
1
Reached, timeout
100
34
1
45
16
0
41
23
6
Never reached, stop
427
51
63
284
9
98
408
25
104
Never reached, timeout
94
91
27
223
280
191
283
346
214
Appendix
Table 10: Clean-subset counts for Fig. 7 . Pred. uses predicted grounding, GT uses ground-truth grounding, and Exp. denotes the frontier-exploration expert.
Vision-and-Language Navigation (VLN) agents may satisfy conventional success criteria while still failing to establish reliable object-level grounding, because current evaluation protocols mainly reward stopping within a 3-meter radius and largely ignore the agent's final orientation and target visibility. We formalize this limitation as the Last-3-Meter Grounding Gap and introduce three instance-centric metrics to quantify proximity precision, target visibility, and final-view grounding. To mitigate this gap, we propose REALM (Region-to-Entity Alignment for Last-3-Meter Navigation), a plug-and-play, architecture-agnostic refinement module that decouples fine-grained target approaching from long-horizon navigation. REALM uses a visibility-aware stopping strategy to reduce premature termination and improve final viewpoint alignment. We further construct REVERIE-AIM, which provides object-instance-level goals and 180K short-horizon training samples for final-stage target approaching. Extensive evaluations across four diverse VLN backbones show that REALM consistently improves proximity precision and visual grounding success, demonstrating its broad applicability.
Xiangyu Shi, Ruoxi Yang, Wei Tao +3
Australian Institute for Machine Learning, Adelaide University · Ruyi Dynamics · Fudan University +1
Vision-language models (VLMs) have become a common foundation for vision-and-language navigation in continuous environments (VLN-CE). Yet most VLM-based methods cast navigation as low-level action prediction, an interface that is ambiguous, tied to short-horizon motion primitives, and inefficient due to repeated VLM querying. We propose Goal2Pixel, a pure pixel-based paradigm that reformulates VLN-CE as navigable pixel grounding. Rather than predicting actions, Goal2Pixel uses the image plane as a unified spatial interface between VLM reasoning and robot motion: the model predicts a visible navigable pixel to the agent, which is back-projected into a 3D waypoint for forward navigation. For non-forward actions, we append auxiliary directive regions to the image plane, where the left/right/bottom regions are interpreted as turning left, turning right, and stopping, respectively. To enable long-horizon navigation, we propose a visibility-aware keyframe memory for compact and informative history representation. To adapt pretrained VLMs to navigable pixel grounding, we introduce semantic embeddings and coordinate-aware auxiliary losses. Goal2Pixel achieves competitive state-of-the-art performance while requiring fewer VLM inference calls than prior methods. On R2R-CE Val-Unseen it achieves 54.1% SR and 52.5% SPL with just 7.75 VLM calls per episode, 6x fewer than the 46.62 required by direct action prediction at 32.9% SR. The same trend holds on RxR-CE.Project Page: https://baobao0926.github.io/Goal2Pixel/.
Muyi Bao, Yuxin Cai, Hang Xu +7
Carnegie Mellon University · Nanyang Technological University
Emerging strong Vision-Language Models (VLMs) have advanced navigation foundation models. However, these models still have unsatisfactory performance on goal-oriented navigation settings. In this paper, we first conduct a motivating study revealing that the key issue lies in the insufficient utilization of streaming observations and imprecise target localization. To tackle these challenges, we propose GG-Nav, a simple yet effective navigation framework that learns a structured chain-of-thought (CoT) mechanism via supervised fine-tuning to explicitly localize goal-relevant instances, accumulate grounding evidence, and support more reliable decision-making. Specifically, the structured CoT is designed with two key stages: (i) Observation-to-Evidence, which reasons over all intermediate frames between adjacent decision steps to retain useful goal-related evidence; and (ii) Evidence-to-Decision, which integrates the current observation with accumulated evidence to ground the target and support the generation of navigation actions or clarification questions. Extensive experiments show that GG-Nav outperforms the state-of-the-art methods in terms of success rates on the HM3D ObjectNav, VL-LN, and CoIN benchmarks by 3.1%, 16.6%, and 5.3%, respectively. Ablation studies further show that accumulating evidence from intermediate observations reduces evidence-missing failures by 11.6%, while explicit grounding reduces stopping errors by 49.4%. We also deploy GG-Nav on a Unitree Go2 robot, demonstrating its zero-shot generalization capability in unseen real-world environments.
Shaohao Zhu, Wensi Huang, Siqi Zhang +6
Zhejiang University · Shanghai AI Laboratory · University of Science and Technology of China +2