StopWatch: Goal Grounding for Object-goal Navigation

Xinhao Liu; Yunsheng Tian; Yuheng Qiu; Pieter Abbeel; Xi Chen; Guanya Shi; Chen Feng

cs.CVOct 16, 2026

StopWatch: Goal Grounding for Object-goal Navigation

Authors: Xinhao Liu, Yunsheng Tian, Yuheng Qiu, Pieter Abbeel, Xi Chen, Guanya Shi, Chen Feng

Organizations: Amazon FAR · New York University · UC Berkeley · Carnegie Mellon University

Abstract

Pretrained VLMs bring rich world knowledge and semantic priors to object-goal navigation, and fine-tuning a VLM into a navigation policy has become a common recipe. Despite this progress, success rates remain low, and it is unclear which capability these fine-tuned policies lack. Our diagnosis identifies goal grounding as the dominant failure. These policies explore well, yet often fail to notice that the goal is already in view or to estimate how far away it is. To address this failure, we ground the goal explicitly. Explicit grounding substantially improves success and reaches state-of-the-art results on six benchmarks across three scene collections. We also correct annotation defects in these benchmarks, so that future methods are evaluated more faithfully. Our analysis on the clean benchmarks shows that, with ground-truth grounding, our policy nearly matches the expert that generated its demonstrations, leaving the headroom in perception rather than policy learning. Our work points to goal grounding as a missing piece of object-goal navigation and offers a simple and effective solution to the task.

Figures & tables

Figure 1: Goal grounding is key to object-goal navigation. (a, b) Baselines explore as well as ours yet walk past the goal; ours stops within 0.25 m. (c) Our system has explicit goal grounding, which enables the agent to stop at the right place. (d) Ours has the highest success rate of all compared systems on six cleaned benchmarks. (e) Success rate against the rate of reaching the goal vicinity. All baselines reach the vicinity far more often than they succeed.

Figure 2: Where episodes end. We trace novel-goal episodes of an end-to-end RGB policy (top) and the same policy with ground-truth grounding (bottom).

Figure 3: Design choices. Three decisions turn an end-to-end policy into our system. Each column contrasts a design used in prior work (top) with our choice (bottom).

Figure 4: System overview. The perception model writes grounding text for each frame. The policy reads the frames interleaved with this text and outputs the next action. The two models are trained separately, both on grounding text from the scene annotations.

Figure 5: Two defects of the evaluation. (a) The same object type is annotated under many names, and only the exact annotated name earns success credit. (b) The annotations contain errors of two kinds, and a manual audit filters out 23% of the val_unseen episodes.

	HM3D						MP3D			HSSD
	OVON val_seen		OVON val_unseen		6 cat.		OV	21 cat.		OV	6 cat.
	official	clean	official	clean	official	clean	ours	official	clean	ours	official	clean
VLFM ( Yokoyama et al., 2024a )	40.0	53.6	39.0	52.8	64.7	66.5	30.8	33.6	37.8	36.3	59.0	65.1
Nav-R 2 ( Xiang et al., 2025 )	39.6	51.6	27.4	34.0	58.1	56.7	28.3	36.1	40.4	13.8	35.7	34.9
DAgRL ( Yokoyama et al., 2024b )	44.8	60.7	21.0	26.0	54.0	53.3	23.0	23.2	22.3	5.8	23.6	24.9
Uni-NaVid ( Zhang et al., 2025 )	35.4	46.4	32.1	41.0	62.5	49.0	30.3	32.2	34.5	22.0	51.4	52.5

Table 1: Comparison with baselines. Official columns use each benchmark’s own success criterion; clean columns use our criterion on the audited clean subsets ( Sec. 4.1 ). GT grounding is the same policy reading GT grounding text, shown for reference. All numbers are success rates in percent (%).

Figure 6: Same episode, different decisions. (a) Paths of both systems through the scene. (b) Views at each system’s closest approach to the goal and at its STOP , labeled with the step and the action taken. Our frames also show the predicted grounding. (c) Goal distance of both systems, with our predicted goal visibility, distance bin, and stop probability. Steps 30 to 64 are enlarged.

Disjoint	Semantic grounding	Distance grounding	Seen	Unseen
			72.7	63.6
	$\checkmark$	$\checkmark$	64.4	50.7
$\checkmark$	$\checkmark$		76.5	68.7
$\checkmark$		$\checkmark$	73.4	60.1
$\checkmark$	$\checkmark$	$\checkmark$	77.6	69.6

Table 2: Ablation study. Success of the design choices on the clean OVON splits. The last row is our complete design and has the best performance.

Figure 7: From our system to the expert. (a) Success on HM3D-OVON val_unseen as predicted grounding fields are replaced with ground truth. Gray marks episodes where the policy stops before ever coming within 0.25 m of the goal. (b) The policy with ground-truth grounding against the expert. Bar height is reach within 0.25 m, and the solid part is success. Clean subsets throughout.

Figure 8: Perception errors of our system. Top, examples of each error type. (a) Final-system failures on OVON val_unseen by cause. (b) Final-system success per goal category against how often the perception model reports the goal in view.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Failures and goal size with VLM grounding. (a) Failed episodes by perception error, as in Fig. 8 a. (b) Success per goal category against its physical size, the median area of its training instances, with an episode-weighted linear fit for each source.

Perception model	Goal recall	Object F1	Success	Step time (s)
Qwen3-VL-4B	83.2	48.6	66.0	1.0
Joint model	70.0	66.4	–	–
GPT-6 Luna	89.7	57.8	72.9	6.4 ∗
GPT-6 Astra	95.9	58.6	81.4	16.6 / 9.3 ∗
Ours	77.9	71.3	78.6	1.0
Ground-truth grounding	–	–	91.1	–

Appendix

Table 3: Comparison on perception models. Goal recall and object F1 on OVON val_unseen frames, and success of the same policy on clean val_unseen with GT goal distance as in the first step of Fig. 7 a. The VLMs are not trained on our task. Among them, success rises with goal recall. ∗ Amazon Bedrock API, with the second GPT-6 Astra value on its ultrafast API.

	OVON val_unseen			MP3D-OV			HSSD-OV
	SR	SPL	SPL FE	SR	SPL	SPL FE	SR	SPL	SPL FE
VLFM ( Yokoyama et al., 2024a )	52.8	30.2	39.8	30.8	16.7	27.1	36.3	18.5	29.0
Nav-R 2 ( Xiang et al., 2025 )	34.0	14.4	19.9	28.3	11.0	19.6	13.8	6.2	9.8
DAgRL ( Yokoyama et al., 2024b )	26.0	10.7	15.7	23.0	9.6	17.8	5.8	1.7	3.5
Uni-NaVid ( Zhang et al., 2025 )	41.0	16.7	23.0	30.3	12.7	22.2	22.0	10.1	16.9
LightNav-0 ( Wang et al., 2026 )	60.8	28.0	38.0	44.3	21.4	34.9	32.8	16.9	25.7

Appendix

Table 4: Path efficiency on the clean open-vocabulary benchmarks.

Evaluation set	Episodes	Scenes	Goal names	Goal groups	Clean
HM3D-OVON val_seen	3,000	36	79	66	1,937
HM3D-OVON val_unseen	3,000	36	49	40	2,309
HM3D ObjectNav six-class	1,000	36	6	n/a	905
MP3D-OV	3,000	11	62	59	1,193
MP3D ObjectNav 21-class	2,195	11	21	n/a	1,078
HSSD-OV	3,000	43	110	104	1,751

Appendix

Table 5: Evaluation sets. Episodes, scenes, goal names, and goal groups describe the raw sets before exclusions. Goal groups count equivalence groups under our vocabulary. Clean counts give the adopted evaluation subsets.

Parameter	Value
Body height / radius	1.41 m / 0.17 m
Camera height	1.31 m
Rendered RGB / horizontal field of view	$640\times 360$ / $120^{\circ}$
Forward step / turn angle	0.25 m / $30^{\circ}$
Action budget / sliding	500 / enabled

Appendix

Table 6: Simulation settings for our system. Image dimensions are width by height.

Field	Format
Objects:	Category names separated by \|
Goal visible:	true or false
Distance:	<0.5m , 0.5-1m , 1-2m , 2-4m , >4m , or not seen

Appendix

Table 7: Grounding text read by the policy. The action prompt follows these fields.

Parameter	Perception	Policy
Initialization	Qwen3-VL-4B-Instruct	Qwen3-VL-2B-Instruct
Model RGB input	$640\times 352$	$320\times 160$
Maximum history frames	128	500
Updates	20,000	40,000
Frames per update, cap	8,192	8,192
Peak learning rate	$10^{-5}$	$10^{-5}$

Appendix

Table 8: Training hyperparameters. Frame budgets are global caps per update. FSDP is fully sharded data parallelism. Training time sums the running time of all job attempts, including setup and checkpointing. The policy run resumed twice after preemption.

Branch or outcome	RGB-only	Ground-truth grounding
All episodes	3,000	3,000
Reached vicinity	2,288	2,818
Success	1,569	2,679
Wrong stop	183	0
Timeout	536	139
Never reached vicinity	712	182

Appendix

Table 9: Counts behind Figure 2 . The ground-truth-grounding policy is the same single-source policy described above.

Figure 10: Same-episode comparisons of our system and LightNav-0. Each row follows the layout of Fig. 6 . The last row is a failure of ours and adds the ground-truth goal visibility.

Figure 11: Failure flows on clean HM3D-OVON val_unseen, as in Fig. 2 . Percentages use the 2,309 clean episodes.

	HM3D-OVON val_unseen 2,309			MP3D-OV 1,193			HSSD-OV 1,751
Outcome	Pred.	GT	Exp.	Pred.	GT	Exp.	Pred.	GT	Exp.
Success	1,606	2,115	2,218	595	883	904	988	1,350	1,426
Reached, wrong stop	82	18	0	46	5	0	31	7	1
Reached, timeout	100	34	1	45	16	0	41	23	6
Never reached, stop	427	51	63	284	9	98	408	25	104
Never reached, timeout	94	91	27	223	280	191	283	346	214

Appendix

Table 10: Clean-subset counts for Fig. 7 . Pred. uses predicted grounding, GT uses ground-truth grounding, and Exp. denotes the frontier-exploration expert.

Object-Goal Navigation

Object-goal navigation studies how robots find and approach requested objects in unfamiliar environments using visual, spatial, and language cues.

See all 82 papers

From Region Arrival to Instance-Level Grounding in Vision-and-Language Navigation
Jul 4, 2026Xiangyu Shi, Ruoxi Yang, Wei Tao +3Vision-Language Navigation Visual Grounding
Australian Institute for Machine Learning, Adelaide University · Ruyi Dynamics · Fudan University +1
Goal2Pixel: Grounding Goals to Pixels for Vision-Language Navigation
Jun 1, 2026Muyi Bao, Yuxin Cai, Hang Xu +7Embodied Navigation VLM Adaptation
Carnegie Mellon University · Nanyang Technological University
GG-Nav: Goal-oriented Grounding Chain-of-Thought Elicits Reasoning in Navigation Foundation Models
Oct 10, 2026Shaohao Zhu, Wensi Huang, Siqi Zhang +6Vision-Language Navigation Robot Navigation
Zhejiang University · Shanghai AI Laboratory · University of Science and Technology of China +2

Jul 4, 2026J/K move · Enter open · S save

From Region Arrival to Instance-Level Grounding in Vision-and-Language Navigation

Xiangyu Shi, Ruoxi Yang, Wei Tao, Jiwen Zhang+2

Australian Institute for Machine Learning, Adelaide University · Ruyi Dynamics · Fudan University+1

Vision-Language Navigation Visual Grounding Object-Goal Navigation

Open paper AskPDF arXiv

Vision-and-Language Navigation (VLN) agents may satisfy conventional success criteria while still failing to establish reliable object-level grounding, because current evaluation protocols mainly reward stopping within a 3-meter radius and largely ignore the agent's final orientation and target visibility. We formalize this limitation as the Last-3-Meter Grounding Gap and introduce three instance-centric metrics to quantify proximity precision, target visibility, and final-view grounding. To mitigate this gap, we propose REALM (Region-to-Entity Alignment for Last-3-Meter Navigation), a plug-and-play, architecture-agnostic refinement module that decouples fine-grained target approaching from long-horizon navigation. REALM uses a visibility-aware stopping strategy to reduce premature termination and improve final viewpoint alignment. We further construct REVERIE-AIM, which provides object-instance-level goals and 180K short-horizon training samples for final-stage target approaching. Extensive evaluations across four diverse VLN backbones show that REALM consistently improves proximity precision and visual grounding success, demonstrating its broad applicability.

Abstract

Object-Goal Navigation

Explore similar work

From Region Arrival to Instance-Level Grounding in Vision-and-Language Navigation

Goal2Pixel: Grounding Goals to Pixels for Vision-Language Navigation

GG-Nav: Goal-oriented Grounding Chain-of-Thought Elicits Reasoning in Navigation Foundation Models