Have You Seen It Clearly? Uncertainty-Guided Belief Refinement for Zero-Shot Object Navigation

Jingzhi Huang; Junkai Huang; Yuhang Peng; Keyu Mu; Jiaru Zhong; Yi Wang; Lujia Wang; Lap-Pui Chau

cs.ROOct 11, 2026

Have You Seen It Clearly? Uncertainty-Guided Belief Refinement for Zero-Shot Object Navigation

Authors: Jingzhi Huang, Junkai Huang, Yuhang Peng, Keyu Mu, Jiaru Zhong, Yi Wang, Lujia Wang, Lap-Pui Chau,

Organizations: Hong Kong Polytechnic University · Institute of Automation, Chinese Academy of Sciences · DeepRobotics · Hong Kong University of Science and Technology (Guangzhou) ·

Abstract

Zero-Shot Object Navigation (ZSON) requires an agent to locate target objects in unfamiliar environments without additional training for the test categories or environments. Recent approaches leverage large language models and vision-language models to construct spatial representations and guide exploration. However, whether the available evidence sufficiently supports a navigation choice, and how new evidence changes the search value of previously visited locations, remain insufficiently addressed. We propose RefBeliefMap, an uncertainty-guided mapping framework that distinguishes target-location belief from spatial and semantic evidence uncertainty. By jointly modeling intra-observation uncertainty and inter-observation disagreement over a sliding memory window, RefBeliefMap assesses the reliability of both evidence channels. When competing frontiers remain insufficiently resolved, these uncertainty estimates guide active observation and semantic reasoning to refine the evidence supporting target-location beliefs before navigation commitment. Building on the refined map, we introduce RefBeliefNav, which jointly evaluates unexplored frontiers and historical waypoints according to their target-finding potential, expected information gain, and navigation cost. This unified objective allows historical locations to be reconsidered as new evidence changes their search value, coordinating exploration and revisitation. Experiments on HM3D and MP3D demonstrate state-of-the-art performance among the evaluated zero-shot baselines, with ablation studies examining the contributions of modules. Real-world experiments on a Unitree Go2 quadruped robot further demonstrate the sim-to-real transferability of our method.

Figures & tables

Figure 1: Overview of RefBeliefNav. RefBeliefNav comprises three components: RefBeliefMap models target relevance and evidence uncertainty; evidence refinement improves insufficient support through active observation and semantic reasoning; unified search planning selects among frontiers and historical waypoints.

Method	Venue	HM3D		MP3D
Method	Venue	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$
ZSON Majumdar et al. (2022)	NeurIPS’22	25.5	12.6	15.3	4.8
ESC Zhou et al. (2023)	ICML’23	39.2	22.3	28.7	14.2
L3MVN Yu et al. (2023)	IROS’23	50.4	23.1	34.9	14.5
PSL Sun et al. (2024)	ECCV’24	42.4	19.2	18.9	6.9
VLFM Yokoyama et al. (2024a)	ICRA’24	52.5	30.4	36.4	17.5

Table 1: Comparison of RefBeliefNav and zero-shot methods on the HM3D and MP3D datasets.

Table 3

Figure 2: Qualitative examples of RefBeliefNav searching for a toilet (top) and a sofa (bottom) in simulation. Each panel shows the evolving RefBeliefMap and the corresponding egocentric RGB observation. Orange, red, and dark blue hollow circles denote historical waypoints, frontiers, and the selected waypoint, respectively. The light blue circle marks the radius- $R$ neighborhood centered on the robot used to evaluate refinement triggers (Sec. 3.4.1 ).

Figure 3: Real-world searches for a vending machine in an open public area (top) and a bottle in a classroom (bottom). The sequences progress from left to right, with each panel pairing an external view of the robot with its egocentric RGB observation. Bounding boxes indicate object detections.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Table 6

Trigger Strategy	LLM Calls $\downarrow$	SR $\uparrow$	SPL $\uparrow$
Fixed steps (per 10)	37.2	66.7	33.5
Fixed steps (per 30)	13.2	66.1	33.2
Adaptive	8.2	69.5	35.5

Appendix

Table 6: Impact of refinement-triggering strategy.

Table 8

Table 9

Table 10

(a) Planning weights
$\lambda_{\mathrm{unc}}$	$\lambda_{\mathrm{cost}}$	SR $\uparrow$	SPL $\uparrow$
0.125	2.5	68.7	34.7
0.125	5.0	68.4	35.0
0.125	10.0	67.0	34.5
0.250	2.5	69.3	34.8
0.250	5.0	69.5	35.5

Appendix

Table 13: Joint parameter sensitivity: (a) planning weights and (b) trigger thresholds.

\lambda_{\mathrm{cost}}

is expressed in

\mathrm{m}^{-1}

Robot Navigation

Robot navigation studies how robots perceive, localize, plan, and move safely toward goals in static or changing environments.

See all 536 papers

ReMemNav: Memory-Based Decision Correction and Target Verification for Zero-Shot Object Navigation
Mar 25, 2026Feng Wu, Wei Zuo, Wenliang Yang +3Robot Navigation Vision-Language Navigation
Fudan University · Tongji University
AECNav: Active Evidence Consolidation for Efficient Zero-Shot Open-Vocabulary Object Navigation
Aug 11, 2026Guanlin Liu, Shaobin Ling, Renyuan Liu +2Open-Vocabulary Object Navigation Object-Goal Navigation
The Chinese University of Hong Kong, Shenzhen · The Hong Kong University of Science and Technology (Guangzhou)
Schrödinger's Navigator: Imagining an Ensemble of Futures for Zero-Shot Object Navigation
Dec 24, 2025Yu He, Da Huang, Zhenyang Liu +53D Scene Prediction Robot Navigation
Fudan University · Shanghai Innovation Institute · ActiMind +3

Mar 25, 2026J/K move · Enter open · S save

ReMemNav: Memory-Based Decision Correction and Target Verification for Zero-Shot Object Navigation

Feng Wu, Wei Zuo, Wenliang Yang, Jun Xiao+2

Fudan University · Tongji University

Robot Navigation Vision-Language Navigation Object-Goal Navigation VLMs for Robotics

Open paper AskPDF arXiv

Zero-shot object navigation requires agents to locate unseen targets in unfamiliar environments without prior maps or task-specific training. Despite the commonsense reasoning ability of vision-language models (VLMs), existing mapless navigators often suffer from repeated exploration and premature stopping caused by limited historical context and false-positive target predictions. We propose ReMemNav, a training-free framework that combines lightweight semantic grounding, memory-based decision correction, and target verification. RAM-derived semantic priors guide panoramic direction selection, while a geometry-triggered correction mechanism retrieves historical scene descriptions when the selected direction points toward previously visited regions. When a target is predicted to be visible, ReMemNav verifies the corresponding single-view observation before the final approach. Depth-based action sampling then converts the high-level decision into collision-free motion. Experiments on HM3D and MP3D show that ReMemNav achieves higher success rates and path efficiency than existing training-free zero-shot baselines. Specifically, ReMemNav achieves absolute SR/SPL gains of 1.7%/5.3% on HM3D v0.1, 18.2%/11.1% on HM3D v0.2, and 8.7%/6.9% on MP3D.

Abstract

Robot Navigation

Explore similar work

ReMemNav: Memory-Based Decision Correction and Target Verification for Zero-Shot Object Navigation

AECNav: Active Evidence Consolidation for Efficient Zero-Shot Open-Vocabulary Object Navigation

Schrödinger's Navigator: Imagining an Ensemble of Futures for Zero-Shot Object Navigation