Have You Seen It Clearly? Uncertainty-Guided Belief Refinement for Zero-Shot Object Navigation
Authors: Jingzhi Huang, Junkai Huang, Yuhang Peng, Keyu Mu, Jiaru Zhong, Yi Wang, Lujia Wang, Lap-Pui Chau, Siyang Gao
Organizations: Hong Kong Polytechnic University · Institute of Automation, Chinese Academy of Sciences · DeepRobotics · Hong Kong University of Science and Technology (Guangzhou) · City University of Hong Kong
Zero-Shot Object Navigation (ZSON) requires an agent to locate target objects in unfamiliar environments without additional training for the test categories or environments. Recent approaches leverage large language models and vision-language models to construct spatial representations and guide exploration. However, whether the available evidence sufficiently supports a navigation choice, and how new evidence changes the search value of previously visited locations, remain insufficiently addressed. We propose RefBeliefMap, an uncertainty-guided mapping framework that distinguishes target-location belief from spatial and semantic evidence uncertainty. By jointly modeling intra-observation uncertainty and inter-observation disagreement over a sliding memory window, RefBeliefMap assesses the reliability of both evidence channels. When competing frontiers remain insufficiently resolved, these uncertainty estimates guide active observation and semantic reasoning to refine the evidence supporting target-location beliefs before navigation commitment. Building on the refined map, we introduce RefBeliefNav, which jointly evaluates unexplored frontiers and historical waypoints according to their target-finding potential, expected information gain, and navigation cost. This unified objective allows historical locations to be reconsidered as new evidence changes their search value, coordinating exploration and revisitation. Experiments on HM3D and MP3D demonstrate state-of-the-art performance among the evaluated zero-shot baselines, with ablation studies examining the contributions of modules. Real-world experiments on a Unitree Go2 quadruped robot further demonstrate the sim-to-real transferability of our method.
Figures & tables
Figure 1: Overview of RefBeliefNav. RefBeliefNav comprises three components: RefBeliefMap models target relevance and evidence uncertainty; evidence refinement improves insufficient support through active observation and semantic reasoning; unified search planning selects among frontiers and historical waypoints.
Method
Venue
HM3D
MP3D
SR ↑
SPL ↑
SR ↑
SPL ↑
ZSON Majumdar et al. (2022)
NeurIPS’22
25.5
12.6
15.3
4.8
ESC Zhou et al. (2023)
ICML’23
39.2
22.3
28.7
14.2
L3MVN Yu et al. (2023)
IROS’23
50.4
23.1
34.9
14.5
PSL Sun et al. (2024)
ECCV’24
42.4
19.2
18.9
6.9
VLFM Yokoyama et al. (2024a)
ICRA’24
52.5
30.4
36.4
17.5
Table 1: Comparison of RefBeliefNav and zero-shot methods on the HM3D and MP3D datasets.
Table 3
Figure 2: Qualitative examples of RefBeliefNav searching for a toilet (top) and a sofa (bottom) in simulation. Each panel shows the evolving RefBeliefMap and the corresponding egocentric RGB observation. Orange, red, and dark blue hollow circles denote historical waypoints, frontiers, and the selected waypoint, respectively. The light blue circle marks the radius- R neighborhood centered on the robot used to evaluate refinement triggers (Sec. 3.4.1 ).
Figure 3: Real-world searches for a vending machine in an open public area (top) and a bottle in a classroom (bottom). The sequences progress from left to right, with each panel pairing an external view of the robot with its egocentric RGB observation. Bounding boxes indicate object detections.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Table 6
Trigger Strategy
LLM Calls ↓
SR ↑
SPL ↑
Fixed steps (per 10)
37.2
66.7
33.5
Fixed steps (per 30)
13.2
66.1
33.2
Adaptive
8.2
69.5
35.5
Appendix
Table 6: Impact of refinement-triggering strategy.
Table 8Table 9Table 10
(a) Planning weights
λunc
λcost
SR ↑
SPL ↑
0.125
2.5
68.7
34.7
0.125
5.0
68.4
35.0
0.125
10.0
67.0
34.5
0.250
2.5
69.3
34.8
0.250
5.0
69.5
35.5
Appendix
Table 13: Joint parameter sensitivity: (a) planning weights and (b) trigger thresholds. λcost is expressed in m−1 .
Zero-shot object navigation requires agents to locate unseen targets in unfamiliar environments without prior maps or task-specific training. Despite the commonsense reasoning ability of vision-language models (VLMs), existing mapless navigators often suffer from repeated exploration and premature stopping caused by limited historical context and false-positive target predictions. We propose ReMemNav, a training-free framework that combines lightweight semantic grounding, memory-based decision correction, and target verification. RAM-derived semantic priors guide panoramic direction selection, while a geometry-triggered correction mechanism retrieves historical scene descriptions when the selected direction points toward previously visited regions. When a target is predicted to be visible, ReMemNav verifies the corresponding single-view observation before the final approach. Depth-based action sampling then converts the high-level decision into collision-free motion. Experiments on HM3D and MP3D show that ReMemNav achieves higher success rates and path efficiency than existing training-free zero-shot baselines. Specifically, ReMemNav achieves absolute SR/SPL gains of 1.7%/5.3% on HM3D v0.1, 18.2%/11.1% on HM3D v0.2, and 8.7%/6.9% on MP3D.
Zero-shot object-goal navigation (ZSON) in open-vocabulary scenarios is challenging, as it requires a robot to locate an arbitrarily specified object in an unseen environment without task-specific training. Currently, the task still suffers from high latency and limited accuracy due to redundant perception pipelines and insufficient evidence for reliable target confirmation. In this letter, we reframe ZSON as an evidence-driven perception-to-decision problem and present AECNav, a training-free pipeline built on three components: i) Evidence-gated perception, which utilizes a shared encoding across all reasoning stages to establish a unified semantic basis and eliminate redundant computations; ii) Evidence consolidation, which aggregates detections into cluster-level log-odds beliefs. This explicitly separates genuine target support from the false confidence of visually similar distractors, while treating the absence of expected detections as negative evidence; and iii) Active evidence acquisition, which sustains productive exploration under weak semantic cues by selecting frontiers that maximize information gain at minimal traversal cost. As a result, AECNav significantly outperforms previous methods and achieves state-of-the-art success rates of 84.7%, 57.3%, and 51.3% on HM3D-v2, HM3D-OVON, and MP3D, respectively, with substantially lower inference overhead, and attains 95% success across 40 trials on a physical quadruped robot at roughly 5Hz. Code will be made publicly available upon acceptance.
Guanlin Liu, Shaobin Ling, Renyuan Liu +2
The Chinese University of Hong Kong, Shenzhen · The Hong Kong University of Science and Technology (Guangzhou)
Zero-shot object navigation (ZSON) requires robots to find target objects in unseen environments without task-specific fine-tuning or pre-built maps, a key capability for general-purpose service robots. Yet methods that perform well in simulation often degrade in cluttered real-world scenes with severe occlusion and latent hazards, where large unseen regions make single-scene inference brittle and unsafe. We propose Schrödinger's Navigator, a belief-aware framework that reasons at inference time over multiple trajectory-conditioned imagined 3D futures. Given candidate paths, a trajectory-conditioned 3D world model predicts hypothetical observations and maintains a superposition of plausible scene realizations rather than committing to one map. An adaptive occluder-aware sampler directs imagination to uncertainty-critical regions, while a Future-Aware Value Map (FAVM) aggregates imagined futures for robust, proactive action selection. Experiments in simulation and on a physical Go2 quadruped show that Schrödinger's Navigator outperforms strong ZSON baselines, improving hidden-target discovery and risk-aware waypoint selection in occlusion-heavy navigation scenarios. These results highlight imagined 3D futures as a scalable and generalizable strategy for zero-shot navigation in uncertain real-world environments.
Yu He, Da Huang, Zhenyang Liu +5
Fudan University · Shanghai Innovation Institute · ActiMind +3