Open-vocabulary 3D scene graphs provide compact semantic memory for language-guided navigation, but mapped objects are often exposed through a single fused feature or committed semantic label. Such commitment can remove minority yet task-relevant hypotheses from the task-time interface. We present OpenBelief-Nav, an evidence-preserving object memory that retains observation-level phrases, reliability cues, and frame-mask provenance while maintaining separate aggregate geometric and visual representations. Semantically related phrases are consolidated into a vocabulary-independent object belief from which task-specific readouts perform fixed-vocabulary projection or free-form retrieval. On five ScanNet200 and eight Replica scenes, full-belief projection achieves mIoU scores of 0.2742 and 0.2912, compared with 0.2393 and 0.2701 for a matched early-commit readout. Across 78 HM3D-YCB navigation trials, consensus and early-commit retrieval each achieve 60/78 successes, compared with 58/78 for belief-weighted retrieval and 55/78 for DualMap. Across 20 Unitree G1 runs organized as 10 matched evaluation cases, a correction policy permitting at most two verified candidate attempts improves target-confirmation success from 6/10 to 8/10 relative to top-1-only execution. Code will be released upon acceptance at https://openbelief-nav.github.io/.
Figures & tables
Fig. 1 : Overview of OpenBelief - Nav. Mapping retains observation-level phrases, reliability cues, frame–mask provenance, and separate aggregate geometric and visual representations. Task-time readouts perform fixed-vocabulary projection or free-form retrieval, while arrival verification enables query-specific candidate correction without modifying the persistent graph.
System
Open-vocab
Evidence @ query
Exec.-grounded correction
Training- free
ConceptGraphs [ 1 ]
✓
✗
✗
✓
HOV-SG [ 2 ]
✓
✗
✗
✓
FSR-VLN [ 3 ]
✓
✗
✗
✓
DualMap [ 4 ]
✓
✗
△
✓
OpenBelief - Nav (ours)
✓
✓
✓
✓
TABLE I : Comparison with related open-vocabulary scene-graph systems. “Evidence @ query” denotes multiple same-object linguistic hypotheses available to task-time readouts. ✓ : supported; △ : supported through a different formulation; ✗: unsupported.
Dataset
Method
mIoU ↑
F-mIoU ↑
mAcc ↑
Replica
ConceptGraphs
0.1501
0.3858
0.3559
HOV-SG
0.2050
0.4846
0.3835
DualMap
0.2538
0.5207
0.4024
Ours (Consensus)
0.2831
0.4647
0.4774
Ours (Full belief)
0.2912
0.5204
0.4822
ScanNet200
ConceptGraphs
0.0882
0.3077
0.3538
TABLE II : Open-vocabulary 3D semantic segmentation. OpenBelief-Nav variants share the same graph. Higher is better; best and second-best results are shown in bold green and amber.
Method
00829
00848
00880
Avg. SR@1m ↑
ConceptGraphs [ 1 ]
69.2
53.8
61.5
61.5
HOV-SG [ 2 ]
53.8
46.2
57.7
52.6
DualMap [ 4 ]
73.1
69.2
69.2
70.5
Consensus-only
80.8
76.9
73.1
76.9
Early-commit
84.6
73.1
73.1
76.9
Belief-weighted
76.9
73.1
73.1
74.4
TABLE III : HM3D–YCB navigation SR@1m (%); 78 trials per method. Best and second-best results are shown in bold green and amber.
Fig. 2 : Real-world platform and deployment procedure.
Condition
Runs per method
Top-1 only Confirmation SR
Full correction Confirmation SR
Full correction mean visits
Correct
5
5/5 (100%)
5/5 (100%)
1.00
Confusable
5
1/5 (20%)
3/5 (60%)
1.80
Overall
10
6/10 (60%)
8/10 (80%)
1.40
TABLE IV : Matched real-world target-confirmation results. Each matched case is run once per method using the same frozen graph, query, object configuration, and reset start pose. The evaluation therefore contains 20 robot runs in total.
Fig. 3 : Qualitative effect of task-time semantic readout on a representative ScanNet200 scene. Parentheses report scene-level mIoU. Consensus recovers the semantic structure missed by visual-only prediction, while full-belief projection resolves additional class confusions.
Variant
ScanNet200 mIoU ↑
Nav SR@1m ↑
YCB SR@1m ↑
Default belief construction
0.2107
76.9 (60/78)
61.1 (11/18)
we=1
0.2010
70.5 (55/78)
55.6 (10/18)
we=cene
0.2117
74.4 (58/78)
50.0 (9/18)
No synonym consolidation
0.2067
75.6 (59/78)
61.1 (11/18)
No low-evidence regularization
0.2095
70.5 (55/78)
50.0 (9/18)
TABLE V : Belief-construction ablation using the consensus semantic readout. Navigation uses 78 HM3D–YCB trials, including 18 inserted-object trials.
Dataset
Semantic readout
mIoU ↑
ScanNet200
Visual-only
0.1286
Consensus-only
0.2107
Early-commit
0.2393
Full-belief projection
0.2742
Replica
Visual-only
0.2287
Consensus-only
0.2831
TABLE VI : Semantic readout ablation. Higher mIoU is better. Best results per dataset are highlighted in green (bold).
Vision-language navigation requires embodied agents to navigate environments using natural language instructions and visual observations. Existing approaches typically decompose navigation into sequential language-guided decisions or rely on online exploration without prior environmental knowledge. Scene graph representations offer compact semantic memory but remain decoupled from downstream navigation, which still depends on dense metric maps. To close this gap, we propose an object--path graph that unifies open-vocabulary semantic reasoning with topological navigation. The proposed representation jointly supports semantic grounding, graph-based localization, and navigation within a single lightweight topological framework. Building on this graph, we introduce a navigation strategy that combines global path planning with local inter-node execution through lightweight node localization and semantic visual servoing, enabling navigation directly over the graph without dense metric reconstruction. Experiments on HM3D and Replica demonstrate competitive performance in open-vocabulary object grounding through the proposed hierarchical graph structure, while achieving effective navigation performance. Real-world robot experiments further validate the practicality of the proposed framework.
Linwei Zheng, Daojie Peng, Bingtao Wang +2
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology
Open-vocabulary navigation requires embodied agents to manage significant perception uncertainty stemming from semantic ambiguity and model errors. However, most existing works settle for local optimal deterministic approaches, depriving complex navigation decision-making over multiple composite possibilities that are critical for globally better solutions. In this paper, we propose Probabilistic Scene Graph Navigation (PSG-Nav), which constructs a 3D Probabilistic Scene Graph that uses full semantic categorical distributions to account for perception uncertainty. To efficiently use the local distributions to compose and reason about the optimal navigation landmarks, we propose Multiverse Decision to sample multiple most likely world settings from the joint distribution, and evaluate navigation landmarks based on the compatibility between landmarks and multiverses. To mitigate false positives due to epistemic uncertainty in open-vocabulary navigation, we introduce the Evidential Experience Calibrator, which enables online lifelong adaptation by cross-validating detections against memories of past successes and failures. Extensive experiments on widely-used benchmarks MP3D, HM3D, and HSSD demonstrate that PSG-Nav establishes new state-of-the-art results, achieving Success Rates of 66.1%, 44.8%, and 67.9%, respectively. Code is available at: https://psg-nav.github.io/
Rufeng Chen, Yue Chang, Xiaqiang Tang +2
The Hong Kong University of Science and Technology (Guangzhou), Guangdong, China · Jilin University, Jilin, China
Object navigation requires an agent to locate a target in an unknown environment through visual observations. Existing methods typically rely on open-vocabulary detectors or vision-language models (VLMs) to answer where to search, but often overlook what not to trust - which semantic cues are unreliable. Open-vocabulary perception is prone to systematic misleading evidence: false positives, outdated static priors, and repeated failed exploration due to lack of embodied verification, which contaminates mapping and decision-making. Such errors are rooted in structured object relations in real-world scenes. To address this, we propose DB-Nav, a framework that reshapes the search space via dual relational biases. It factorizes target-centric relations into an Activation Bias (propagates contextual evidence) and an Inhibition Bias (suppresses unreliable regions via perceptual confusion and action-level falsification). These biases are unified into a Relational Activation-Inhibition Exploration Graph that modulates frontier exploration values using online observations and failed accesses. Experiments on ObjectNav benchmarks show that DB-Nav significantly outperforms existing methods in success rate (SR) and Success weighted by Path Length (SPL), offering a lightweight, interpretable, and robust navigation framework without costly online VLM reasoning.
Weitao An, Chenghao Xu, Xu Yang +1
School of Electronic Engineering, Xidian University, Xi’an 710071, China · School of Information Science and Engineering, Hohai University, Nanjing 210098, China