OpenBelief-Nav: Evidence-Preserving Object Memory for Open-Vocabulary Language-Guided Navigation

Dinh Tuan Nguyen; Anh Dao; Phuong Nam Dang; Quan-Dung Pham; Tuyen P. Le; Truong Nguyen; Quan Nguyen

cs.CVAug 14, 2026

OpenBelief-Nav: Evidence-Preserving Object Memory for Open-Vocabulary Language-Guided Navigation

Authors: Dinh Tuan Nguyen, Anh Dao, Phuong Nam Dang, Quan-Dung Pham, Tuyen P. Le, Truong Nguyen, Quan Nguyen

Organizations: VinMotion, Inc., Vietnam · University of Southern California, USA

Abstract

Open-vocabulary 3D scene graphs provide compact semantic memory for language-guided navigation, but mapped objects are often exposed through a single fused feature or committed semantic label. Such commitment can remove minority yet task-relevant hypotheses from the task-time interface. We present OpenBelief-Nav, an evidence-preserving object memory that retains observation-level phrases, reliability cues, and frame-mask provenance while maintaining separate aggregate geometric and visual representations. Semantically related phrases are consolidated into a vocabulary-independent object belief from which task-specific readouts perform fixed-vocabulary projection or free-form retrieval. On five ScanNet200 and eight Replica scenes, full-belief projection achieves mIoU scores of 0.2742 and 0.2912, compared with 0.2393 and 0.2701 for a matched early-commit readout. Across 78 HM3D-YCB navigation trials, consensus and early-commit retrieval each achieve 60/78 successes, compared with 58/78 for belief-weighted retrieval and 55/78 for DualMap. Across 20 Unitree G1 runs organized as 10 matched evaluation cases, a correction policy permitting at most two verified candidate attempts improves target-confirmation success from 6/10 to 8/10 relative to top-1-only execution. Code will be released upon acceptance at https://openbelief-nav.github.io/.

Figures & tables

Fig. 1 : Overview of OpenBelief - Nav. Mapping retains observation-level phrases, reliability cues, frame–mask provenance, and separate aggregate geometric and visual representations. Task-time readouts perform fixed-vocabulary projection or free-form retrieval, while arrival verification enables query-specific candidate correction without modifying the persistent graph.

System	Open-vocab	Evidence @ query	Exec.-grounded correction	Training- free
ConceptGraphs [ 1 ]	✓	✗	✗	✓
HOV-SG [ 2 ]	✓	✗	✗	✓
FSR-VLN [ 3 ]	✓	✗	✗	✓
DualMap [ 4 ]	✓	✗	$\triangle$	✓
OpenBelief - Nav (ours)	✓	✓	✓	✓

TABLE I : Comparison with related open-vocabulary scene-graph systems. “Evidence @ query” denotes multiple same-object linguistic hypotheses available to task-time readouts. ✓ : supported;

\triangle

: supported through a different formulation; ✗: unsupported.

Dataset	Method	mIoU $\uparrow$	F-mIoU $\uparrow$	mAcc $\uparrow$
Replica	ConceptGraphs	0.1501	0.3858	0.3559
	HOV-SG	0.2050	0.4846	0.3835
	DualMap	0.2538	0.5207	0.4024
	Ours (Consensus)	0.2831	0.4647	0.4774
	Ours (Full belief)	0.2912	0.5204	0.4822
ScanNet200	ConceptGraphs	0.0882	0.3077	0.3538

TABLE II : Open-vocabulary 3D semantic segmentation. OpenBelief-Nav variants share the same graph. Higher is better; best and second-best results are shown in bold green and amber.

Method	00829	00848	00880	Avg. SR@1m $\uparrow$
ConceptGraphs [ 1 ]	69.2	53.8	61.5	61.5
HOV-SG [ 2 ]	53.8	46.2	57.7	52.6
DualMap [ 4 ]	73.1	69.2	69.2	70.5
Consensus-only	80.8	76.9	73.1	76.9
Early-commit	84.6	73.1	73.1	76.9
Belief-weighted	76.9	73.1	73.1	74.4

TABLE III : HM3D–YCB navigation SR@1m (%); 78 trials per method. Best and second-best results are shown in bold green and amber.

Panel of Figure 5 — Fig. 2 : Real-world platform and deployment procedure.

Condition	Runs per method	Top-1 only Confirmation SR	Full correction Confirmation SR	Full correction mean visits
Correct	5	5/5 (100%)	5/5 (100%)	1.00
Confusable	5	1/5 (20%)	3/5 (60%)	1.80
Overall	10	6/10 (60%)	8/10 (80%)	1.40

TABLE IV : Matched real-world target-confirmation results. Each matched case is run once per method using the same frozen graph, query, object configuration, and reset start pose. The evaluation therefore contains 20 robot runs in total.

Fig. 3 : Qualitative effect of task-time semantic readout on a representative ScanNet200 scene. Parentheses report scene-level mIoU. Consensus recovers the semantic structure missed by visual-only prediction, while full-belief projection resolves additional class confusions.

Variant	ScanNet200 mIoU $\uparrow$	Nav SR@1m $\uparrow$	YCB SR@1m $\uparrow$
Default belief construction	0.2107	76.9 (60/78)	61.1 (11/18)
$w_{e}=1$	0.2010	70.5 (55/78)	55.6 (10/18)
$w_{e}=c_{e}n_{e}$	0.2117	74.4 (58/78)	50.0 (9/18)
No synonym consolidation	0.2067	75.6 (59/78)	61.1 (11/18)
No low-evidence regularization	0.2095	70.5 (55/78)	50.0 (9/18)

TABLE V : Belief-construction ablation using the consensus semantic readout. Navigation uses 78 HM3D–YCB trials, including 18 inserted-object trials.

Dataset	Semantic readout	mIoU $\uparrow$
ScanNet200	Visual-only	0.1286
	Consensus-only	0.2107
	Early-commit	0.2393
	Full-belief projection	0.2742
Replica	Visual-only	0.2287
	Consensus-only	0.2831

TABLE VI : Semantic readout ablation. Higher mIoU is better. Best results per dataset are highlighted in green (bold).

Vision-Language Navigation

Vision-Language Navigation studies how embodied agents use natural-language instructions and visual observations to plan and execute routes.

See all 290 papers

A Topological Representation with Object-Path Graphs for Open-Vocabulary Instance Navigation
Sep 21, 2026Linwei Zheng, Daojie Peng, Bingtao Wang +2Robot Navigation Open-Vocabulary Object Navigation
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology
PSG-Nav: Probabilistic Scene Graph Navigation via Multiverse Decision Making
May 31, 2026Rufeng Chen, Yue Chang, Xiaqiang Tang +2Continual Robot Learning Embodied Navigation
The Hong Kong University of Science and Technology (Guangzhou), Guangdong, China · Jilin University, Jilin, China
Rethinking Embodied Navigation via Relational Inductive Bias
Jun 9, 2026Weitao An, Chenghao Xu, Xu Yang +1Embodied Navigation Relational Reasoning
School of Electronic Engineering, Xidian University, Xi’an 710071, China · School of Information Science and Engineering, Hohai University, Nanjing 210098, China

Sep 21, 2026J/K move · Enter open · S save

A Topological Representation with Object-Path Graphs for Open-Vocabulary Instance Navigation

Linwei Zheng, Daojie Peng, Bingtao Wang, Haoang Li+1

The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology

Robot Navigation Open-Vocabulary Object Navigation Vision-Language Navigation Graph-Based Planning Object-Goal Navigation

Open paper AskPDF arXiv

Vision-language navigation requires embodied agents to navigate environments using natural language instructions and visual observations. Existing approaches typically decompose navigation into sequential language-guided decisions or rely on online exploration without prior environmental knowledge. Scene graph representations offer compact semantic memory but remain decoupled from downstream navigation, which still depends on dense metric maps. To close this gap, we propose an object--path graph that unifies open-vocabulary semantic reasoning with topological navigation. The proposed representation jointly supports semantic grounding, graph-based localization, and navigation within a single lightweight topological framework. Building on this graph, we introduce a navigation strategy that combines global path planning with local inter-node execution through lightweight node localization and semantic visual servoing, enabling navigation directly over the graph without dense metric reconstruction. Experiments on HM3D and Replica demonstrate competitive performance in open-vocabulary object grounding through the proposed hierarchical graph structure, while achieving effective navigation performance. Real-world robot experiments further validate the practicality of the proposed framework.

Abstract

Vision-Language Navigation

Explore similar work

A Topological Representation with Object-Path Graphs for Open-Vocabulary Instance Navigation

PSG-Nav: Probabilistic Scene Graph Navigation via Multiverse Decision Making

Rethinking Embodied Navigation via Relational Inductive Bias