CSG-Nav: Confidence-Aware Semantic-Geometric Fusion for Robust Exploration and Accurate Zero-Shot Object-Goal Navigation
Authors: Shoubin Chen, Rui Xu, Zehao Wu, Kai Zhang, Tao Huang, Zhou Liu, Qingquan Li
Organizations: Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen, 518132, Guangdong, China · Shenzhen University, Shenzhen, 518061, Guangdong, China
Zero-shot object-goal navigation requires autonomous agents to explore unseen environments and locate target objects without task-specific training, posing challenges in perceptual reliability under observation uncertainty and physically grounded goal termination. Although foundation-model-based approaches improve generalization, they remain vulnerable to unreliable long-range perception, hallucination-induced directional bias, and semantically plausible but geometrically incorrect stopping decisions. To address these limitations, we propose CSG-Nav, a zero-shot navigation framework centered on confidence-aware multimodal information fusion. CSG-Nav integrates three complementary fusion mechanisms. First, semantic-source fusion integrates open-vocabulary detection results, detection confidence, scene observations, and relational reasoning to produce more reliable direction-level semantic scores and suppress hallucination-induced directional bias. Second, spatial-confidence fusion weights multi-view semantic evidence using a Gaussian decay function, suppressing unreliable distant observations while accumulating spatially consistent exploration preferences. Third, semantic-geometric decision fusion combines semantic target detection with depth-based 3D localization to support physically grounded stopping decisions. Experiments on HM3Dv1, HM3Dv2, and MP3D demonstrate competitive zero-shot performance, achieving success rates of 58.9%, 71.6%, and 44.3%. Fusion-level ablations and real-world deployments in indoor and outdoor environments further verify the effectiveness and practicality of CSG-Nav for robust navigation under uncertain perception. Code is available at https://xr-ui.github.io/CSGNav.github.io/
Figures & tables
Figure 1: Motivation of CSG-Nav from an information fusion perspective. Existing semantic-dominant zero-shot navigation methods may suffer from unreliable long-range semantic evidence and false arrival caused by semantic-only stopping. CSG-Nav addresses these issues through confidence-aware semantic-spatial fusion for robust exploration and semantic-geometric decision fusion for physically grounded arrival verification.
Figure 2: Overview of the CSG-Nav multimodal fusion architecture. The framework integrates semantic-source fusion, spatial-confidence fusion, and semantic-geometric decision fusion to support exploration planning and goal-termination verification.
Fusion level
Sources fused
Output
Semantic-source fusion
YOLO-World detections, detection confidence, VLM scene descriptions, and LLM commonsense reasoning
Directional semantic scores
Spatial-confidence fusion
Multi-view semantic scores, Gaussian distance-conditioned reliability, and accessibility constraints
Global preference map
Semantic-geometric fusion
Semantic target detection, depth-based 3D localization, and distance threshold
Stop / continue decision
Table 1: Fusion hierarchy of CSG-Nav.
Figure 3: Semantic-source fusion in CSG-Nav. For each candidate direction, object-level detection results and scene evidence are fused through structured LLM reasoning to produce a semantic score and rationale.
Figure 4: Spatial-Confidence fusion. (A) Distance-aware reliability is modeled by a Gaussian decay function, where nearby observations receive higher reliability weights and distant observations receive lower weights. (B) Semantic scores are combined with accessibility and distance-aware reliability to form current spatial evidence, which is then integrated with historical preference and confidence information to update the global preference map. (C) The bottom comparison shows that confidence-aware map updating suppresses unreliable distant evidence and produces more stable exploration preferences.
Figure 5: Semantic-geometric decision fusion. CSG-Nav combines object-level semantic detection with depth-based 3D localization to evaluate whether the target satisfies the navigation arrival condition. The agent stops only when the detected target matches the goal category and satisfies the geometric distance threshold.
Method
ZS
MP
HM3Dv1
HM3Dv2
MP3D
SR ↑
SPL ↑
SR ↑
SPL ↑
SR ↑
SPL ↑
Habitat-Web [ 27 ]
×
41.5
16.0
-
-
31.6
8.5
ProcThor [ 9 ]
×
54.4
31.8
-
-
-
-
PONI [ 24 ]
×
-
-
-
-
31.8
12.1
SemEXP [ 4 ]
×
-
-
-
-
36.0
14.4
PIRLNav [ 26 ]
×
64.1
27.1
66.0
27.0
-
-
Table 2: Object Goal Navigation results on HM3Dv1, HM3Dv2 and MP3D. All values are reported as percentages (%). ZS denotes zero-shot. MP denotes the largest individual-model parameter count when publicly available; for proprietary models with undisclosed parameter counts, the corresponding model names are reported instead.
Spatial fusion strategy
SR ↑
SPL ↑
Baseline
69
28.19
Uniform weighting
67
26.38
Linear decay
64
27.64
Gaussian decay
71
28.06
Table 3: Effects of different spatial-fusion strategies on HM3Dv2.
Semantic Source
Scoring
SR ↑
SPL ↑
Raw RGB
VLM
71
28.06
YOLO-World detections
LLM
62
20.57
Raw RGB + YOLO-World detections
VLM
72
29.17
VLM scene descriptions + YOLO-World detections
LLM
75
27.62
Table 4: Effects of different semantic-source fusion configurations on navigation performance in HM3Dv2.
Arrival verification strategy
SR ↑
SPL ↑
Semantic only
75
27.62
VLM-based semantic-geometric [ 18 ]
74
27.30
YOLO-based semantic-geometric (ours)
77
28.99
Table 5: Effects of semantic-geometric decision fusion strategies on the fixed HM3Dv2 ablation subset.
Target / Setting
Distance
YOLO Det.
YOLO Conf.
VLM Recog.
Small plant, unobstructed
1 m
5/5
0.739
5/5
2 m
5/5
0.780
3/5
3 m
5/5
0.727
3/5
4 m
5/5
0.660
3/5
5 m
5/5
0.563
1/5
Large chair, doorway occlusion
1 m
5/5
0.868
5/5
Table 6: Target perception reliability at different observation distances. Det. denotes detection rate, Conf. denotes average detection confidence, and Recog. denotes recognition rate.
Figure 6: Sensitivity of navigation performance to the Gaussian decay parameter σ on HM3Dv2.
Configuration
VLM/LLM latency
YOLO latency
Spatial Fusion
Geo. Fusion
Nav. updating
Peak Memory
Qwen3-VL-4B + Qwen3-4B
1.64
0.35
0.82
0.02
16.61
7.74
Qwen2.5-VL-7B + Qwen3-8B
2.00
0.36
0.84
0.02
16.62
16.60
Table 7: Computational efficiency of different model configurations. All latency values are reported in seconds (s), and peak GPU memory is reported in GB. Geo. Fusion denotes semantic-geometric decision fusion, and Nav. updating denotes navigability updating and planning.
Figure 7: Failure analysis of the final CSG-Nav configuration on the fixed HM3Dv2 subset.
Figure 8: Navigation trajectory comparison on HM3Dv2. CSG-Nav is shown in green and WMNav in red.
Figure 9: Real-world scene demonstrations. The outdoor environment is a park with multiple regions, while the indoor environments include an office, a hallway, and an activity room.
Scene
Goal
Test
SR
Test 1
Test 2
Test 3
Test 4
Test 5
Outdoor
Fire box
×
✓
✓
×
×
2/5
Trash can
✓
×
×
×
✓
2/5
Wooden bench
×
×
×
✓
×
1/5
Electrical box
×
×
✓
×
×
1/5
Stone bench
✓
✓
×
✓
✓
4/5
Table 8: Real-world navigation results in outdoor and indoor environments.
Figure 10: Real-world deployment of CSG-Nav for object-goal navigation. Snapshots show the navigation process from start to goal across indoor and outdoor environments, demonstrating the practical feasibility of confidence-aware multimodal fusion in real robotic scenarios.
Zero-shot object-goal navigation aims to find target objects in unseen environments using only egocentric observation. Recent methods leverage foundation models' comprehension and reasoning capabilities to enhance navigation performance. However, when faced with poor viewpoints or weak semantic cues, foundation models often fail to support reliable reasoning in both perception and planning, resulting in inefficient or failed navigation. We observe that inherent relationships among objects and regions encode structured scene priors, which help agents infer plausible target locations even under partial observations. Motivated by this insight, we propose Spatial Relation-aware Navigation (SR-Nav), a framework that models both observed and experience-based spatial relationships to enhance both perception and planning. Specifically, SR-Nav first constructs a Dynamic Spatial Relationship Graph (DSRG) that encodes the target-centered spatial relationships through the foundation models and updates dynamically with real-time observations. We then introduce a Relation-aware Matching Module. It utilizes relationship matching instead of naive detection, leveraging diverse relationships in the DSRG to verify and correct errors, enhancing visual perception robustness. Finally, we design a Dynamic Relationship Planning Module to reduce the planning search space by dynamically computing the optimal paths based on the DSRG from the current position, thereby guiding planning and reducing exploration redundancy. Experiments on HM3D show that our method achieves state-of-the-art performance in both success rate and navigation efficiency. The code will be publicly available at https://github.com/Mzyw-1314/SR-Nav
Leyuan Fang, Zan Mao, Zijing Wang +1
School of Artificial Intelligence and Robotics, Hunan University, Changsha, China, 410114, China
Zero-shot object navigation requires agents to locate unseen targets in unfamiliar environments without prior maps or task-specific training. Despite the commonsense reasoning ability of vision-language models (VLMs), existing mapless navigators often suffer from repeated exploration and premature stopping caused by limited historical context and false-positive target predictions. We propose ReMemNav, a training-free framework that combines lightweight semantic grounding, memory-based decision correction, and target verification. RAM-derived semantic priors guide panoramic direction selection, while a geometry-triggered correction mechanism retrieves historical scene descriptions when the selected direction points toward previously visited regions. When a target is predicted to be visible, ReMemNav verifies the corresponding single-view observation before the final approach. Depth-based action sampling then converts the high-level decision into collision-free motion. Experiments on HM3D and MP3D show that ReMemNav achieves higher success rates and path efficiency than existing training-free zero-shot baselines. Specifically, ReMemNav achieves absolute SR/SPL gains of 1.7%/5.3% on HM3D v0.1, 18.2%/11.1% on HM3D v0.2, and 8.7%/6.9% on MP3D.
Existing zero-shot Object Goal Navigation (ObjectNav) methods often exploit commonsense knowledge from large language or vision-language models to guide navigation. However, such knowledge arises from internet-scale text rather than embodied 3D experience, and episodic observations collected during navigation are typically discarded, preventing the accumulation of lifelong experience. To this end, we propose Trajectory RAG (TrajRAG), a retrieval-augmented generation framework that enhances large-model reasoning by retrieving geometric-semantic experiences. TrajRAG incrementally accumulates episodic observations from past navigation episodes. To structure these observations, we propose a topological-polar (topo-polar) trajectory representation that compactly encodes spatial layouts and semantic contexts, effectively removing redundancies in raw episodic observations. A hierarchical chunking structure further organizes similar topo-polar trajectories into unified summaries, enabling coarse-to-fine retrieval. During navigation, candidate frontiers generate multiple trajectory hypotheses that query TrajRAG for similar past trajectories, guiding large-model reasoning for waypoint selection. New experiences are continually consolidated into TrajRAG, enabling the accumulation of lifelong navigation experience. Experiments on MP3D, HM3D-v1, and HM3D-v2 show that TrajRAG effectively retrieves relevant geometric-semantic experiences and improves zero-shot ObjectNav performance.
Yiyao Wang, Sixian Zhang, Keming Zhang +3
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing · University of Chinese Academy of Sciences, Beijing · Institute of Computing Technology, Chinese Academy of Sciences, Beijing