CSG-Nav: Confidence-Aware Semantic-Geometric Fusion for Robust Exploration and Accurate Zero-Shot Object-Goal Navigation
Authors: Shoubin Chen, Rui Xu, Zehao Wu, Kai Zhang, Tao Huang, Zhou Liu, Qingquan Li
Organizations: Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen, 518132, Guangdong, China · Shenzhen University, Shenzhen, 518061, Guangdong, China
Zero-shot object-goal navigation requires autonomous agents to explore unseen environments and locate target objects without task-specific training, posing challenges in perceptual reliability under observation uncertainty and physically grounded goal termination. Although foundation-model-based approaches improve generalization, they remain vulnerable to unreliable long-range perception, hallucination-induced directional bias, and semantically plausible but geometrically incorrect stopping decisions. To address these limitations, we propose CSG-Nav, a zero-shot navigation framework centered on confidence-aware multimodal information fusion. CSG-Nav integrates three complementary fusion mechanisms. First, semantic-source fusion integrates open-vocabulary detection results, detection confidence, scene observations, and relational reasoning to produce more reliable direction-level semantic scores and suppress hallucination-induced directional bias. Second, spatial-confidence fusion weights multi-view semantic evidence using a Gaussian decay function, suppressing unreliable distant observations while accumulating spatially consistent exploration preferences. Third, semantic-geometric decision fusion combines semantic target detection with depth-based 3D localization to support physically grounded stopping decisions. Experiments on HM3Dv1, HM3Dv2, and MP3D demonstrate competitive zero-shot performance, achieving success rates of 58.9%, 71.6%, and 44.3%. Fusion-level ablations and real-world deployments in indoor and outdoor environments further verify the effectiveness and practicality of CSG-Nav for robust navigation under uncertain perception. Code is available at https://xr-ui.github.io/CSGNav.github.io/
Figures & tables
Figure 1: Motivation of CSG-Nav from an information fusion perspective. Existing semantic-dominant zero-shot navigation methods may suffer from unreliable long-range semantic evidence and false arrival caused by semantic-only stopping. CSG-Nav addresses these issues through confidence-aware semantic-spatial fusion for robust exploration and semantic-geometric decision fusion for physically grounded arrival verification.
Figure 2: Overview of the CSG-Nav multimodal fusion architecture. The framework integrates semantic-source fusion, spatial-confidence fusion, and semantic-geometric decision fusion to support exploration planning and goal-termination verification.
Fusion level
Sources fused
Output
Semantic-source fusion
YOLO-World detections, detection confidence, VLM scene descriptions, and LLM commonsense reasoning
Directional semantic scores
Spatial-confidence fusion
Multi-view semantic scores, Gaussian distance-conditioned reliability, and accessibility constraints
Global preference map
Semantic-geometric fusion
Semantic target detection, depth-based 3D localization, and distance threshold
Stop / continue decision
Table 1: Fusion hierarchy of CSG-Nav.
Figure 3: Semantic-source fusion in CSG-Nav. For each candidate direction, object-level detection results and scene evidence are fused through structured LLM reasoning to produce a semantic score and rationale.
Figure 4: Spatial-Confidence fusion. (A) Distance-aware reliability is modeled by a Gaussian decay function, where nearby observations receive higher reliability weights and distant observations receive lower weights. (B) Semantic scores are combined with accessibility and distance-aware reliability to form current spatial evidence, which is then integrated with historical preference and confidence information to update the global preference map. (C) The bottom comparison shows that confidence-aware map updating suppresses unreliable distant evidence and produces more stable exploration preferences.
Figure 5: Semantic-geometric decision fusion. CSG-Nav combines object-level semantic detection with depth-based 3D localization to evaluate whether the target satisfies the navigation arrival condition. The agent stops only when the detected target matches the goal category and satisfies the geometric distance threshold.
Method
ZS
MP
HM3Dv1
HM3Dv2
MP3D
SR ↑
SPL ↑
SR ↑
SPL ↑
SR ↑
SPL ↑
Habitat-Web [ 27 ]
×
41.5
16.0
-
-
31.6
8.5
ProcThor [ 9 ]
×
54.4
31.8
-
-
-
-
PONI [ 24 ]
×
-
-
-
-
31.8
12.1
SemEXP [ 4 ]
×
-
-
-
-
36.0
14.4
PIRLNav [ 26 ]
×
64.1
27.1
66.0
27.0
-
-
Table 2: Object Goal Navigation results on HM3Dv1, HM3Dv2 and MP3D. All values are reported as percentages (%). ZS denotes zero-shot. MP denotes the largest individual-model parameter count when publicly available; for proprietary models with undisclosed parameter counts, the corresponding model names are reported instead.
Spatial fusion strategy
SR ↑
SPL ↑
Baseline
69
28.19
Uniform weighting
67
26.38
Linear decay
64
27.64
Gaussian decay
71
28.06
Table 3: Effects of different spatial-fusion strategies on HM3Dv2.
Semantic Source
Scoring
SR ↑
SPL ↑
Raw RGB
VLM
71
28.06
YOLO-World detections
LLM
62
20.57
Raw RGB + YOLO-World detections
VLM
72
29.17
VLM scene descriptions + YOLO-World detections
LLM
75
27.62
Table 4: Effects of different semantic-source fusion configurations on navigation performance in HM3Dv2.
Arrival verification strategy
SR ↑
SPL ↑
Semantic only
75
27.62
VLM-based semantic-geometric [ 18 ]
74
27.30
YOLO-based semantic-geometric (ours)
77
28.99
Table 5: Effects of semantic-geometric decision fusion strategies on the fixed HM3Dv2 ablation subset.
Target / Setting
Distance
YOLO Det.
YOLO Conf.
VLM Recog.
Small plant, unobstructed
1 m
5/5
0.739
5/5
2 m
5/5
0.780
3/5
3 m
5/5
0.727
3/5
4 m
5/5
0.660
3/5
5 m
5/5
0.563
1/5
Large chair, doorway occlusion
1 m
5/5
0.868
5/5
Table 6: Target perception reliability at different observation distances. Det. denotes detection rate, Conf. denotes average detection confidence, and Recog. denotes recognition rate.
Figure 6: Sensitivity of navigation performance to the Gaussian decay parameter σ on HM3Dv2.
Configuration
VLM/LLM latency
YOLO latency
Spatial Fusion
Geo. Fusion
Nav. updating
Peak Memory
Qwen3-VL-4B + Qwen3-4B
1.64
0.35
0.82
0.02
16.61
7.74
Qwen2.5-VL-7B + Qwen3-8B
2.00
0.36
0.84
0.02
16.62
16.60
Table 7: Computational efficiency of different model configurations. All latency values are reported in seconds (s), and peak GPU memory is reported in GB. Geo. Fusion denotes semantic-geometric decision fusion, and Nav. updating denotes navigability updating and planning.
Figure 7: Failure analysis of the final CSG-Nav configuration on the fixed HM3Dv2 subset.
Figure 8: Navigation trajectory comparison on HM3Dv2. CSG-Nav is shown in green and WMNav in red.
Figure 9: Real-world scene demonstrations. The outdoor environment is a park with multiple regions, while the indoor environments include an office, a hallway, and an activity room.
Scene
Goal
Test
SR
Test 1
Test 2
Test 3
Test 4
Test 5
Outdoor
Fire box
×
✓
✓
×
×
2/5
Trash can
✓
×
×
×
✓
2/5
Wooden bench
×
×
×
✓
×
1/5
Electrical box
×
×
✓
×
×
1/5
Stone bench
✓
✓
×
✓
✓
4/5
Table 8: Real-world navigation results in outdoor and indoor environments.
Figure 10: Real-world deployment of CSG-Nav for object-goal navigation. Snapshots show the navigation process from start to goal across indoor and outdoor environments, demonstrating the practical feasibility of confidence-aware multimodal fusion in real robotic scenarios.
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing · University of Chinese Academy of Sciences, Beijing · Institute of Computing Technology, Chinese Academy of Sciences, Beijing