Zero-shot object-goal navigation aims to find target objects in unseen environments using only egocentric observation. Recent methods leverage foundation models' comprehension and reasoning capabilities to enhance navigation performance. However, when faced with poor viewpoints or weak semantic cues, foundation models often fail to support reliable reasoning in both perception and planning, resulting in inefficient or failed navigation. We observe that inherent relationships among objects and regions encode structured scene priors, which help agents infer plausible target locations even under partial observations. Motivated by this insight, we propose Spatial Relation-aware Navigation (SR-Nav), a framework that models both observed and experience-based spatial relationships to enhance both perception and planning. Specifically, SR-Nav first constructs a Dynamic Spatial Relationship Graph (DSRG) that encodes the target-centered spatial relationships through the foundation models and updates dynamically with real-time observations. We then introduce a Relation-aware Matching Module. It utilizes relationship matching instead of naive detection, leveraging diverse relationships in the DSRG to verify and correct errors, enhancing visual perception robustness. Finally, we design a Dynamic Relationship Planning Module to reduce the planning search space by dynamically computing the optimal paths based on the DSRG from the current position, thereby guiding planning and reducing exploration redundancy. Experiments on HM3D show that our method achieves state-of-the-art performance in both success rate and navigation efficiency. The code will be publicly available at https://github.com/Mzyw-1314/SR-Nav
Figures & tables
Fig. 1: (a) Our Spatial Relationship Graph (SRG) integrates experiential spatial priors and perceptual observations to enhance target perception via relational matching and guide efficient path planning through spatial reasoning. (b) Comparison of successful episodes by step count: our perception+experience approach (red) achieves higher success rates at fewer exploration steps than perception-only methods (blue), demonstrating that experiential knowledge provides effective guidance in early, observation-limited navigation.
Fig. 2: Comparison with existing methods. (a) Geometry-based methods perceive targets via object detection and solely rely on geometric cues for planning. (b) Semantic-based methods incorporate target semantics to guide planning. (c) Our method employs spatial relationship matching for target perception and utilizes target-associated spatial relationships for planning guidance.
Fig. 3: Compared to scene graphs, which are limited by perception and mostly target-irrelevant, our method leverages both observed and experience-based relationships, which are predictive and target-relevant, enabling more effective navigation guidance.
Fig. 4: Overview of the SR-Nav. The LLM initializes a Dynamic Spatial Relationships Graph (DSRG) for the target. At each time-step, the agent collects RGB-D observations and updates the DSRG via VLM reasoning. The RAMM module corrects FP/FN errors using spatial priors. When the target is not detected, DRPM integrates DSRG localization and relational cues to generate prompts, ranks scene regions via VLM semantic similarity, and selects the highest-scoring frontier for local navigation.
Fig. 5: Relationship-aware Matching Module. This module refines raw detections using target-related spatial priors from the DSRG. It uses relationship matching to suppress false positives and identify potential false negatives, ensuring more reliable detection results.
Fig. 6: Examples of RAMM for perception.
Fig. 7: Examples of DRPM for planning.
Method
Venue
Unsupervised
Zero-shot
HM3D
MP3D
SR ↑
SPL ↑
SR ↑
SPL ↑
SemExp
NeurIPS’20
✗
✗
37.9
18.8
36.0
14.4
PONI
CVPR’22
✗
✗
-
-
31.8
12.1
SGM
CVPR’24
✗
✗
60.2
30.8
37.7
14.7
ZSON
NeurIPS’22
✗
✓
25.5
12.6
15.3
4.8
PixNav
ICRA’24
✗
✓
37.9
20.5
-
-
TABLE I: Zero-shot object navigation results on HM3D v0.1 and MP3D benchmarks.
Fig. 8: Qualitative comparison of navigation trajectories on the HM3D v0.1 dataset. Rows 1 and 2 demonstrate higher navigational efficiency and overall success rate of our method, respectively.
Fig. 9: Visualization of the navigation process of SR-Nav.
Fig. 10: Validation of RAMM and DRPM. (a) Superior detection precision and recall across all object categories via spatial relationship matching. (b) Higher semantic similarity scores throughout planning with spatial relationship guidance.
Base
DRPM
RAMM
SR
SPL
Object
Region
(%)
(%)
✓
✗
✗
✗
52.5
30.3
✓
✓
✗
✗
53.1
29.2
✓
✓
✓
✗
54.5
32.3
✓
✓
✓
✓
58.3
33.0
TABLE II: An ablation study of different modules on HM3D v0.1. ’Object’ and ’Region’ indicate the incorporation of object-level and region-level spatial relations during planning, respectively. ’RAMM’ denotes the Relation-Aware Matching Module, which is used to verify and correct detection results.
Method
Model
SR(%)
SPL(%)
Times ↓ (min/episode)
SG-Nav
LLaMA-7B
53.9
24.8
13
WMNav
Qwen-2.5-VL-7B
46.1
20.7
2.0
Gemini 1.5 Pro
58.1
31.2
–
ours
InternVL-3-8B
55.1
32.6
1.5
Qwen-2.5-VL-7B
58.3
33.0
TABLE III: Performance and computational efficiency comparison.
Disturbance Type
Prompt Content
SR(%) ↑
SPL(%) ↑
Seems like there is a < target >ahead
55.4
32.1
A < target >can be in the vicinity.
56.5
32.3
Template
Seems like a < target >ahead
55.7
32.4
You may find < target >nearby
55.3
30.9
< target >
51.3
28.4
W/o Rdis
56.0
31.5
TABLE IV: Prompt Sensitivity and Relationship Ablation Study.
Method
bed
chair
plant
sofa
toilet
tv_monitor
Average
L3MVN
52.9
51.6
46.4
50.1
41.5
54.2
49.5
VLFM
50.1
66.6
44.0
48.1
58.5
34.9
52.5
SR-Nav (ours)
55.4
68.9
42.8
54.1
65.3
44.8
58.3
SR-Nav (ours) ∗
-
-
-
-
-
-
61.3 ∗
TABLE V: Performance comparison on object-goal navigation across six target categories and average. Best results are in bold . We manually verified and annotated the corresponding results. Entries based on manual annotation are marked with an asterisk ( ∗ ).
Fig. 11: Real-world deployment. SR-Nav enables zero-shot object navigation in diverse and complex real-world environments by leveraging experience-based spatial priors to guide exploration and correct target detection results.
Zero-shot object navigation requires agents to locate unseen targets in unfamiliar environments without prior maps or task-specific training. Despite the commonsense reasoning ability of vision-language models (VLMs), existing mapless navigators often suffer from repeated exploration and premature stopping caused by limited historical context and false-positive target predictions. We propose ReMemNav, a training-free framework that combines lightweight semantic grounding, memory-based decision correction, and target verification. RAM-derived semantic priors guide panoramic direction selection, while a geometry-triggered correction mechanism retrieves historical scene descriptions when the selected direction points toward previously visited regions. When a target is predicted to be visible, ReMemNav verifies the corresponding single-view observation before the final approach. Depth-based action sampling then converts the high-level decision into collision-free motion. Experiments on HM3D and MP3D show that ReMemNav achieves higher success rates and path efficiency than existing training-free zero-shot baselines. Specifically, ReMemNav achieves absolute SR/SPL gains of 1.7%/5.3% on HM3D v0.1, 18.2%/11.1% on HM3D v0.2, and 8.7%/6.9% on MP3D.
Navigating to instance-level targets in complex environments is a challenging problem. Many existing zero-shot methods achieve strong performance by modeling the entire environment and leveraging large language models for scene understanding. However, such strategies primarily focus on exploring new regions while lacking a deeper exploitation of information from previously explored areas. Consequently, when targets are missed or misidentified within previously visited regions, navigation failures occur frequently. To address these limitations, we propose MCNav, a memory-aware navigation framework with a dynamic cognitive map. This map stores efficiently queryable information about relevant objects in explored areas. Building on this memory structure, MCNav introduces two memory-aware exploration strategies: goal re-validation, which re-assesses previously seen objects to correct matching failures, and missed goal re-exploration, which estimates the likelihood that a target is present in an explored region from contextual cues. These strategies are further stabilized by a blacklist mechanism to prevent repeated errors and a double-check mechanism for high-confidence confirmation. We evaluate MCNav on the HM3Dv1 and HM3Dv2 datasets across three different tasks, where it achieves state-of-the-art performance, particularly on the instance-level goal navigation task.
Jingyu Li, Zhe Liu, Wenxiao Wu +1
Fudan University · Shanghai Innovation Institute · University of Hong Kong +1
Zero-shot object-goal navigation (ZSON) is a challenging problem in robotics that requires a comprehensive understanding of both language and visual observations. Contextual cues from rooms and objects are critical, but their relative importance depends on the target: some objects are strongly tied to specific room types, while others are better predicted by nearby co-located objects. Existing methods overlook this distinction, leading to inefficient and inaccurate exploration. We present CLUE, a novel navigation framework that adaptively balances the use of contextual rooms and objects by leveraging commonsense knowledge extracted from an offline large language model (LLM). By estimating a target's association with room types using LLM, the agent prioritizes room cues for predictable objects and object cues for those with weak room associations. Our framework constructs a unified semantic value map that integrates both types of contextual information, adaptively weighted by the target's ambiguity to guide exploration. Combined with multi-viewpoint verification and an exploration strategy informed by contextual cues, CLUE achieves robust and efficient navigation. Extensive experiments in simulation and real-world deployments show that our method consistently outperforms state-of-the-art baselines in both success rate (SR) and success weighted by path length (SPL), demonstrating its effectiveness and practicality for real-world navigation tasks.
Taeyun Kim, Alvin Jinsung Choi, Dasol Hong +1
School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology), Daejeon, 34141, Republic of Korea.