Navigating to out-of-sight targets from human instructions in unfamiliar environments is a core capability for service robots. Despite substantial progress, most approaches underutilize reusable, persistent memory, constraining performance in lifelong settings. Many are additionally limited to single-modality inputs and employ myopic greedy policies, which often induce inefficient back-and-forth maneuvers (BFMs). To address such limitations, we introduce SSMG-Nav, a framework for object navigation built on a Semantic Skeleton Memory Graph (SSMG) that consolidates past observations into a spatially aligned, persistent memory anchored by topological keypoints (e.g., junctions, room centers). SSMG clusters nearby entities into subgraphs, unifying entity- and space-level semantics to yield a compact set of candidate destinations. To support multimodal targets (images, objects, and text), we integrate a vision-language model (VLM). For each subgraph, a multimodal prompt synthesized from memory guides the VLM to infer a target belief over destinations. A long-horizon planner then trades off this belief against traversability costs to produce a visit sequence that minimizes expected path length, thereby reducing backtracking. Extensive experiments on challenging lifelong benchmarks and standard ObjectNav benchmarks demonstrate that, compared to strong baselines, our method achieves higher success rates and greater path efficiency, validating the effectiveness of SSMG-Nav.
Figures & tables
Fig. 1 : An overview of the proposed SSMG-Nav.
Method
Zero-shot
GOAT val_seen
GOAT val_unseen
s-SR (↑)
e-SR (↑)
SPL (↑)
s-SR (↑)
e-SR (↑)
SPL (↑)
SenseAct-NN Skill Chain [ 8 ]
×
0.292
–
0.128
0.295
–
0.113
SenseAct-NN Monolithic [ 8 ]
×
0.168
–
0.094
0.123
–
0.068
SenseAct-NN Monolithic †
×
0.159
0.003
0.104
0.126
0.008
0.088
CoW [ 3 ]
✓
0.148
–
0.087
0.161
–
0.104
Modular GOAT [ 8 ]
✓
0.263
–
0.175
0.249
–
0.172
TABLE I : Lifelong Multimodal Object Navigation Comparison.
Method
Zero- shot
HM3D
MP3D
SR (↑)
SPL (↑)
SR (↑)
SPL (↑)
PONI [ 9 ]
×
–
–
0.318
0.121
Habitat-Web [ 2 ]
×
0.415
0.160
0.316
0.085
ZSON [ 16 ]
×
0.255
0.126
0.153
0.048
SGM [ 15 ]
×
0.602
0.308
0.377
0.147
ESC [ 10 ]
✓
0.392
0.223
0.287
0.142
TABLE II : Object Navigation Comparison.
Fig. 2 : Performance over different modalities.
Fig. 3 : Performance over time.
Method
s-SR
e-SR
SPL
Map
Planner
Semantic Map
Greedy
0.389
0.056
0.278
Value Map
Greedy
0.442
0.083
0.258
SSMG
Greedy
0.453
0.056
0.328
SSMG
LHP w/o revisiting
0.479
0.083
0.361
SSMG
LHP
0.502
0.083
0.376
TABLE III : Ablation Study.
Fig. 4 : Real-world demonstration of SSMG-Nav in an office environment with three subtasks.
Vision-and-Language Navigation (VLN) requires agents to reason over accumulated observations while continuously exploring unseen regions. However, existing environment representations often struggle to jointly support explicit semantic memory and continuous exploration guidance. To address this challenge, we propose Cognitive Graph-Field Memory (CGFM), a persistent multimodal scene representation that couples explicit relational memory with continuous spatial intuition. CGFM organizes objects, spatial relations, and visual observations into a multimodal scene graph, enabling target retrieval and long-horizon reasoning across navigation tasks. When no reliable target match is identified, graph-based evidence is projected into a goal-conditioned semantic-frontier field to guide exploration toward semantically promising frontiers and regions. Building upon CGFM, we introduce CGFM-Nav, a foundation-model-based framework for lifelong multimodal navigation that integrates task-relevant subgraph selection, VLM reasoning, and verification feedback into a closed decision loop. Preliminary experiments on GOAT-Bench show that, under the same Qwen3-VL-8B backbone, CGFM-Nav improves the overall success rate from 53.2% to 63.0% and SPL from 30.0% to 39.6%, demonstrating the effectiveness of combining explicit semantic memory with semantic-guided exploration.
Yuxiang Xiao, Xibei Chen, Xin Zhou +3
Department of Mechanical Engineering, National University of Singapore, Singapore
Object Goal Navigation (ObjectNav) refers to an agent navigating to an object in an unseen environment, which is an ability often required in the accomplishment of complex tasks. While existing methods demonstrate proficiency in isolated single object navigation, their limitations emerge in the restricted applicability of lifelong memory representations, which ultimately hinders effective navigation toward continual targets over extended periods. To address this problem, we propose OVAL, a novel lifelong open-vocabulary memory framework, which enables efficient and precise execution of long-term navigation in semantically open tasks. Within this framework, we introduce memory descriptors to facilitate structured management of the memory model. Additionally, we propose a novel probability-based exploration strategy, utilizing a multi-value frontier scoring to enhance lifelong exploration efficiency. Extensive experiments demonstrate the efficiency and robustness of the proposed system.
Jiahua Pei, Yi Liu, Guoping Pan +3
Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
Multimodal lifelong navigation requires an agent to autonomously explore unseen environments while sequentially completing navigation tasks specified by object categories, language descriptions, or reference images. Existing methods primarily accomplish these tasks by constructing state-centric semantic scene graphs. By treating scene graphs as persistent repositories of semantic observations, these methods struggle to distinguish similar instances, jointly represent semantic targets and exploration frontiers, and effectively exploit navigation memory and trajectory experience. To address these limitations, we propose Spatio-Temporal Event Graph Navigation (STEGNav), a training-free framework that extends conventional scene graphs into spatio-temporal event graphs along complementary spatial and temporal axes. The spatial axis performs query-conditioned instance grounding and jointly represents semantic targets and occupancy-aware exploration frontiers characterized by reachability, path cost, and exploration utility. The temporal axis employs trajectory-aware dual-window memory to retain recent decision--trajectory events and verified cross-subtask navigation outcomes. A VLM-based navigation agent reasons over the resulting spatio-temporal event graph and selects either a target instance or an exploration frontier as its next navigation goal. STEGNav achieves 66.3% SR and 39.7 SPL on GOAT-Bench, as well as SR scores of 64.0% and 69.4% on HM3Dv1 and HM3Dv2, respectively. Ablation studies and error analyses validate the complementary effects of the two axes, demonstrating that event-driven spatio-temporal representations improve navigation reliability and cross-subtask experience reuse.
Yang Chen, Zhenyu Huang, Wenbo Fu +4
State Key Laboratory of Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · School of Artificial Intelligence, Nanjing University