STEGNav: Spatio-Temporal Event Graph Reasoning for Multimodal Lifelong Object Navigation

Yang Chen; Zhenyu Huang; Wenbo Fu; Danyang Peng; Shi-Yu Tian; Kun-Yang Yu; Lan-Zhe Guo

cs.ROAug 28, 2026

STEGNav: Spatio-Temporal Event Graph Reasoning for Multimodal Lifelong Object Navigation

Authors: Yang Chen, Zhenyu Huang, Wenbo Fu, Danyang Peng, Shi-Yu Tian, Kun-Yang Yu, Lan-Zhe Guo

Organizations: State Key Laboratory of Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · School of Artificial Intelligence, Nanjing University

Abstract

Multimodal lifelong navigation requires an agent to autonomously explore unseen environments while sequentially completing navigation tasks specified by object categories, language descriptions, or reference images. Existing methods primarily accomplish these tasks by constructing state-centric semantic scene graphs. By treating scene graphs as persistent repositories of semantic observations, these methods struggle to distinguish similar instances, jointly represent semantic targets and exploration frontiers, and effectively exploit navigation memory and trajectory experience. To address these limitations, we propose Spatio-Temporal Event Graph Navigation (STEGNav), a training-free framework that extends conventional scene graphs into spatio-temporal event graphs along complementary spatial and temporal axes. The spatial axis performs query-conditioned instance grounding and jointly represents semantic targets and occupancy-aware exploration frontiers characterized by reachability, path cost, and exploration utility. The temporal axis employs trajectory-aware dual-window memory to retain recent decision--trajectory events and verified cross-subtask navigation outcomes. A VLM-based navigation agent reasons over the resulting spatio-temporal event graph and selects either a target instance or an exploration frontier as its next navigation goal. STEGNav achieves 66.3% SR and 39.7 SPL on GOAT-Bench, as well as SR scores of 64.0% and 69.4% on HM3Dv1 and HM3Dv2, respectively. Ablation studies and error analyses validate the complementary effects of the two axes, demonstrating that event-driven spatio-temporal representations improve navigation reliability and cross-subtask experience reuse.

Figures & tables

Figure 1: Limitations of state-centric scene graphs. Spatially, they suffer from instance ambiguity and semantic–spatial separation; temporally, the lack of explicit navigation history leads to memory loss and repeated exploration.

Figure 2: Overview of STEGNav. The spatial axis unifies query-conditioned instances with occupancy-aware frontiers, while the temporal axis organizes recent trajectories and verified cross-subtask outcomes. Their integration forms a spatio-temporal event graph for VLM-based navigation, verification, and continual update.

Method	Training-free	SR $\uparrow$	SPL $\uparrow$
SenseAct-NN Monolithic [CVPR24]	✗	12.3	6.8
SenseAct-NN Skill Chain [CVPR24]	✗	29.5	11.3
MTU3D [ICCV25]	✗	47.2	27.7
Modular CLIP on Wheels [CVPR24]	✓	16.1	10.4
Modular GOAT [CVPR24]	✓	24.9	17.2
VLMNav [NeSy25]	✓	20.1	9.6

Table 1: Main results on the GOAT-Bench Val-Unseen split. MSGNav does not report SPL under our evaluation protocol.

Method	Training-Free	HM3Dv1		HM3Dv2
Method	Training-Free	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$
ZSON [NeurIPS22]	✗	25.5	12.6	–	–
PixNav [ICRA24]	✗	37.9	20.5	–	–
SPNet [RA-L23]	✗	31.2	10.1	–	–
CompassNav [ICLR26]	✗	56.6	27.6	59.6	26.9
SGM [CVPR24]	✗	60.2	30.8	–	–

Table 2: Object navigation results on the HM3D benchmarks.

Module				Object		Language		Image		Overall
3D Scene Graph	QIG	OFG	DWM	SR	SPL	SR	SPL	SR	SPL	SR	SPL
✗	✗	✗	✗	50.5	24.0	41.8	22.5	44.3	27.0	45.7	24.5
✓	✗	✗	✗	61.6	30.5	51.6	32.0	54.5	38.0	55.8	33.4
✓	✓	✗	✗	64.1	32.0	55.6	35.3	58.0	41.7	59.4	36.2
✓	✓	✓	✗	70.7	37.8	61.5	38.6	68.2	47.5	66.9	41.1
✓	✓	✓	✓	74.8	40.8	63.7	38.7	73.9	49.1	70.9	42.7

Table 3: Component ablation results on the GOAT-Bench Val-Unseen subset. QIG denotes query-conditioned instance grounding, OFG denotes occupancy-aware frontier grounding, and DWM denotes trajectory-aware dual-window memory. The best result in each column is highlighted in bold .

Figure 3: Failure counts of the 3D scene-graph baseline and STEGNav on the GOAT-Bench Val-Unseen subset (278).

Figure 4: Successful language-guided navigation. Given the query “refrigerator next to the kitchen cabinet” , STEGNav progressively explores informative regions and grounds the target instance using semantic and spatial cues. The trajectory achieves an SPL of 0.587 with only 4 navigation steps, reaching the target within 0.40 m.

Figure 5: Successful object navigation. Given the query “Can you find the piano?” , STEGNav selects informative frontiers, discovers the target region, and successfully reaches the target. The trajectory achieves an SPL of 0.913 with 6 navigation steps, reaching the target within 0.97 m.

Vision-Language Navigation

Vision-Language Navigation studies how embodied agents use natural-language instructions and visual observations to plan and execute routes.

See all 296 papers

CGFM-Nav: Cognitive Graph-Field Memory for Semantic-Guided Lifelong Multimodal Embodied Navigation
Aug 29, 2026Yuxiang Xiao, Xibei Chen, Xin Zhou +3Embodied Navigation Vision-Language Navigation
Department of Mechanical Engineering, National University of Singapore, Singapore
SSTG-Nav: Metric-Grounded Spatial-Semantic Topological Graphs for Reusable Object Navigation
Aug 1, 2026Daojie Peng, Bingtao Wang, Jun MaSemantic Mapping Object-Goal Navigation
The Hong Kong University of Science and Technology (Guangzhou) · Shandong University
A Topological Representation with Object-Path Graphs for Open-Vocabulary Instance Navigation
Sep 21, 2026Linwei Zheng, Daojie Peng, Bingtao Wang +2Robot Navigation Open-Vocabulary Object Navigation
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology

Aug 29, 2026J/K move · Enter open · S save

CGFM-Nav: Cognitive Graph-Field Memory for Semantic-Guided Lifelong Multimodal Embodied Navigation

Yuxiang Xiao, Xibei Chen, Xin Zhou, Jie Chen+2

Department of Mechanical Engineering, National University of Singapore, Singapore

Embodied Navigation Vision-Language Navigation Graph-Based Agent Memory Multimodal Graph Learning

Open paper AskPDF arXiv

Vision-and-Language Navigation (VLN) requires agents to reason over accumulated observations while continuously exploring unseen regions. However, existing environment representations often struggle to jointly support explicit semantic memory and continuous exploration guidance. To address this challenge, we propose Cognitive Graph-Field Memory (CGFM), a persistent multimodal scene representation that couples explicit relational memory with continuous spatial intuition. CGFM organizes objects, spatial relations, and visual observations into a multimodal scene graph, enabling target retrieval and long-horizon reasoning across navigation tasks. When no reliable target match is identified, graph-based evidence is projected into a goal-conditioned semantic-frontier field to guide exploration toward semantically promising frontiers and regions. Building upon CGFM, we introduce CGFM-Nav, a foundation-model-based framework for lifelong multimodal navigation that integrates task-relevant subgraph selection, VLM reasoning, and verification feedback into a closed decision loop. Preliminary experiments on GOAT-Bench show that, under the same Qwen3-VL-8B backbone, CGFM-Nav improves the overall success rate from 53.2% to 63.0% and SPL from 30.0% to 39.6%, demonstrating the effectiveness of combining explicit semantic memory with semantic-guided exploration.

Abstract

Vision-Language Navigation

Explore similar work

CGFM-Nav: Cognitive Graph-Field Memory for Semantic-Guided Lifelong Multimodal Embodied Navigation

SSTG-Nav: Metric-Grounded Spatial-Semantic Topological Graphs for Reusable Object Navigation

A Topological Representation with Object-Path Graphs for Open-Vocabulary Instance Navigation