Graph-MambaNav: Spatial-Temporal Graph Mamba Leveraging Object-Relation Knowledge for Object-Goal Navigation
Authors: Leyuan Sun, Genxin Chen, Linwei Ye, Yan Zhang, Xi Kan, Yanfei Sun
Organizations: School of Internet of Things Engineering, Wuxi University, Wuxi, Jiangsu 214105, China · School of Communications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing, Jiangsu 210003, China · Wuxi Key Laboratory of Artificial Intelligence and Security, Wuxi, Jiangsu 214105, China · Wuxi University, Wuxi, Jiangsu 214105, China
Object-goal navigation requires an agent to reason over object relationships and prioritize target-relevant objects for efficient decision making in unseen environments. While existing graph-based methods incorporate target-awareness at the feature or attention level, they remain permutation-invariant and lack an explicit mechanism to control information propagation order, limiting their ability to model target-dependent importance and long-range dependencies. In contrast, Graph-Mamba highlights that node prioritization through sequence ordering is critical for effective global reasoning. In this work, we investigate the node prioritization mechanism in Graph-Mamba and study its role in object navigation. We propose Graph-MambaNav, a target-aware spatial-temporal graph encoding framework that introduces a heuristic ordering over objects based on their relevance to the target, allowing more informative objects to be processed later to aggregate richer context. Both node ordering and edge weights are initialized from LLM-derived commonsense object relationships, providing a unified prior for structured reasoning. A spatial module integrates local message passing with global GraphMamba-based selective scanning, while a temporal module applies Mamba-based sequence modeling over object-wise temporal orders, allowing selective aggregation of historical context for long-range temporal reasoning. Experiments on AI2-THOR and RoboTHOR demonstrate improved navigation performance with generalization, and additional real-world robot deployment further validates the effectiveness of our proposed approach.
Figures & tables
Fig. 1 : Motivation and overview of Graph-MambaNav. Given an egocentric RGB observation, an object-centric graph is constructed from detected objects. LLM-derived commonsense relations provide a unified prior for both edge attribution and target-aware heuristic ordering . The ordered spatial scan enables more target-relevant objects to aggregate richer context under Mamba-based scanning, while the temporal scan models object-wise dynamics across time for action prediction.
Fig. 2 : Overview of Graph-MambaNav (Left) and its object-graph spatial-temporal encoding pipeline (Right). Given an egocentric RGB observation ot , ResNet extracts global scene-level features V , while DETR provides complementary object-level features, boxes, and detection scores for object graph construction. LLM-derived commonsense priors initialize edge embeddings and define target-aware node ordering. The enlarged module on the right shows the three key graph encoding operations: local edge-aware message passing, global target-aware Graph-Mamba scanning based on object-node prioritization, and temporal scanning over object-wise memory. The target-conditioned attention fusion module produces graph feature G , which is fused with V and previous-action embedding Aact for LSTM-based A3C policy learning.
Fig. 3 : Left: LLM-derived object affinity matrices across two datasets. Middle: Qualitative comparison of TSOG [ 20 ] and Graph-MambaNav in AI2-THOR, covering four different room types. Right: Graph-MambaNav results in RoboTHOR, where apartments are constructed modularly from a shared asset library. Object-relation heatmaps for the target are shown to the left of the top-down maps. In the top-down views, targets are highlighted in red and related objects in green, with the corresponding first-person observations shown below.
Method
Params
GFLOPs
GPU Memory
FPS
(M)
(GB) per agent
T35-Transformer variant
62.13
82.39
6.72
11.32
T35-Graph-MambaNav
12.58
23.47
2.53
22.12
TABLE I : Comparison of computational complexity. We report the floating point operations (FLOPs), number of model parameters, FPS, and GPU memory consumption of the proposed T35-Graph-MambaNav and the alternative Transformer-based variant, both using a memory sequence length of 35.
Method
AI2-THOR
RoboTHOR
ALL
L≥5
ALL
L≥5
SR ↑ (%)
SPL ↑ (%)
SR ↑ (%)
SPL ↑ (%)
SR ↑ (%)
SPL ↑ (%)
SR ↑ (%)
SPL ↑ (%)
Random
3.56
1.73
0.27
0.07
-
-
-
-
Baseline [ 27 ]
57.35
33.78
45.77
30.65
26.41
16.61
17.42
12.23
SP [ 17 ]
62.16
37.01
50.86
34.17
28.04
17.63
21.66
15.14
SAVN [ 28 ]
63.32
37.62
52.38
35.31
28.42
17.82
22.13
15.34
TABLE II : Comparisons with other methods in the AI2-THOR and RoboTHOR environments
Fig. 4 : Navigation performance across varying T . Red pentagon markers denote the optimal configurations.
Fig. 5 : SR versus relation variance of each target object between Graph-MambaNav and TSOG. Graph-MambaNav shows a steeper trend and higher SR, demonstrating its advantage in leveraging prior LLM-derived relational knowledge.
Method
L≥10 / L≥15 / L≥20
SR ↑ (%)
SPL ↑ (%)
VTNet
42.5 / 29.6 / 10.9
29.1 / 18.5 / 6.3
DOA
54.88 / 25.18 / 10.91
37.15 / 17.83 / 8.35
TSOG
58.49 / 35.31 / 23.64
37.18 / 22.11 / 16.31
M-MambaNav
60.23 / 42.35/ 29.35
38.81 / 25.93 / 16.21
Ours w/o TS
61.23 / 41.02 / 27.32
40.33 / 28.19 / 15.65
TABLE III : Comparison under different path lengths (10, 15, and 20) in AI2-THOR, where TS denotes Temporal Scan
Setting
SR ↑ (%)
SPL ↑ (%)
Top-5 ordered nodes
77.05±0.91
43.10±0.76
Top-10 ordered nodes
81.56±0.78
45.65±0.64
Top-15 ordered nodes
82.63±0.72
46.32±0.59
Top-20 ordered nodes
82.84±0.69
46.30±0.57
All-22 ordered nodes
83.22±0.75
46.52±0.61
All-22 + 10% noisy relations
82.31±0.83
45.51±0.71
TABLE IV : Analysis of ordering scalability and robustness to noisy object-relation priors. Due to the fixed object categories in the offline benchmark, we vary the number of nodes using target-aware ordering and perturb the LLM-derived affinity matrix to simulate noisy relations.
ID
Method
ALL / L≥5
LS
GS
TS
SR ↑ (%)
SPL ↑ (%)
I
1
63.83 / 52.45
33.75 / 30.33
2
✓
66.15 / 61.42
41.35 / 35.43
3
✓
✓
78.34 / 71.66
43.39 / 42.85
4
✓
✓
✓
83.22 / 76.09
46.52 / 46.20
II
5
Ours w/ F-R LS edges
66.26 / 62.98
40.34 / 38.23
TABLE V : Ablation study, impacts of different components. All variants in Group II retain the full LS+GS+TS pipeline and modify only the specified relation prior or scan order.
Fig. 6 : Real-world navigation illustration. Left: robot platform; middle: trajectory visualization; right: perception results. Bottom: object relation heatmap for the target book . The point cloud and odometry are for visualization only and are not accessible to the robot.
Object-Goal Navigation (ObjNav) requires embodied agents to autonomously locate specified targets using only egocentric visual observations. Existing monolithic methods struggle with long-horizon reasoning and generalize poorly to novel environments. To address these limitations, we propose SAGE-Nav, a novel hierarchical framework that integrates the reasoning capabilities of Large Language Models (LLMs) with dynamic scene graphs. Crucially, it decouples asynchronous global semantic planning from the high-frequency reactive control loop. The LLM serves as a global planner, decomposing abstract instructions into a sequence of semantically grounded waypoints. To translate these plans into dense multi-modal guidance, we design a Hierarchical Scene Graph Encoder (HSGE) that leverages relational graph convolutions to produce structure-aware embeddings preserving both semantic and spatial topology. Furthermore, we develop the Goal-aware Alignment-Fusion Network (GAFN) to dynamically fuse real-time perception with these structural priors. Using an adaptive gating mechanism with an explicit inductive bias, GAFN ensures robust visual-topological alignment for the low-level policy. Extensive evaluations in the i-THOR and RoboTHOR environments demonstrate that SAGE-Nav achieves state-of-the-art performance, delivering substantial gains in navigation efficiency and zero-shot generalization while maintaining the low control latency required for physical robotic deployment.
Multimodal lifelong navigation requires an agent to autonomously explore unseen environments while sequentially completing navigation tasks specified by object categories, language descriptions, or reference images. Existing methods primarily accomplish these tasks by constructing state-centric semantic scene graphs. By treating scene graphs as persistent repositories of semantic observations, these methods struggle to distinguish similar instances, jointly represent semantic targets and exploration frontiers, and effectively exploit navigation memory and trajectory experience. To address these limitations, we propose Spatio-Temporal Event Graph Navigation (STEGNav), a training-free framework that extends conventional scene graphs into spatio-temporal event graphs along complementary spatial and temporal axes. The spatial axis performs query-conditioned instance grounding and jointly represents semantic targets and occupancy-aware exploration frontiers characterized by reachability, path cost, and exploration utility. The temporal axis employs trajectory-aware dual-window memory to retain recent decision--trajectory events and verified cross-subtask navigation outcomes. A VLM-based navigation agent reasons over the resulting spatio-temporal event graph and selects either a target instance or an exploration frontier as its next navigation goal. STEGNav achieves 66.3% SR and 39.7 SPL on GOAT-Bench, as well as SR scores of 64.0% and 69.4% on HM3Dv1 and HM3Dv2, respectively. Ablation studies and error analyses validate the complementary effects of the two axes, demonstrating that event-driven spatio-temporal representations improve navigation reliability and cross-subtask experience reuse.
Yang Chen, Zhenyu Huang, Wenbo Fu +4
State Key Laboratory of Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · School of Artificial Intelligence, Nanjing University
Vision-language navigation requires embodied agents to navigate environments using natural language instructions and visual observations. Existing approaches typically decompose navigation into sequential language-guided decisions or rely on online exploration without prior environmental knowledge. Scene graph representations offer compact semantic memory but remain decoupled from downstream navigation, which still depends on dense metric maps. To close this gap, we propose an object--path graph that unifies open-vocabulary semantic reasoning with topological navigation. The proposed representation jointly supports semantic grounding, graph-based localization, and navigation within a single lightweight topological framework. Building on this graph, we introduce a navigation strategy that combines global path planning with local inter-node execution through lightweight node localization and semantic visual servoing, enabling navigation directly over the graph without dense metric reconstruction. Experiments on HM3D and Replica demonstrate competitive performance in open-vocabulary object grounding through the proposed hierarchical graph structure, while achieving effective navigation performance. Real-world robot experiments further validate the practicality of the proposed framework.
Linwei Zheng, Daojie Peng, Bingtao Wang +2
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology