Visual navigation requires the robot to reach a specified goal such as an image, based on a sequence of first-person visual observations. While recent learning-based approaches have made significant progress, they often focus on improving policy heads or decision strategies while relying on simplistic feature encoders and temporal pooling to represent visual input. This leads to the loss of fine-grained spatial and temporal structure, ultimately limiting accurate action prediction and progress estimation. In this paper, we propose a unified spatio-temporal representation framework that enhances visual encoding for robotic navigation. Our approach extracts features from both image sequences and goal observations, and fuses them using the designed spatio-temporal fusion module. This module performs spatial graph reasoning within each frame and models temporal dynamics using a hybrid temporal shift module combined with multi-resolution difference-aware convolution. Experimental results demonstrate that our approach consistently improves navigation performance and offers a generalizable visual backbone for goal-conditioned control. Code is available at https://github.com/hren20/STRNet.
Figures & tables
Figure 1: t-SNE projections of feature embeddings colored by ground-truth temporal distances. (a) Conventional temporal-context pooling encoder (NoMaD [ 39 ] ) produces entangled embeddings, mixing near- and far-to-goal states. (b) Proposed STRNet, using graph-based spatial aggregation and hybrid spatio-temporal fusion, yields clearly separated embeddings, effectively capturing spatial and temporal cues.
Figure 2: Pipeline of the Proposed Model for Action Prediction: The model processes input observations and goal images through feature extraction, spatial feature aggregation, temporal feature fusion, and hybrid temporal shift, followed by task-specific processing, including temporal distance computation and diffusion denoising to obtain final action prediction.
Figure 3: (a) A Grid structure representing a partitioned image, and (b) A Graph structure illustrating the relationships between different regions of the image. The graph structure organizes the context in a more flexible way that aligns with semantic topological relationships, avoiding the local receptive field or predefined order between patches.
Figure 4: Qualitative navigation trajectories (blue) produced by STRNet in 2D-3D-S and Citysim Environments.
Scene
Method
Basic Task
Adaptation Task
Length (m)
Collision
Success
Length (m)
Collision
Success
Indoor (2D-3D-S)
ViNT
56.84 ± 0.098
0
100%
37.60 ± 0.116
0.06
80%
NoMaD
59.08 ± 2.563
0.52
70%
37.49 ± 0.278
0.38
72%
NoMaD-0
57.06 ± 9.384
0.24
84%
39.38 ± 0.602
0.26
82%
NaviBridger
57.60 ± 1.982
0.20
90%
37.37 ± 0.090
0
100%
STRNet (Ours)
56.02 ± 0.071
0
100%
37.07 ± 0.110
0.02
100%
Table 1: Quantitative comparison between the proposed method with baselines in simulation environments
Method
SR (%)
Avg. Colli.
Avg. SPL
VINT
0.68
0.71
0.77
NoMaD
0.51
1.95
0.33
NaviBridger
0.72
0.59
0.71
STRNet(Ours)
0.79
0.51
0.80
Table 2: Comparison of Different Methods on GRScenes
Method
Length (m)
Collision
Success
ViNT
148.53 ± 0.420
1.2
68%
NoMaD
159.65 ± 157.9642
1.08
30%
NaviBridger
156.23 ± 1.164
1.73
58%
STRNet (Ours)
145.63 ± 6.80
0.02
98%
Table 3: Performance metrics for long-horizon task in 2D-3D-S environment [ 3 ] .
Method
Length (m)
Collision
Success
ViNT
19.02 ± 0.27
0.52
68%
NoMaD
20.51 ± 9.29
0.90
34%
NaviBridger
19.28 ± 0.82
0.47
76%
STRNet (Ours)
19.10 ± 0.70
0.34
84%
Table 4: Comparison of different methods after adding noise to sub-target selection.
Figure 5: Failure cases of the NoMaD in visual navigation. (a) Suboptimal behavior caused by poor representation. (b) Hesitation due to incorrect understanding. (c) Incorrect motion direction and erratic trajectory. (d) Increased collisions.
SA
Temporal Fusion
Length (m)
Collision
Success
✓
✓
145.63 ± 6.80
0.02
98%
✗
✓
144.60 ± 14.63
0.12
88%
✓
✗
146.28 ± 4.83
0.78
38%
✗
✗
145.31 ± 1.15
0.86
28%
Table 5: Ablation studies on long-horizon task.
Figure 6: Schematic diagram of front-view projection visualization for action prediction in real-world experiments.
Method
Parameters
FLOPs
Time
ViNT
14.04M
26.89M
30.08ms
NoMaD
13.60M
1.70M
36.06ms
STRNet (Ours)
11.83M
3.56M
36.52ms
Table 6: Model complexity and inference time comparison.