Vision-language Navigation (VLN) requires an agent to understand visual observations and language instructions to navigate in unseen environments. Most existing approaches rely on static scene assumptions and struggle to generalize in dynamic, real-world scenarios. To address this challenge, we propose DyGeoVLN, a dynamic geometry-aware VLN framework. Our method infuses a dynamic geometry foundation model into the VLN framework through cross-branch feature fusion to enable explicit 3D spatial representation and visual-semantic reasoning. To efficiently compress historical token information in long-horizon, dynamic navigation, we further introduce a novel pose-free and adaptive-resolution token-pruning strategy. This strategy can remove spatio-temporal redundant tokens to reduce inference cost. Extensive experiments demonstrate that our approach achieves state-of-the-art performance on multiple benchmarks and exhibits strong robustness in real-world environments.
Figures & tables
Figure 1: The system overview of our DyGeoVLN. Given instruction and sequential images, the vision encoder and our proposed dynamic geometry foundation model (DGFM) respectively produce visual tokens and geometry tokens in a stream manner. Then these tokens are fused via cross-branch feature fusion to obtain spatial-semantic fused tokens. Finally, the LLM integrates the fused tokens and instruction tokens, followed by de-tokenizing the predicted sequence directly into robot actions. To efficiently compress the historical token, we build a spatial token pruning strategy on top of our DGFM , yielding a compact set of pruned spatial-semantic tokens.
Figure 2: Overview of the proposed DGFM. (a) The architecture of our dynamic geometry foundation model: 3D latent representations and 2D visual semantic features are fused and subsequently aligned along the temporal axis to achieve globally consistent and fine-grained reconstruction. (b) Qualitative reconstruction results compared against Ground Truth, π3 [ 23 ] , and VGGT [ 21 ] . The red bounding boxes highlight regions where both π3 and VGGT suffer from poor human reconstruction. In contrast, our method preserves precise human positioning and maintains stable geometric structures.
Figure 3: Example case of our DyHM3D dataset.
Methods
Observation Data
Val Seen
Val Unseen
Complex Data
Simple Data
NE ↓
TCR ↓
CR ↓
SR ↑
NE ↓
TCR ↓
CR ↓
SR ↑
VLN-CMA [ 11 ]
✓
7.63
63.09
0.75
0.04
7.34
47.06
0.77
0.07
HA-VLN-VL [ 7 ]
✓
5.02
4.44
0.52
0.20
5.25
6.63
0.59
0.14
BEVBert ∗ [ 1 ]
✓
5.53
3.64
0.46
0.27
5.51
4.71
0.55
0.21
ETPNav ∗ [ 2 ]
✓
5.17
4.07
0.43
0.24
5.43
6.94
0.58
0.17
g3D-LF ∗ [ 24 ]
✓
5.12
3.58
0.41
0.32
5.30
4.54
0.49
0.27
Table 1: Comparison with state-of-the-art methods on dynamic HA-VLN [ 7 ] benchmark. ∗ indicates methods using the pretrained waypoint predictor from [ 9 ] . Complex Data indicates the method simultaneously uses panorama RGB-D images and/or odometry. Simple Data indicates the method only uses Monocular RGB images.
Figure 4: Qualitative comparison between state-of-the-art method StreamVLN [ 30 ] and our DyGeoVLN on the dynamic HA-VLN [ 7 ] benchmark. StreamVLN loses track of the path and fails to handle dynamic human context, while DyGeoVLN better understands human behaviors and accurately reaches the target destination.
Methods
Observation Data
R2R-CE Val-Unseen
Complex Data
Simple Data
NE ↓
OSR ↑
SR ↑
SPL ↑
GridMM ∗ [ 26 ]
✓
5.11
61.0
49.0
41.0
ETPNav ∗ [ 2 ]
✓
4.71
65.0
57.0
49.0
ScaleVLN ∗ [ 28 ]
✓
4.80
64.0
55.0
51.0
g3D-LF ∗ [ 24 ]
✓
4.53
68.0
61.0
52.0
CM2 [ 8 ]
✓
7.02
41.5
34.3
27.6
Table 2: Comparison with state-of-the-art methods on R2R-CE [ 11 ] dataset. ∗ indicates methods using the waypoint predictor from [ 9 ] . Complex Data indicates the method simultaneously uses panorama RGB-D images and/or odometry. Simple Data indicates the method only uses Monocular RGB images.
Figure 5: Qualitative comparison between state-of-the-art method StreamVLN [ 30 ] and our DyGeoVLN on the R2R-CE [ 11 ] benchmark. DyGeoVLN demonstrates improved trajectory consistency, whereas StreamVLN stops midway and fails to reach the destination.
Figure 6: Quantitative comparison of success rates (%) across varying real-world scenarios. Our proposed DyGeoVLN consistently outperforms other baseline models, demonstrating particular robustness across diverse environments.
Figure 7: Qualitative results of our DyGeoVLN model. The robot successfully follows instructions to reach the goal while avoiding collisions with moving pedestrians.
Methods
NE ↓
TCR ↓
CR ↓
SR ↑
DyGeoVLN (Full)
5.12
3.69
0.38
0.40
w/o Visual Semantic
6.82
4.96
0.51
0.30
w/o Spatial Geometry
5.54
4.03
0.43
0.34
w/o Dynamic Spatial Injection
5.37
4.10
0.43
0.36
w/o Spatial Token Pruning
5.33
3.94
0.39
0.37
Table 3: Ablation Study of each component in our proposed DyGeoVLN.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Example cases of our DyHM3D dataset.
Figure 9: Qualitative comparison of reconstruction performance on dynamic scenes.
Figure 10: Qualitative results of our DyGeoVLN model on dynamic HA-VLN [ 7 ] benchmark. (Cases 1-2)
Figure 11: Qualitative results of our DyGeoVLN model on dynamic HA-VLN [ 7 ] benchmark. (Cases 3-5 continued).
Figure 12: Qualitative results of our DyGeoVLN model on R2R-CE [ 11 ] benchmark.
Figure 13: Setup of real-world experiments: (a) Hardware configuration of the quadruped robot; (b) Schematic of the real-world deployment pipeline.
Figure 14: Qualitative results of our DyGeoVLN model on real-world experiments: including dynamic and static scenarios.
Vision-Language Navigation (VLN) approaches have currently followed two primary paradigms: the end-to-end Vision-Language Model (VLM) policy fine-tuned on navigation trajectories to directly predict actions, and the zero-shot modular pipeline integrating pre-trained Multimodal Large Language Model (MLLM) for training-free generalization to unseen environments. However, end-to-end methods struggle with long-horizon navigation and lack dynamic reasoning ability, whereas zero-shot methods are constrained by limited spatial grounding for reliable planning and require substantial reasoning time. To bridge this gap, we introduce SEDualVLN, a spatially-enhanced dual-system VLN framework. System 1 is a VLM model enhanced with both global and local spatial awareness, tailored for high-frequency action generation. System 2 integrates a general MLLM with a mapping module, wherein the MLLM performs low-frequency waypoint planning by leveraging top-down views of the real-time 3D map alongside streams of rendered path images. Both systems exploit complementary spatial representations to improve the agent's spatial understanding in VLN tasks. Ultimately, they cooperate to complete the navigation task through a fast-slow coordinated approach. SEDualVLN achieves state-of-the-art performance on VLN-CE benchmarks, and further ablation studies demonstrate the effectiveness of each system and module.
Jingzhi Huang, Junkai Huang, Wenxuan Song +5
Hong Kong Polytechnic University · Institute of Automation, Chinese Academy of Sciences · Hong Kong University of Science and Technology (Guangzhou)
Vision-and-Language Navigation (VLN) increasingly relies on large vision-language models, but their inference cost conflicts with real-time deployment. Token caching is a promising training-free strategy that avoids redundant computation by reusing stable visual tokens across frames. However, existing methods assume a static camera and fixed semantic focus, assumptions that VLN fundamentally violates. We identify two failure modes: (1) visual dynamics, where viewpoint shift displaces token positions across frames, causing position-wise matching to pair misaligned content; (2) semantic dynamics, where token relevance shifts across task stages as navigation progresses, making cached states stale. We propose VLN-Cache, a visual-dynamic-aware and semantic-dynamic-aware caching framework that introduces view-aligned remapping to recover geometric correspondences and a task-relevance saliency filter to veto reuse at semantic transitions. A layer-adaptive entropy policy further balances the per-layer reuse budget. Experiments on the R2R-CE simulation benchmark show up to 1.52x speedup while maintaining competitive navigation success rates.
Zihao Zheng, Zhihao Mao, Xingyue Zhou +9
School of Computer Science, Peking University, Beijing, China. · School of Computer Science, China University of Geosciences (Wuhan), Wuhan, China. · School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China. +1
Vision-and-Language Navigation (VLN) policies increasingly benefit from strong semantic priors provided by large vision-language models (VLMs). However, standard action supervision does not explicitly encourage intermediate representations to preserve scene geometry, relative orientation, or global episode progress. Incorporating depth estimators, explicit maps, point clouds, or geometry foundation models at inference can provide such structure but introduces additional computation, memory overhead, and architectural dependence during deployment. We introduce StageVLN, a training framework that shapes navigation representations through privileged spatial and trajectory guidance while preserving the original inference pathway. A frozen geometry foundation model provides multi-level spatial guidance to hierarchical navigator states, while relative-heading and expert-route progress objectives provide complementary trajectory-state supervision. All auxiliary components are used only during training and removed at deployment. On R2R-CE validation-unseen, StageVLN achieves 56.3% SR and 51.4% SPL with a 4B-parameter backbone, without an additional geometry encoder at inference. On RxR-CE, it achieves 54.3% SR without additional navigation training data or a geometry encoder at inference.
Anh Dao, Quan-Dung Pham, Le Danh Vinh +6
VinMotion, Inc., Vietnam · University of Southern California, USA