DyGeoVLN: Infusing Dynamic Geometry Foundation Model into Vision-Language Navigation

Xiangchen Liu; Hanghan Zheng; Jeil Jeong; Minsung Yoon; Lin Zhao; Zhide Zhong; Haoang Li; Sung-Eui Yoon

cs.ROMar 22, 2026

DyGeoVLN: Infusing Dynamic Geometry Foundation Model into Vision-Language Navigation

Authors: Xiangchen Liu, Hanghan Zheng, Jeil Jeong, Minsung Yoon, Lin Zhao, Zhide Zhong, Haoang Li, Sung-Eui Yoon

Organizations: KAIST · HKUST(GZ) · JD Explore Academy

Abstract

Vision-language Navigation (VLN) requires an agent to understand visual observations and language instructions to navigate in unseen environments. Most existing approaches rely on static scene assumptions and struggle to generalize in dynamic, real-world scenarios. To address this challenge, we propose DyGeoVLN, a dynamic geometry-aware VLN framework. Our method infuses a dynamic geometry foundation model into the VLN framework through cross-branch feature fusion to enable explicit 3D spatial representation and visual-semantic reasoning. To efficiently compress historical token information in long-horizon, dynamic navigation, we further introduce a novel pose-free and adaptive-resolution token-pruning strategy. This strategy can remove spatio-temporal redundant tokens to reduce inference cost. Extensive experiments demonstrate that our approach achieves state-of-the-art performance on multiple benchmarks and exhibits strong robustness in real-world environments.

Figures & tables

Figure 1: The system overview of our DyGeoVLN. Given instruction and sequential images, the vision encoder and our proposed dynamic geometry foundation model (DGFM) respectively produce visual tokens and geometry tokens in a stream manner. Then these tokens are fused via cross-branch feature fusion to obtain spatial-semantic fused tokens. Finally, the LLM integrates the fused tokens and instruction tokens, followed by de-tokenizing the predicted sequence directly into robot actions. To efficiently compress the historical token, we build a spatial token pruning strategy on top of our DGFM , yielding a compact set of pruned spatial-semantic tokens.

Figure 2: Overview of the proposed DGFM. (a) The architecture of our dynamic geometry foundation model: 3D latent representations and 2D visual semantic features are fused and subsequently aligned along the temporal axis to achieve globally consistent and fine-grained reconstruction. (b) Qualitative reconstruction results compared against Ground Truth, $\pi^{3}$ [ 23 ] , and VGGT [ 21 ] . The red bounding boxes highlight regions where both $\pi^{3}$ and VGGT suffer from poor human reconstruction. In contrast, our method preserves precise human positioning and maintains stable geometric structures.

Figure 3: Example case of our DyHM3D dataset.

Methods	Observation Data		Val Seen				Val Unseen
Methods	Complex Data	Simple Data	NE $\downarrow$	TCR $\downarrow$	CR $\downarrow$	SR $\uparrow$	NE $\downarrow$	TCR $\downarrow$	CR $\downarrow$	SR $\uparrow$
VLN-CMA [ 11 ]	✓		7.63	63.09	0.75	0.04	7.34	47.06	0.77	0.07
HA-VLN-VL [ 7 ]	✓		5.02	4.44	0.52	0.20	5.25	6.63	0.59	0.14
BEVBert ∗ [ 1 ]	✓		5.53	3.64	0.46	0.27	5.51	4.71	0.55	0.21
ETPNav ∗ [ 2 ]	✓		5.17	4.07	0.43	0.24	5.43	6.94	0.58	0.17
g3D-LF ∗ [ 24 ]	✓		5.12	3.58	0.41	0.32	5.30	4.54	0.49	0.27

Table 1: Comparison with state-of-the-art methods on dynamic HA-VLN [ 7 ] benchmark.

*

indicates methods using the pretrained waypoint predictor from [ 9 ] . Complex Data indicates the method simultaneously uses panorama RGB-D images and/or odometry. Simple Data indicates the method only uses Monocular RGB images.

Figure 4: Qualitative comparison between state-of-the-art method StreamVLN [ 30 ] and our DyGeoVLN on the dynamic HA-VLN [ 7 ] benchmark. StreamVLN loses track of the path and fails to handle dynamic human context, while DyGeoVLN better understands human behaviors and accurately reaches the target destination.

Methods	Observation Data		R2R-CE Val-Unseen
Methods	Complex Data	Simple Data	NE $\downarrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$
GridMM ∗ [ 26 ]	✓		5.11	61.0	49.0	41.0
ETPNav ∗ [ 2 ]	✓		4.71	65.0	57.0	49.0
ScaleVLN ∗ [ 28 ]	✓		4.80	64.0	55.0	51.0
g3D-LF ∗ [ 24 ]	✓		4.53	68.0	61.0	52.0
CM2 [ 8 ]	✓		7.02	41.5	34.3	27.6

Table 2: Comparison with state-of-the-art methods on R2R-CE [ 11 ] dataset.

*

indicates methods using the waypoint predictor from [ 9 ] . Complex Data indicates the method simultaneously uses panorama RGB-D images and/or odometry. Simple Data indicates the method only uses Monocular RGB images.

Figure 5: Qualitative comparison between state-of-the-art method StreamVLN [ 30 ] and our DyGeoVLN on the R2R-CE [ 11 ] benchmark. DyGeoVLN demonstrates improved trajectory consistency, whereas StreamVLN stops midway and fails to reach the destination.

Figure 6: Quantitative comparison of success rates (%) across varying real-world scenarios. Our proposed DyGeoVLN consistently outperforms other baseline models, demonstrating particular robustness across diverse environments.

Figure 7: Qualitative results of our DyGeoVLN model. The robot successfully follows instructions to reach the goal while avoiding collisions with moving pedestrians.

Methods	NE $\downarrow$	TCR $\downarrow$	CR $\downarrow$	SR $\uparrow$
DyGeoVLN (Full)	5.12	3.69	0.38	0.40
w/o Visual Semantic	6.82	4.96	0.51	0.30
w/o Spatial Geometry	5.54	4.03	0.43	0.34
w/o Dynamic Spatial Injection	5.37	4.10	0.43	0.36
w/o Spatial Token Pruning	5.33	3.94	0.39	0.37

Table 3: Ablation Study of each component in our proposed DyGeoVLN.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Example cases of our DyHM3D dataset.

Figure 9: Qualitative comparison of reconstruction performance on dynamic scenes.

Panel of Figure 10 — Figure 10: Qualitative results of our DyGeoVLN model on dynamic HA-VLN [ 7 ] benchmark. (Cases 1-2)

Panel of Figure 11 — Figure 11: Qualitative results of our DyGeoVLN model on dynamic HA-VLN [ 7 ] benchmark. (Cases 3-5 continued).

Panel of Figure 12 — Figure 12: Qualitative results of our DyGeoVLN model on R2R-CE [ 11 ] benchmark.

Figure 13: Setup of real-world experiments: (a) Hardware configuration of the quadruped robot; (b) Schematic of the real-world deployment pipeline.

Figure 14: Qualitative results of our DyGeoVLN model on real-world experiments: including dynamic and static scenarios.

Vision-Language Navigation

Vision-Language Navigation studies how embodied agents use natural-language instructions and visual observations to plan and execute routes.

See all 310 papers

SEDualVLN: A Spatially-Enhanced Dual-System Framework for Vision-Language Navigation
May 17, 2026Jingzhi Huang, Junkai Huang, Wenxuan Song +5Visual Spatial Reasoning Vision-Language Navigation
Hong Kong Polytechnic University · Institute of Automation, Chinese Academy of Sciences · Hong Kong University of Science and Technology (Guangzhou)
VLN-Cache: Enabling Token Caching for VLN Models with Visual/Semantic Dynamics Awareness
Mar 7, 2026Zihao Zheng, Zhihao Mao, Xingyue Zhou +9Efficient VLM Inference Vision-Language Navigation
School of Computer Science, Peking University, Beijing, China. · School of Computer Science, China University of Geosciences (Wuhan), Wuhan, China. · School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China. +1
StageVLN: Spatial and Trajectory Auxiliary Guidance for Efficient Vision-Language Navigation
Oct 5, 2026Anh Dao, Quan-Dung Pham, Le Danh Vinh +6Vision-Language Navigation
VinMotion, Inc., Vietnam · University of Southern California, USA

May 17, 2026J/K move · Enter open · S save

SEDualVLN: A Spatially-Enhanced Dual-System Framework for Vision-Language Navigation

Jingzhi Huang, Junkai Huang, Wenxuan Song, Yuhang Peng+4

Hong Kong Polytechnic University · Institute of Automation, Chinese Academy of Sciences · Hong Kong University of Science and Technology (Guangzhou)

Visual Spatial Reasoning Vision-Language Navigation Vision-Language Grounding VLM Reasoning

Open paper AskPDF arXiv

Vision-Language Navigation (VLN) approaches have currently followed two primary paradigms: the end-to-end Vision-Language Model (VLM) policy fine-tuned on navigation trajectories to directly predict actions, and the zero-shot modular pipeline integrating pre-trained Multimodal Large Language Model (MLLM) for training-free generalization to unseen environments. However, end-to-end methods struggle with long-horizon navigation and lack dynamic reasoning ability, whereas zero-shot methods are constrained by limited spatial grounding for reliable planning and require substantial reasoning time. To bridge this gap, we introduce SEDualVLN, a spatially-enhanced dual-system VLN framework. System 1 is a VLM model enhanced with both global and local spatial awareness, tailored for high-frequency action generation. System 2 integrates a general MLLM with a mapping module, wherein the MLLM performs low-frequency waypoint planning by leveraging top-down views of the real-time 3D map alongside streams of rendered path images. Both systems exploit complementary spatial representations to improve the agent's spatial understanding in VLN tasks. Ultimately, they cooperate to complete the navigation task through a fast-slow coordinated approach. SEDualVLN achieves state-of-the-art performance on VLN-CE benchmarks, and further ablation studies demonstrate the effectiveness of each system and module.

Abstract

Vision-Language Navigation

Explore similar work

SEDualVLN: A Spatially-Enhanced Dual-System Framework for Vision-Language Navigation

VLN-Cache: Enabling Token Caching for VLN Models with Visual/Semantic Dynamics Awareness

StageVLN: Spatial and Trajectory Auxiliary Guidance for Efficient Vision-Language Navigation