Vision-and-Language Navigation (VLN) has recently benefited from Multimodal Large Language Models (MLLMs), enabling zero-shot navigation. While recent exploration-based zero-shot methods have shown promising results by leveraging global scene priors, they rely on high-quality human-crafted scene reconstructions, which are impractical for real-world robot deployment. When encountering an unseen environment, a robot should build its own priors through pre-exploration. However, these self-built reconstructions are inevitably incomplete and noisy, which severely degrade methods that depend on high-quality scene reconstructions. To address these issues, we propose SpatialAnt, a zero-shot navigation framework designed to bridge the gap between imperfect self-reconstructions and robust execution. SpatialAnt introduces a physical grounding strategy to recover the absolute metric scale for monocular-based reconstructions. Furthermore, rather than treating the noisy self-reconstructed scenes as absolute spatial references, we propose a novel visual anticipation mechanism. This mechanism leverages the noisy point clouds to render future observations, enabling the agent to perform counterfactual reasoning and prune paths that contradict human instructions. Extensive experiments in both simulated and real-world environments demonstrate that SpatialAnt significantly outperforms existing zero-shot methods. We achieve a 66% Success Rate (SR) on R2R-CE and 50.8% SR on RxR-CE benchmarks. Physical deployment on a Hello Robot further confirms the efficiency and efficacy of our framework, achieving a 52% SR in challenging real-world settings.
Figures & tables
Figure 1: The reality gap for pre-exploration based zero-shot VLN agents. Previous works adopt the ideal scene reconstructions provided by existing datasets ( Chang et al., 2017 ) , treating these global priors as accurate maps, and are sensitive to noise. Instead, we use the real robot reconstructed noisy scenes and propose a visual anticipation approach to transform the noisy scene into supportable sub-paths.
Figure 2: Overview of our proposed SpatialAnt framework. We firstly let the agent to pre-explore the environment, gathering the coordinates of navigable places. Then, we use these information to plan visibility-aware tours to collect geometrically coherent RGB videos and IMU traces for subsequent regional point cloud reconstruction. The reconstructed point clouds are then aligned with physical scale and merged into a global point cloud. Afterall, this global point cloud provides extra global context by a novel visual anticipation mechanism for MLLM to conduct sub-path based high-level reasoning.
#
Methods
Pre-Exp
R2R-CE
RxR-CE
NE ( ↓ )
OSR ( ↑ )
SR ( ↑ )
SPL ( ↑ )
nDTW ( ↑ )
NE ( ↓ )
SR ( ↑ )
SPL ( ↑ )
nDTW ( ↑ )
Supervised Learning:
1
NavFoM ( Zhang et al., 2025a )
–
4.61
72.1
61.7
55.3
–
4.74
64.4
56.2
65.8
2
Efficient-VLN ( Zheng et al., 2025 )
–
4.18
73.7
64.2
55.9
–
3.88
67.0
54.3
68.4
Zero-Shot:
3
Open-Nav ( Qiao et al., 2025 )
×
6.70
23.0
19.0
16.1
45.8
–
–
–
–
Table 1: Comparison in continuous environments on R2R-CE and RxR-CE Val-Unseen splits. The best supervised results are highlighted in bold , while the best zero-shot results are underlined . “Pre-Exp” denotes whether the zero-shot agent adopts the pre-exploration based navigation settings.
Methods
#Backtrack
#Steps(↓)
TL(↓)
SPL(↑)
SpatialNav ( Zhang et al., 2026 )
8 / 51 (15.7%)
8.60
10.77
77.9
SpatialAnt (Ours)
8 / 66 ( 12.1% )
8.26
10.48
84.9
Table 2: Comparison of backtrack times on the successful tasks. We report the results by agent-reconstructed scene point clouds on sampled val-unseen splits of R2R-CE.
Methods
Scene
R2R-CE
RxR-CE
OSR(↑)
SR(↑)
SPL(↑)
OSR(↑)
SR(↑)
SPL(↑)
SpatialNav ( Zhang et al., 2026 )
H
66.0
64.0
51.1
47.5
32.4
24.1
A
61.0
51.0
39.8
40.3
26.1
19.3
Performance gap
-5.0
-13.0
-11.3
-7.2
-6.3
-4.8
SpatialAnt
H
80.0
69.0
52.7
69.2
53.9
37.6
A
76.0
66.0
54.4
63.1
50.8
35.6
Table 3: Comparison between the human-curated (H) and agent-reconstructed (A) scene point clouds.
Methods
HFOV
NE(↓)
OSR(↑)
SR(↑)
SPL(↑)
nDTW(↑)
SpatialAnt (w/o VA)
-
5.19
77.0
55.0
41.4
70.0
SpatialAnt (w/ VA)
60 ∘
4.33
72.0
65.0
55.2
69.2
90 ∘
4.42
76.0
66.0
54.4
69.5
120 ∘
4.85
74.0
59.0
48.7
67.7
Table 4: Ablation study on visual anticipation mechanism (VA) and its horizon field of view (HFOV). We report the results on sampled val-unseen splits of R2R-CE.
Figure 3: Comparison across different MLLM backbones. We report the results on sampled subset of R2R-CE. The default HFOV of visual anticipation (VA) is set as 90 ∘ .
Method
LP Time (s)
GP Time (s)
SR (↑ , %)
NE ( ↓ , m)
Supervised Learning:
VLN-BERT ( Hong et al., 2021 )
22.40
-
20.0
3.92
Zero-Shot:
SmartWay ( Shi et al., 2025 )
22.40
-
32.0
3.01
SpatialAnt (Ours)
22.40
0.57
52.0
2.49
Table 5: Real-world navigation performance. We compare SpatialAnt against representative supervised-learning (VLN-BERT) and zero-shot (SmartWay) baselines.
Figure 4: Representative cases in real environments. We show frames of the agent egocentric front view, panoramic observations (from right to left), and candidate actions on the spatial map together with our anticipated views.
Vision-Language Navigation in Continuous Environments (VLN-CE) requires an embodied agent to execute long-horizon instructions in unknown environments. Existing zero-shot VLN-CE systems typically maintain spatial states through geometric localization or coordinate-based representations. Recent geometry-constrained navigation removes depth and globally consistent coordinates, but maintaining persistent spatial awareness for place confirmation, progress verification, and recovery remains challenging. We present Navi-Agent, a zero-shot VLN-CE agent that constructs a coordinate-free spatial state from visual observations and executed motion histories. Navi-Agent organizes this state as a navigation topology, where nodes represent visual places and edges represent motion transitions. This representation enables observation-based approximate self-localization, task progress verification, and visual revisitation-based recovery. Navi-Agent performs closed-loop navigation by decomposing instructions into sub-goals, executing local visual navigation, and verifying visited places through the constructed spatial state. Experiments on zero-shot VLN-CE benchmark and real-world robot platforms show that Navi-Agent achieves state-of-the-art performance among geometry-constrained methods while remaining competitive with approaches relying on geometric localization.
Wenyuan Xie, Mengyang Hong, Yongzhong Wang +9
Shanghai Jiao Tong University · Southern University of Science and Technology · Xinjiang University of Finance and Economics +1
Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires an embodied agent to navigate unseen environments by following natural language instructions. Current zero-shot VLN-CE methods either rely on pre-trained waypoint predictors or require multiple queries to large models per step. To address prohibitive inference latency and computational overhead, we propose O2C-Nav, an efficient zero-shot navigation framework that calls only a single large model once per decision step. Our approach introduces a training-free structured waypoint generator and a novel abstract representation that projects sparse, history-aware candidate waypoints directly onto RGB images as visual markers. The MLLM selects a waypoint or generates a fallback target bounding box at each step, while a low-level Fast Marching Method (FMM) planner converts the selected target into an executable collision-free path. This paradigm provides the model with concrete spatial perception and explicit memory while significantly reducing the visual processing load. Extensive evaluations on the R2R-CE and RxR-CE benchmarks demonstrate that O2C-Nav outperforms current state-of-the-art zero-shot methods, highlighting its great potential for real-time robotic deployment. Code is available at https://github.com/kkpsq/O2C-Nav-Code.
Shiqi Pan, Qi Zheng, Hanqin Sun +3
College of Electronics and Information Engineering, Shenzhen University, Shenzhen 518060, China · School of Computer Science, The University of Sydney, Darlington 2008, Australia · College of Computer Science and Software Engineering, Shenzhen University, Shenzhen 518060, China
Vision-and-language navigation (VLN) requires an embodied agent to ground natural-language instructions into executable navigation actions in unseen environments. Existing zero-shot methods typically rely on additional waypoint prediction modules, which often entangle high-level directional reasoning with fine-grained local grounding, leading to error-prone and unstable decisions. In this paper, we propose P2DNav, a hierarchical framework for zero-shot vision-and-language navigation. P2DNav consists of three core components: Panorama-to-Downview (P2D), Sliding-Window Dialogue Memory (SDM), and Reflective Reorientation Mechanism (RRM). P2D explicitly decomposes navigation decision-making into two stages: panoramic direction selection and downview local grounding. It first selects the instruction-relevant direction from a 360° panorama, and then predicts a pixel-level target point from the downview RGB observation in that direction. In addition, SDM organizes navigation history as a multi-turn dialogue context and maintains recent visual observations within a sliding window to support long-horizon navigation. RRM further enables reflective reorientation by assessing the reliability of local grounding based on the downview observation and returning to panoramic direction selection when necessary. Experiments on the R2R-CE benchmark show that P2DNav achieves strong performance among zero-shot methods. In particular, compared with the state-of-the-art (SOTA) zero-shot waypoint-based and waypoint-free methods, P2DNav achieves SR gains of 146.6% and 58.9%, respectively, demonstrating the effectiveness of P2D, SDM, and RRM for zero-shot VLN. Code will be released for public use.
Kai Sheng, Liuyi Wang, Haojie Dai +5
Department of Control Science and Engineering, Tongji University, Shanghai 210804, China