Can Vision Foundation Models Navigate? Zero-Shot Real-World Evaluation and Lessons Learned

Maeva Guerrier; Karthik Soma; Jana Pavlasek; Giovanni Beltrame

cs.ROMar 26, 2026

Can Vision Foundation Models Navigate? Zero-Shot Real-World Evaluation and Lessons Learned

Authors: Maeva Guerrier, Karthik Soma, Jana Pavlasek, Giovanni Beltrame

Organizations: Polytechnique Montreal

Abstract

Visual Navigation Models (VNMs) promise generalizable, robot navigation by learning from large-scale visual demonstrations. Despite growing real-world deployment, existing evaluations rely almost exclusively on success rate, whether the robot reaches its goal, which conceals trajectory quality, collision behavior, and robustness to environmental change. We present a real-world evaluation of five state-of-the-art VNMs (GNM, ViNT, NoMaD, NaviBridger, and CrossFormer) across two robot platforms and five environments spanning indoor and outdoor settings. Beyond success rate, we combine path-based metrics with vision-based goal-recognition scores and assess robustness through controlled image perturbations (motion blur, sunflare). Our analysis uncovers three systematic limitations: (a) even architecturally sophisticated diffusion and transformer-based models exhibit frequent collisions, indicating limited geometric understanding; (b) models fail to discriminate between different locations that are perceptually similar, however some semantics differences are present, causing goal prediction errors in repetitive environments; and (c) performance degrades under distribution shift. We will publicly release our evaluation codebase and dataset to facilitate reproducible benchmarking of VNMs.

Figures & tables

Fig. 1 : Visual Navigation Models promise to deliver generalizable navigation in novel environments. We provide a comprehensive analysis of 5 models with 2 robot embodiments over 5 real-world environments and propose metrics to better capture their performance.

Fig. 2 : A Visual Navigation Model (VNM) takes as input a sequence of $k$ image observations $O_{t}$ and a goal image $o_{g}$ at timestep $t$ . An image and distance encoder encode the observation sequence and the current ( $o_{t}$ ) and goal observation respectively. The backbone and decoder vary by method. The model outputs action $\hat{A}_{t}$ , which can take the form of a single next waypoint or a trajectory, and can also include the temporal distance $d$ .

Model	Backbone	Specifics	Output
GNM [ 1 ]	Fully Connected	Shared action space	$(\{x_{i}\}_{t+1}^{t+H},d)$
ViNT [ 6 ]	Transformer	Goal Fusion	$(x_{t},d)$
NoMaD [ 7 ]	Diffusion	Goal Masking	$\{x_{i}\}_{t+1}^{t+H}$
NaviBridger [ 10 ]	Diffusion	Learned action prior	$\{x_{i}\}_{t+1}^{t+H}$
CrossFormer [ 11 ]	Transformer	Embodiment specific head	$x_{t}$

TABLE I : The specific architectures of the Visual Navigation Models (VNMs) considered in this paper. Methods differ in their backbone architecture and output type. Some models introduce specific architecture choices within the VNM pipeline.

	Corridor		Office loop
Method	dist.	p.len.	dist.	p.len.
GNM	0.33 $\pm$ 0.13	4.52 $\pm$ 0.16	9.49 $\pm$ 9.43	19.28 $\pm$ 10.03
ViNT	0.40 $\pm$ 0.09	4.81 $\pm$ 0.14	3.53 $\pm$ 2.19	32.51 $\pm$ 2.57
NoMaD	0.56 $\pm$ 0.09	5.25 $\pm$ 0.15	7.22 $\pm$ 5.80	30.04 $\pm$ 9.45
NaviBridger	0.63 $\pm$ 0.16	4.46 $\pm$ 0.63	17.19 $\pm$ 5.29	10.66 $\pm$ 5.61
CrossFormer	1.73 $\pm$ 0.00	2.34 $\pm$ 0.04	-	-

TABLE II: Path length ( p.len ) and Goal Distance ( dist. ) across environments and models (mean

\pm

standard-deviation).

Fig. 3 : Real-world evaluation environments (indoor and outdoor). Blue panels correspond to rover while yellow are quadruped deployments.

Panel of Figure 6 — Fig. 4: Image quality metrics (LPIPS, PSNR, DSSIM) for goal-predicted cases in the Arena (Top) and Snow (Bottom) environments.

	Corridor		Office loop		Arena		Stairs		Snow
Method	n.err.	g.pred.	n.err.	g.pred.	n.err.	g.pred.	n.err.	g.pred.	n.err.	g.pred.
GNM	0.36 $\pm$ 0.49	3/3	3.23 $\pm$ 5.47	5/10	6.94 $\pm$ 1.96	2/3	0.89 $\pm$ 0.55	3/3	1.95 $\pm$ 1.05	3/3
ViNT	0.43 $\pm$ 0.49	3/3	7.34 $\pm$ 13.58	9/10	7.43 $\pm$ 3.05	2/5	0.94 $\pm$ 0.66	3/3	3.82 $\pm$ 5.52	3/3
NoMaD	0.49 $\pm$ 0.70	0/3	8.53 $\pm$ 15.04	8/10	10.59 $\pm$ 7.76	5/5	0.98 $\pm$ 0.60	3/3	3.95 $\pm$ 3.65	3/3
NaviBridger	0.72 $\pm$ 0.44	1/3	2.37 $\pm$ 2.59	1/10	11.04 $\pm$ 6.91	3/5	2.01 $\pm$ 1.29	3/3	2.47 $\pm$ 1.92	3/3
CrossFormer	0.41 $\pm$ 0.49	0/3	-	-	13.21 $\pm$ 6.97	5/5	-	-	-	-

TABLE III : Topological Node Error ( n.err. ) and Predicted Goal Counter ( g.pred. ) across environments and models.

Method	Corridor	Office loop
	col.	col.
GNM	0/3	5/10
ViNT	0/3	1/10
NoMaD	3/3	2/10
NaviBridger	2/3	9/10
CrossFormer	3/3	-

TABLE IV : Collision ( col. ) results for Corridor and Office loop.

Fig. 5 : Corridor deployment results. Green (left): goal reached without collision in all trials. Red (right): at least one collision occurred. See Table IV for details. (Note: overlapping collisions at the same location are shown only once for clarity).

Panel of Figure 10 — Fig. 6 : All Stairs trajectories for GNM , ViNT , NoMaD and NaviBridger (see Table III ) with the reference trajectory .

Panel of Figure 11 — Fig. 7 : All Snow trajectories for GNM , ViNT , NoMaD and NaviBridger (see Table II and III ) with the reference trajectory .

Panel of Figure 12 — Fig. 8 : One Arena trajectory for GNM , ViNT , NoMaD , NaviBridger , and CrossFormer (see Table II and III ) with the reference trajectory .

Panel of Figure 13 — Fig. 9 : Last three Office loop trajectories for GNM , ViNT , NoMaD and NaviBridger (see Table II , III and IV ) with the reference trajectory .

	Precision		Generalization
Model	dist.	p. len.	n. err.	g. pred.
GNM	0.62 $\pm$ 0.32	4.68 $\pm$ 0.35	0.34 $\pm$ 0.53	1/3
ViNT	0.30 $\pm$ 0.35	4.70 $\pm$ 0.34	0.50 $\pm$ 0.67	2/3
GNM	0.40 $\pm$ 0.15	4.15 $\pm$ 0.04	0.44 $\pm$ 0.59	2/3
ViNT	0.21 $\pm$ 0.04	4.33 $\pm$ 0.03	0.39 $\pm$ 0.52	3/3

TABLE V : Navigation metrics of Corridor environment with perturbations: BLUR (gray) vs SUNFLARE (yellow).

Visual Navigation

Visual navigation enables robots and embodied agents to reach goals by interpreting camera images and acting in unfamiliar or changing environments.

See all 70 papers

VISTA: Scale-Aware Visual Navigation via Action History Conditioning
Jun 15, 2026Maeva Guerrier, Koki Kobayashi, Simon Roy +2Vision Foundation Models Robot Navigation
Polytechnique Montreal, MILA · Institute of Science Tokyo · Polytechnique Montreal +2
A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation
Jul 9, 2026Liuyi Wang, Kai Sheng, Zongtao He +8Embodied Navigation Mobile Robotics
College of Electronic and Information Engineering, Tongji University, Shanghai, China
CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigation
Feb 20, 2026Xia Su, Ruiqi Chen, Benlin Liu +4Embodied Spatial Reasoning VLN Benchmarks
University of Washington Seattle, WA, USA · University of California, Santa Cruz Santa Cruz, CA, USA

Jun 15, 2026J/K move · Enter open · S save

VISTA: Scale-Aware Visual Navigation via Action History Conditioning

Maeva Guerrier, Koki Kobayashi, Simon Roy, Jana Pavlasek+1

Polytechnique Montreal, MILA · Institute of Science Tokyo · Polytechnique Montreal+2

Vision Foundation Models Robot Navigation Visual Robot Navigation Robot Policy Generalization

Open paper AskPDF arXiv

Vision Navigation Foundation Models (VNMs) promise end-to-end learned navigation policies capable of zero-shot deployment across diverse embodiments and environments. To maintain generality, many vision-based navigation models predict normalized actions. However, this normalization introduces a critical deployment vulnerability: applying different scaling factors to the same normalized trajectory alters its physical geometry, which degrades navigation performance and increases collision risks. We address this vulnerability by conditioning the model on normalized action histories alongside image observations, providing explicit context on the relationship between the model's predictions and the robot's actual physical displacement. Furthermore, current VNMs often struggle in visually repetitive environments that lack distinct features. To resolve this issue, we integrate a DINOv3 encoder, whose richer representations enable our model to capture both spatial and geometric dimensions between observations. VISTA generalizes robustly to out-of-distribution environments, achieving 100% goal prediction accuracy in zero-shot, real-world deployment in Outdoor, Forest and Office settings, and an average of 95% checkpoints crossed, demonstrating consistent path following in unseen environments.

Abstract

Visual Navigation

Explore similar work

VISTA: Scale-Aware Visual Navigation via Action History Conditioning

A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation

CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigation