Representative vision-language navigation (VLN) methods typically couple a vision-language model (VLM) brain with a learnable action head and fine-tune the VLM brain on navigation-domain data to predict geometric goals. However, such adaptation can substantially alter base VLM weights and degrade general visual capabilities. We introduce ZeroVLN, a slow-fast navigation framework that achieves effective navigation with zero navigation-domain SFT of the VLM brain. Rather than teaching the brain to predict geometric goals, we equip it with a semantic tool-calling interface for conveying movement decisions to an action expert in natural language. We first adapt the action expert to carry out these movement requests, then jointly improve the brain's movement decisions and the action expert's continuous execution through closed-loop reinforcement learning (RL). Trained on the same navigation episodes, ZeroVLN substantially outperforms representative SFT-based hierarchical VLN methods on standard VLN-CE (continuous environments) benchmarks, while inducing only minor changes to the brain's weights and largely preserving its general capabilities. To our knowledge, ZeroVLN is the first slow-fast VLN framework with zero navigation-domain SFT of the VLM brain, and the first to jointly optimize its slow and fast policies through closed-loop agentic RL in VLN-CE.
Figures & tables
Figure 1: General Capability after Navigation Adaptation. Compared with the base Qwen2.5-VL-7B model, DualVLN loses substantial accuracy on most visual and spatial benchmarks, motivating navigation learning without brain SFT.
Figure 2: Framework of ZeroVLN. The brain delegates semantic requests to an action expert. Request-grounded supervision first adapts the expert; joint agentic RL then improves both policies without navigation-domain SFT of the brain.
Figure 3: Action Expert Architecture. Semantic queries ( Nq ) and visual tokens ( Nv ) condition action generation. Nf flow steps predict an H -step sequence; only its h -step prefix is executed before visual replanning.
Figure 4: Request Grounding And Reference Trajectory Generation. The trajectory planner maps the original movement request to a scene instance and reachable endpoint. Red curves mark grounding correspondences. Semantic maps and mesh geometry are used only during training.
Figure 5
Figure 5: Reference Trajectory Lengths. Across 1,000 sampled anchors per dataset, reference trajectories span multiple distance scales. The concentration of short trajectories partly reflects the 0.1 m collection threshold.
Method
Training Episodes
Brain SFT
SAGE-3D
R2R-CE
RxR-CE
SR ↑
SPL ↑
NE ↓
O SR ↑
SR ↑
SPL ↑
NE ↓
O SR ↑
SR ↑
SPL ↑
Controlled Comparisons
Direct Tool Use
0
×
39.5
24.5
8.69
35.8
19.0
10.2
10.2
35.8
21.8
13.5
DualVLN Wei et al. (2025a)
34.4K
✓
21.2
18.0
8.82
39.0
15.4
11.2
10.8
41.1
17.7
9.99
NaVILA Cheng et al. (2024)
34.4K
✓
23.5
22.1
6.06
51.8
46.5
41.8
6.65
55.6
48.7
43.0
ZeroVLN (w/o RL)
34.4K
×
36.9
20.3
9.51
42.4
17.0
8.40
11.4
44.1
18.7
9.61
Table 1: Navigation Results. With matched backbones and training episodes, ZeroVLN achieves the highest SR across all three benchmarks without brain SFT. Higher OSR but lower SPL than NaVILA on R2R/RxR reveals a reachability–efficiency trade-off.
Figure 6: Navigation Training Mechanisms. SFT fits the brain to prescribed navigation outputs. ZeroVLN instead reinforces its sampled semantic requests through navigation returns, building on the brain’s existing capabilities.
Figure 7: Small Brain Updates and Retained Capabilities. ZeroVLN largely preserves general visual capabilities with substantially smaller VLM weight changes than NaVILA and DualVLN.
Figure 8: General Capabilities beyond Answer Formatting. We read the probabilities of answer option tokens directly from the model’s next-token vocabulary distribution, bypassing formatted answer generation. ZeroVLN largely retains base performance, whereas NaVILA and DualVLN perform close to chance on every benchmark.
Figure 9: Execution-Prefix versus Full-Sequence Supervision. With identical reference trajectories and a frozen brain, supervising the executed prefix yields stronger navigation performance and provides a useful initialization for joint RL.
Figure 10: Recovery from Blocked Motion in R2R. The brain changes its view and revises blocked climbing requests, reaches the second landing, and stops at round 25, illustrating feedback-driven coordination with the action expert.
Figure 11: Round-Level versus Trajectory-Level RLOO. Under matched rewards, trajectory-level credit improves SR and SPL, lowers NE, and narrows the OSR–SR gap, better connecting goal reaching with successful completion.
Figure 12: Joint Optimization and Reward Ablations. Joint RL sustains improvements beyond updating either policy alone. Removing arrival, terminal-success, or process rewards weakens goal reaching, task completion, or training stability.
Figure 13: Joint Optimization and Rewards under Round-Level Credit. We obtain conclusions consistent with those under trajectory-level credit (Figure 12 ).
Figure 14: Consistency between predicted trajectories and planner goals before and after joint RL, with additional details provided in Appendix E.2 .
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Setting Value
Semantic Interface
Input Natural-Language Movement Request Contextual Queries 16 Queries Of Dimension 2,048
Table 2: ZeroVLN Action Expert Architecture. We combine semantic queries with visual features to condition a flow matching decoder, then execute the first four of its 16 predicted action steps before visual replanning.
Table 4: Training Data Collected With The Trajectory Planner. Retention is the percentage of collected anchors passing the length and execution-validity filters. Each retained anchor contains four execution cycles.
Training Data
Pairs
Anchors
Samples
Original Data
—
124,458
465,973
Direction Augmentation
49,647
297,882
297,882
Object/Relation Augmentation
19,114
38,228
134,930
Appendix
Table 5: Data for Scene-Conditioned Augmentation. Pairs are counterfactual pairs; samples are execution cycles. Direction augmentation uses two directions and three templates per pair. Object/relation augmentation retains two branches and up to four cycles per pair.
Request Source
w/o Augmentation (%)
w/ Direction Augmentation (%)
Training Template
0.00
95.90
Held-Out Template
0.00
96.27
Brain-Generated Requests
64.70
56.91
Appendix
Table 6: Transfer from Constructed to Brain-Generated Requests. Paired success requires both left/right requests to satisfy their respective criteria. The tests use 67 constructed and 199 brain-generated pairs, with four draws per pair.
Figure 15: Request Execution and Navigation Transfer. (a,b) Constructed-request accuracy does not ensure navigation gains. (c,d) Request-grounded supervision provides stronger initialization than fixed-route anchoring under the reported pipeline settings. Stars mark RL initialization.
Benchmark
Base Model
ZeroVLN
DualVLN
NaVILA
CV-Bench-3D
93.17
93.58
0.00
0.00
CV-Bench-2D
82.10
82.14
0.00
0.00
DA-2K
69.78
70.60
0.00
0.00
ERQA
51.75
52.50
0.00
0.00
EmbSpatialBench
79.20
79.42
0.00
0.00
RoboSpatialHome
41.71
43.43
0.00
0.00
Appendix
Table 7: General Capabilities Before And After Navigation Adaptation. All methods share the Qwen3.6-35B-A3B base model. Scores are percentages. The mean weights all twelve benchmarks equally. 3DSRBench uses the vanilla setting.
Although vision-language navigation (VLN) has progressed rapidly, zero-shot VLN in continuous environments (VLN-CE) remains highly challenging when using lightweight vision-language models (VLMs), whose limited reasoning capacity makes long-horizon navigation unreliable. In this paper, we propose LightZeroNav to tackle the three major bottlenecks when using lightweight VLMs in zero-shot VLN-CE,i.e.,information redundancy from multi-source inputs, inaccurate progress estimation caused by noisy textual memory, and task entanglement between action execution and stage transition. Using only RGB observations and a lightweight open-source Qwen3-VL-8B backbone, LightZeroNav achieves competitive performance with GPT-4o (~200B) without task-specific training, graph search, or waypoint predictors, demonstrating its effectiveness in zero-shot VLN-CE.
Kun Luo, Xiangyu Dong, Xiaoguang Ma +2
Foshan Graduate School of Innovation, Northeastern University, Foshan 528311, China · Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China · QingniaoAI, China +1
Vision-Language Navigation (VLN) approaches have currently followed two primary paradigms: the end-to-end Vision-Language Model (VLM) policy fine-tuned on navigation trajectories to directly predict actions, and the zero-shot modular pipeline integrating pre-trained Multimodal Large Language Model (MLLM) for training-free generalization to unseen environments. However, end-to-end methods struggle with long-horizon navigation and lack dynamic reasoning ability, whereas zero-shot methods are constrained by limited spatial grounding for reliable planning and require substantial reasoning time. To bridge this gap, we introduce SEDualVLN, a spatially-enhanced dual-system VLN framework. System 1 is a VLM model enhanced with both global and local spatial awareness, tailored for high-frequency action generation. System 2 integrates a general MLLM with a mapping module, wherein the MLLM performs low-frequency waypoint planning by leveraging top-down views of the real-time 3D map alongside streams of rendered path images. Both systems exploit complementary spatial representations to improve the agent's spatial understanding in VLN tasks. Ultimately, they cooperate to complete the navigation task through a fast-slow coordinated approach. SEDualVLN achieves state-of-the-art performance on VLN-CE benchmarks, and further ablation studies demonstrate the effectiveness of each system and module.
Jingzhi Huang, Junkai Huang, Wenxuan Song +5
Hong Kong Polytechnic University · Institute of Automation, Chinese Academy of Sciences · Hong Kong University of Science and Technology (Guangzhou)
Vision-and-Language Navigation in continuous environments (VLN-CE) requires an agent to ground language in egocentric observations and plan in unseen scenes. Although recent multimodal large models and world-model-based methods have improved navigation, they often preserve excessive task-irrelevant detail, weakening generalization and increasing computational burden. We propose BrainNav, a navigation framework grounded in the Principle of Minimal Sufficiency. BrainNav consists of three components: a Logical Anchor Model that implements instruction-aware selective perception to suppress environmental noise, a Minimalist Constraint Alignment module that serves as a compact cross-modal bottleneck, efficiently synchronizing discrete linguistic intent with continuous latent dynamics while filtering out redundant information, and a Compression World Model that predicts action-conditioned states within a condensed, low-rank latent space. These modules align semantic intent with spatial perception, enhancing the agent's robustness and efficiency in complex tasks. Experiments show that BrainNav improves over prior SOTA by 2.0 % / 1.0 in SR/SPL on R2R-CE val-unseen and 0.94 % / 0.78 on RxR-CE val-unseen. These results indicate that minimally sufficient world representations provide an effective foundation for robust VLN.
Yihao Wu, Chenyi Xu, Liqi Yan +6
1Hangzhou Dianzi University · University of Zurich · University of Exeter