Representative vision-language navigation (VLN) methods typically couple a vision-language model (VLM) brain with a learnable action head and fine-tune the VLM brain on navigation-domain data to predict geometric goals. However, such adaptation can substantially alter base VLM weights and degrade general visual capabilities. We introduce ZeroVLN, a slow-fast navigation framework that achieves effective navigation with zero navigation-domain SFT of the VLM brain. Rather than teaching the brain to predict geometric goals, we equip it with a semantic tool-calling interface for conveying movement decisions to an action expert in natural language. We first adapt the action expert to carry out these movement requests, then jointly improve the brain's movement decisions and the action expert's continuous execution through closed-loop reinforcement learning (RL). Trained on the same navigation episodes, ZeroVLN substantially outperforms representative SFT-based hierarchical VLN methods on standard VLN-CE (continuous environments) benchmarks, while inducing only minor changes to the brain's weights and largely preserving its general capabilities. To our knowledge, ZeroVLN is the first slow-fast VLN framework with zero navigation-domain SFT of the VLM brain, and the first to jointly optimize its slow and fast policies through closed-loop agentic RL in VLN-CE.
Figures & tables
Figure 1: General Capability after Navigation Adaptation. Compared with the base Qwen2.5-VL-7B model, DualVLN loses substantial accuracy on most visual and spatial benchmarks, motivating navigation learning without brain SFT.
Figure 2: Framework of ZeroVLN. The brain delegates semantic requests to an action expert. Request-grounded supervision first adapts the expert; joint agentic RL then improves both policies without navigation-domain SFT of the brain.
Figure 3: Action Expert Architecture. Semantic queries ( Nq ) and visual tokens ( Nv ) condition action generation. Nf flow steps predict an H -step sequence; only its h -step prefix is executed before visual replanning.
Figure 4: Request Grounding And Reference Trajectory Generation. The trajectory planner maps the original movement request to a scene instance and reachable endpoint. Red curves mark grounding correspondences. Semantic maps and mesh geometry are used only during training.
Figure 5
Figure 5: Reference Trajectory Lengths. Across 1,000 sampled anchors per dataset, reference trajectories span multiple distance scales. The concentration of short trajectories partly reflects the 0.1 m collection threshold.
Method
Training Episodes
Brain SFT
SAGE-3D
R2R-CE
RxR-CE
SR ↑
SPL ↑
NE ↓
O SR ↑
SR ↑
SPL ↑
NE ↓
O SR ↑
SR ↑
SPL ↑
Controlled Comparisons
Direct Tool Use
0
×
39.5
24.5
8.69
35.8
19.0
10.2
10.2
35.8
21.8
13.5
DualVLN Wei et al. (2025a)
34.4K
✓
21.2
18.0
8.82
39.0
15.4
11.2
10.8
41.1
17.7
9.99
NaVILA Cheng et al. (2024)
34.4K
✓
23.5
22.1
6.06
51.8
46.5
41.8
6.65
55.6
48.7
43.0
ZeroVLN (w/o RL)
34.4K
×
36.9
20.3
9.51
42.4
17.0
8.40
11.4
44.1
18.7
9.61
Table 1: Navigation Results. With matched backbones and training episodes, ZeroVLN achieves the highest SR across all three benchmarks without brain SFT. Higher OSR but lower SPL than NaVILA on R2R/RxR reveals a reachability–efficiency trade-off.
Figure 6: Navigation Training Mechanisms. SFT fits the brain to prescribed navigation outputs. ZeroVLN instead reinforces its sampled semantic requests through navigation returns, building on the brain’s existing capabilities.
Figure 7: Small Brain Updates and Retained Capabilities. ZeroVLN largely preserves general visual capabilities with substantially smaller VLM weight changes than NaVILA and DualVLN.
Figure 8: General Capabilities beyond Answer Formatting. We read the probabilities of answer option tokens directly from the model’s next-token vocabulary distribution, bypassing formatted answer generation. ZeroVLN largely retains base performance, whereas NaVILA and DualVLN perform close to chance on every benchmark.
Figure 9: Execution-Prefix versus Full-Sequence Supervision. With identical reference trajectories and a frozen brain, supervising the executed prefix yields stronger navigation performance and provides a useful initialization for joint RL.
Figure 10: Recovery from Blocked Motion in R2R. The brain changes its view and revises blocked climbing requests, reaches the second landing, and stops at round 25, illustrating feedback-driven coordination with the action expert.
Figure 11: Round-Level versus Trajectory-Level RLOO. Under matched rewards, trajectory-level credit improves SR and SPL, lowers NE, and narrows the OSR–SR gap, better connecting goal reaching with successful completion.
Figure 12: Joint Optimization and Reward Ablations. Joint RL sustains improvements beyond updating either policy alone. Removing arrival, terminal-success, or process rewards weakens goal reaching, task completion, or training stability.
Figure 13: Joint Optimization and Rewards under Round-Level Credit. We obtain conclusions consistent with those under trajectory-level credit (Figure 12 ).
Figure 14: Consistency between predicted trajectories and planner goals before and after joint RL, with additional details provided in Appendix E.2 .
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Setting Value
Semantic Interface
Input Natural-Language Movement Request Contextual Queries 16 Queries Of Dimension 2,048
Table 2: ZeroVLN Action Expert Architecture. We combine semantic queries with visual features to condition a flow matching decoder, then execute the first four of its 16 predicted action steps before visual replanning.
Table 4: Training Data Collected With The Trajectory Planner. Retention is the percentage of collected anchors passing the length and execution-validity filters. Each retained anchor contains four execution cycles.
Training Data
Pairs
Anchors
Samples
Original Data
—
124,458
465,973
Direction Augmentation
49,647
297,882
297,882
Object/Relation Augmentation
19,114
38,228
134,930
Appendix
Table 5: Data for Scene-Conditioned Augmentation. Pairs are counterfactual pairs; samples are execution cycles. Direction augmentation uses two directions and three templates per pair. Object/relation augmentation retains two branches and up to four cycles per pair.
Request Source
w/o Augmentation (%)
w/ Direction Augmentation (%)
Training Template
0.00
95.90
Held-Out Template
0.00
96.27
Brain-Generated Requests
64.70
56.91
Appendix
Table 6: Transfer from Constructed to Brain-Generated Requests. Paired success requires both left/right requests to satisfy their respective criteria. The tests use 67 constructed and 199 brain-generated pairs, with four draws per pair.
Figure 15: Request Execution and Navigation Transfer. (a,b) Constructed-request accuracy does not ensure navigation gains. (c,d) Request-grounded supervision provides stronger initialization than fixed-route anchoring under the reported pipeline settings. Stars mark RL initialization.
Benchmark
Base Model
ZeroVLN
DualVLN
NaVILA
CV-Bench-3D
93.17
93.58
0.00
0.00
CV-Bench-2D
82.10
82.14
0.00
0.00
DA-2K
69.78
70.60
0.00
0.00
ERQA
51.75
52.50
0.00
0.00
EmbSpatialBench
79.20
79.42
0.00
0.00
RoboSpatialHome
41.71
43.43
0.00
0.00
Appendix
Table 7: General Capabilities Before And After Navigation Adaptation. All methods share the Qwen3.6-35B-A3B base model. Scores are percentages. The mean weights all twelve benchmarks equally. 3DSRBench uses the vanilla setting.
Foshan Graduate School of Innovation, Northeastern University, Foshan 528311, China · Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China · QingniaoAI, China +1