cs.AINov 19, 2026

ZeroVLN: Learning to Navigate without Supervised Fine-Tuning of the VLM Brain

Authors: Zixuan Huang, Yang Zhou, Guli Zhang, Sunzhu Li, Hongyan Xie, Zehao Chen, Fuzhen Zhuang, Kaixuan Wang, 

Organizations: Beihang University · Zhuoyu Technology, Shenzhen, China · Zhejiang University

Abstract

Representative vision-language navigation (VLN) methods typically couple a vision-language model (VLM) brain with a learnable action head and fine-tune the VLM brain on navigation-domain data to predict geometric goals. However, such adaptation can substantially alter base VLM weights and degrade general visual capabilities. We introduce ZeroVLN, a slow-fast navigation framework that achieves effective navigation with zero navigation-domain SFT of the VLM brain. Rather than teaching the brain to predict geometric goals, we equip it with a semantic tool-calling interface for conveying movement decisions to an action expert in natural language. We first adapt the action expert to carry out these movement requests, then jointly improve the brain's movement decisions and the action expert's continuous execution through closed-loop reinforcement learning (RL). Trained on the same navigation episodes, ZeroVLN substantially outperforms representative SFT-based hierarchical VLN methods on standard VLN-CE (continuous environments) benchmarks, while inducing only minor changes to the brain's weights and largely preserving its general capabilities. To our knowledge, ZeroVLN is the first slow-fast VLN framework with zero navigation-domain SFT of the VLM brain, and the first to jointly optimize its slow and fast policies through closed-loop agentic RL in VLN-CE.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. LightZeroNav: Zero-Shot Vision Language Navigation in Continuous Environments Based on Lightweight VLMs

    Mar 16, 2026Kun Luo, Xiangyu Dong, Xiaoguang Ma +2Vision-Language NavigationVision-Language Models

  2. SEDualVLN: A Spatially-Enhanced Dual-System Framework for Vision-Language Navigation

    May 17, 2026Jingzhi Huang, Junkai Huang, Wenxuan Song +5Visual Spatial ReasoningVision-Language Navigation

  3. Towards Dual-Brain Minimal Sufficient Representation for Vision-Language Navigation

    Jul 25, 2026Yihao Wu, Chenyi Xu, Liqi Yan +6Representation LearningInformation Bottleneck