Vision-Language Navigation

Also known as VLN

Vision-Language Navigation (VLN) asks embodied agents—such as mobile robots and drones—to follow language instructions by interpreting what they see and choosing actions over time. Approaches range from end-to-end vision-language policies to hierarchical systems that combine large models with maps, scene graphs, spatial memory, geometric planners, and world models; imitation learning and reinforcement learning are common training strategies. Methods are tested on benchmarks such as R2R, RxR, REVERIE, VLN-CE, and HM3D, with growing attention to object goals, outdoor and aerial settings, and real-robot deployment. A central challenge is turning correct language and scene understanding into safe, efficient behavior over long routes, where localization drift, changing scenes, limited compute, and the gap between simulation and reality can cause failures.

Latest papers

313 papers52 papers in the last four weeks, up 16% on the four weeks before. 0.3% of all new papers.

VLN-NF: Feasibility-Aware Vision-and-Language Navigation with False-Premise Instructions
Apr 12, 2026Hung-Ting Su, Ting-Jun Wang, Jia-Fong Yeh +2VLN Benchmarks Vision-Language Navigation
National Taiwan University · DRIC · National Tsing Hua University
WorldMAP: Bootstrapping Vision-Language Navigation Trajectory Prediction with Generative World Models
Apr 9, 2026Hongjin Chen, Shangyun Jiang, Tonghua Su +4Vision-Language Navigation
Harbin Institute of Technology · Zhongguancun Academy · Shandong University +2
Vision-Language Navigation for Aerial Robots: Towards the Era of Large Language Models
Apr 9, 2026Xingyu Xia, Lekai Zhou, Yujie Tang +3Vision-Language Navigation Aerial VLN
Defense Innovation Institute, Chinese Academy of Military Sciences, Beijing 10071, China · Intelligent Game and Decision Laboratory, Beijing 100071, China · Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China
FSUNav: A Cerebrum-Cerebellum Architecture for Fast, Safe, and Universal Zero-Shot Goal-Oriented Navigation
Apr 3, 2026Mingao Tan, Yiyang Li, Shanze Wang +2Open-Vocabulary Object Navigation Robot Navigation
School of Computer Science, Shanghai Jiao Tong University, Shanghai, P. R. China · College of Information Science and Technology, Eastern Institute of Technology, Ningbo, P. R. China
Stop Wandering: Efficient Vision-Language Navigation via Metacognitive Reasoning
Apr 2, 2026Xueying Li, Feng Lyu, Hao Wu +3Vision-Language Navigation Memory-Augmented VLMs
Central South University, Changsha, China · Nanjing University, Nanjing, China · State Grid Hubei Electric Power Research Institute, Wuhan, China +1
AURA: Multimodal Shared Autonomy for Real-World Urban Navigation
Apr 2, 2026Yukai Ma, Honglin He, Selina Song +2Shared Autonomy Robot Navigation
University of California, Los Angeles · Zhejiang University
LatentPilot: Scene-Aware Vision-and-Language Navigation by Dreaming Ahead with Latent Visual Reasoning
Mar 31, 2026Haihong Hao, Lei Chen, Mingfei Han +5Vision-Language Navigation VLMs for Robotics
University of Science and Technology of China · MBZUAI · Stanford University +2
Structured Observation Language for Efficient and Generalizable Vision-Language Navigation
Mar 29, 2026Daojie Peng, Fulong Ma, Jun MaVision-Language Navigation
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology
Beyond Textual Knowledge-Leveraging Multimodal Knowledge Bases for Enhancing Vision-and-Language Navigation
Mar 27, 2026Dongsheng Yang, Yinfeng Yu, Liejun WangVision-Language Navigation Vision-Language Grounding
SpatialAnt: Autonomous Zero-Shot Robot Navigation via Active Scene Reconstruction and Visual Anticipation
Mar 27, 2026Jiwen Zhang, Xiangyu Shi, Siyuan Wang +3Robot Navigation Vision-Language Navigation
Fudan University · Adelaide University · University of Southern California +1
Policy-Guided World Model Planning for Language-Conditioned Visual Navigation
Mar 26, 2026Amirhosein Chahe, Lifeng ZhouEmbodied Navigation Vision-Language Navigation
Drexel University Philadelphia PA 19104, USA
ReMemNav: Memory-Based Decision Correction and Target Verification for Zero-Shot Object Navigation
Mar 25, 2026Feng Wu, Wei Zuo, Wenliang Yang +3Robot Navigation Vision-Language Navigation
Fudan University · Tongji University
Language-Conditioned World Modeling for Visual Navigation
Mar 23, 2026Yifei Dong, Fengyi Wu, Yilong Dai +8World Model Learning Vision-Language Navigation
University of Washington · National University of Singapore · Clemson University +2
DyGeoVLN: Infusing Dynamic Geometry Foundation Model into Vision-Language Navigation
Mar 22, 2026Xiangchen Liu, Hanghan Zheng, Jeil Jeong +5Vision-Language Navigation Efficient VLM Inference
KAIST · HKUST(GZ) · JD Explore Academy
SpatialFly: Implicit 3D Prior-Guided Visual Reparameterization for Continuous UAV Vision-and-Language Navigation
Mar 22, 2026Wen Jiang, Kangyao Huang, Li Wang +9Visual Representation Learning 3D Spatial Reasoning
School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China · Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China · Chongqing Innovation Center, Beijing Institute of Technology, Chongqing 401120, China +5
Does Peer Observation Help? Vision-Sharing Collaboration for Vision-Language Navigation
Mar 21, 2026Qunchao Jin, Yiliao Song, Qi WuVision-Language Navigation Multi-Robot Coordination
Australian Institute for Machine Learning, Adelaide University · Responsible AI Research Centre, Australian Institute for Machine Learning
NavTrust: Benchmarking Trustworthiness for Embodied Navigation
Mar 19, 2026Huaide Jiang, Yash Chaudhary, Yuping Wang +8Embodied Navigation Robot Navigation Benchmarks
Northwestern University, IL, USA. · University of California, Riverside, CA, USA. · University of Michigan, Ann Arbor, MI, USA. +5
Meanings and Measurements: Multi-Agent Probabilistic Grounding for Vision-Language Navigation
Mar 19, 2026Swagat Padhan, Lakshya Jain, Bhavya Minesh Shah +3Visual Grounding for Robotics Spatial Grounding for VLA Models
Arizona State University · Haverford College
GSMem: 3D Gaussian Splatting as Persistent Spatial Memory for Zero-Shot Embodied Exploration and Reasoning
Mar 19, 2026Yiren Lu, Yi Du, Disheng Liu +3Long-Term Memory for Embodied Agents Embodied AI
Case Western Reserve University · Spatial AI & Robotics (SAIR) Lab, University at Buffalo
GoalVLM: VLM-driven Object Goal Navigation for Multi-Agent System
Mar 18, 2026MoniJesu James, Amir Atef Habel, Aleksey Fedoseev +1Open-Vocabulary Object Navigation Object-Goal Navigation
Intelligent Space Robotics Laboratory, Center for Digital Engineering, Skolkovo Institute of Science and Technology, Moscow, Russia
AgentVLN: Towards Agentic Vision-and-Language Navigation
Mar 18, 2026Zihao Xin, Wentong Li, Yixuan Jiang +6Vision-Language Navigation Robot Navigation
Nanjing University of Aeronautics and Astronautics · Shandong University · Zhejiang University
P $^{3}$ Nav: End-to-End Perception, Prediction and Planning for Vision-and-Language Navigation
Mar 18, 2026Tianfu Li, Wenbo Chen, Haoxuan Xu +2Vision-Language Navigation
HKUST(GZ), Guangzhou, China
FloorPlan-VLN: A New Paradigm for Floor Plan Guided Vision-Language Navigation
Mar 18, 2026Kehan Chen, Yan Huang, Dong An +5Vision-Language Navigation Embodied Navigation
New Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences and School of Artificial Intelligence, University of Chinese Academy of Sciences, China · AMap, Alibaba Group · BAAI +2
OmniVLN: Omnidirectional 3D Perception and Token-Efficient LLM Reasoning for Visual-Language Navigation across Air and Ground Platforms
Mar 18, 2026Zhongyuang Liu, Min He, Shaonan Yu +5Vision-Language Navigation Robot Navigation
CertaintyX. · the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798. · the Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA. +1
Trajectory-Diversity-Driven Robust Vision-and-Language Navigation
Mar 16, 2026Jiangyang Li, Cong Wan, SongLin Dong +4Vision-Language Navigation
Xi’an Jiaotong University · Faculty of Computility Microelectronics, Shenzhen University of Advanced Technology
LightZeroNav: Zero-Shot Vision Language Navigation in Continuous Environments Based on Lightweight VLMs
Mar 16, 2026Kun Luo, Xiangyu Dong, Xiaoguang Ma +2Vision-Language Navigation Vision-Language Models
Foshan Graduate School of Innovation, Northeastern University, Foshan 528311, China · Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China · QingniaoAI, China +1
HiMemVLN: Enhancing Reliability of Open-Source Zero-Shot Vision-and-Language Navigation with Hierarchical Memory System
Mar 16, 2026Kailin Lyu, Kangyi Wu, Pengna Li +9Vision-Language Navigation Memory-Augmented VLMs
Institute of Automation, Chinese Academy of Sciences, and the Zhongguancun Academy, Beijing, China · Zhongguancun Academy, Beijing, China · Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, China +7
CORAL: COntextual Reasoning And Local Planning in A Hierarchical VLM Framework for Underwater Monitoring
Mar 16, 2026Zhenqi Wu, Yuanjie Lu, Xuesu Xiao +1Autonomous Underwater Navigation Vision-Language Navigation
University of South Florida · George Mason University
All-day Multi-scenes Lifelong Vision-and-Language Navigation with Tucker Adaptation
Mar 15, 2026Xudong Wang, Gan Li, Zhiyu Liu +3Tucker Decomposition Vision-Language Navigation
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · North University of China +1
ImagiNav: Scalable Embodied Navigation via Generative Visual Prediction and Inverse Dynamics
Mar 14, 2026Jie Chen, Yuxin Cai, Yizhuo Wang +5Robot Navigation Vision-Language Navigation
Department of Mechanical Engineering, National University of Singapore, Singapore · Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR), Singapore · Nanyang Technological University (NTU), Singapore

Apr 12, 2026J/K move · Enter open · S save

VLN-NF: Feasibility-Aware Vision-and-Language Navigation with False-Premise Instructions

Hung-Ting Su, Ting-Jun Wang, Jia-Fong Yeh, Min Sun+1

National Taiwan University · DRIC · National Tsing Hua University

VLN Benchmarks Vision-Language Navigation Robot Navigation Instruction Following

Open paper AskPDF arXiv

Conventional Vision-and-Language Navigation (VLN) benchmarks assume instructions are feasible and the referenced target exists, leaving agents ill-equipped to handle false-premise goals. We introduce VLN-NF, a benchmark with false-premise instructions where the target is absent from the specified room and agents must navigate, gather evidence through in-room exploration, and explicitly output NOT-FOUND. VLN-NF is constructed via a scalable pipeline that rewrites VLN instructions using an LLM and verifies target absence with a VLM, producing plausible yet factually incorrect goals. We further propose REV-SPL to jointly evaluate room reaching, exploration coverage, and decision correctness. To address this challenge, we present ROAM, a two-stage hybrid that combines supervised room-level navigation with LLM/VLM-driven in-room exploration guided by a free-space clearance prior. ROAM achieves the best REV-SPL among compared methods, while baselines often under-explore and terminate prematurely under unreliable instructions. VLN-NF project page can be found at https://vln-nf.github.io/.