Vision-Language Navigation

Also known as VLN

Vision-Language Navigation (VLN) asks embodied agents—such as mobile robots and drones—to follow language instructions by interpreting what they see and choosing actions over time. Approaches range from end-to-end vision-language policies to hierarchical systems that combine large models with maps, scene graphs, spatial memory, geometric planners, and world models; imitation learning and reinforcement learning are common training strategies. Methods are tested on benchmarks such as R2R, RxR, REVERIE, VLN-CE, and HM3D, with growing attention to object goals, outdoor and aerial settings, and real-robot deployment. A central challenge is turning correct language and scene understanding into safe, efficient behavior over long routes, where localization drift, changing scenes, limited compute, and the gap between simulation and reality can cause failures.

Latest papers

313 papers52 papers in the last four weeks, up 16% on the four weeks before. 0.3% of all new papers.

Your Vision-Language-Action Model Already Has Attention Heads For Path Deviation Detection
Mar 14, 2026Jaehwan Jeong, Evelyn Zhu, Jinying Lin +5Robotic Anomaly Detection Failure Detection for VLA Models
Korea University · University of California, Los Angeles · NVIDIA
DecoVLN: Decoupling Observation, Reasoning, and Correction for Vision-and-Language Navigation
Mar 13, 2026Zihao Xin, Wentong Li, Yixuan Jiang +4Vision-Language Navigation
Nanjing University of Aeronautics and Astronautics · Shandong University
HaltNav: Reactive Visual Halting over Lightweight Topological Priors for Robust Vision-Language Navigation
Mar 13, 2026Zihui Yu, Pingcong Li, Bichi Zhang +1Vision-Language Navigation
Key Laboratory of Intelligent Perception and Human-Machine Collaboration - ShanghaiTech University, Ministry of Education, China.
OnFly: Onboard Zero-Shot Aerial Vision-Language Navigation toward Safety and Efficiency
Mar 11, 2026Guiyong Zheng, Yueting Ban, Mingjie Zhang +2Vision-Language Navigation Aerial VLN
School of Artificial Intelligence, Sun Yat-Sen University, Zhuhai, China. · Southern University of Science and Technology, Shenzhen, China · The Hong Kong University of Science and Technology, Guangzhou, China.
BEACON: Language-Conditioned Navigation Affordance Prediction under Occlusion
Mar 10, 2026Xinyu Gao, Gang Chen, Javier Alonso-MoraRobot Navigation Vision-Language Navigation
Department of Cognitive Robotics (CoR), Delft University of Technology
Let's Reward Step-by-Step: Step-Aware Contrastive Alignment for Vision-Language Navigation in Continuous Environments
Mar 10, 2026Haoyuan Li, Rui Liu, Hehe Fan +1Vision-Language Navigation RL for VLMs
College of Computer Science and Technology, Zhejiang University, Hangzhou, China
Context-Nav: Context-Driven Exploration and Viewpoint-Aware 3D Spatial Reasoning for Instance Navigation
Mar 10, 2026Won Shik Jang, Ue-Hwan KimVision-Language Navigation 3D Spatial Reasoning
Department of AI Convergence, Gwangju Institute of Science and Technology, Gwangju, South Korea
Implicit Geometry Representations for Vision-and-Language Navigation from Web Videos
Mar 10, 2026Mingfei Han, Haihong Hao, Liang Ma +6Vision-Language Navigation 3D Scene Representation
Department of Computer Vision, Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates. · School of Information Science and Technology, University of Science and Technology of China. · Shenzhen Campus of Sun Yat-Sen University.
SPAN-Nav: Generalized Spatial Awareness for Versatile Embodied Navigation
Mar 10, 2026Jiahang Liu, Tianyu Xu, Jiawei Chen +9Embodied Navigation Vision-Language Navigation
Peking University · Galbot · BAAI
R2F: Repurposing Ray Frontiers for LLM-free Object Navigation
Mar 9, 2026Francesco Argenziano, John Mark Alexis Marcelo, Michele Brienza +5Open-Vocabulary Object Navigation Vision-Language Navigation
Department of Computer, Automation and Management Engineering, Sapienza University of Rome, Rome, Italy · Department of Electronics and Automation, Mohamed Khider University of Biskra, Biskra, Algeria · International University of Rome UNINT, Rome, Italy
ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigation
Mar 9, 2026Haoyu Tong, Xiangyu Dong, Xiaoguang Ma +3Aerial VLN Visual Navigation
Tianmushan Laboratory, Beihang University, Hangzhou 311115, China · Hangzhou International Innovation Institute, Beihang University, Hangzhou 311115, China · Foshan Graduate School of Innovation, Northeastern University, Foshan, China +3
CMMR-VLN: Vision-and-Language Navigation via Continual Multimodal Memory Retrieval
Mar 9, 2026Haozhou Li, Xiangyu Dong, Huiyan Jiang +2Vision-Language Navigation Memory-Augmented Agents
The authors are with the Foshan Graduate School of Innovation at Northeastern University, Foshan, China · The authors are with the College of Software at Northeastern University, Shenyang, China · The authors are with the School of Aeronautic Science and Engineering at Beihang University, Beijing, China +1
VLN-Cache: Enabling Token Caching for VLN Models with Visual/Semantic Dynamics Awareness
Mar 7, 2026Zihao Zheng, Zhihao Mao, Xingyue Zhou +9Efficient VLM Inference Vision-Language Navigation
School of Computer Science, Peking University, Beijing, China. · School of Computer Science, China University of Geosciences (Wuhan), Wuhan, China. · School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China. +1
History-Conditioned Spatio-Temporal Visual Token Pruning for Efficient Vision-Language Navigation
Mar 6, 2026Qitong Wang, Yijun Liang, Ming Li +2Efficient VLM Inference Token Pruning
The authors are with the Department of Computer and Information Sciences at the University of Delaware, Newark, DE 19711, USA. · The authors are with the University of Maryland’s Department of Computer Science, College Park, MD 20742, USA. · The author is with the Mohamed bin Zayed University of Artificial Intelligence, Masdar City, Abu Dhabi 7909, UAE.
Lifelong Embodied Navigation Learning
Mar 6, 2026Xudong Wang, Jiahua Dong, Baichen Liu +3Embodied Navigation Continual Learning for LLM Agents
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Mohamed bin Zayed University of Artificial Intelligence
OpenFrontier: General Navigation with Visual-Language Grounded Frontiers
Mar 5, 2026Esteban Padilla-Cerdio, Boyang Sun, Marc Pollefeys +1Robot Navigation Vision-Language Navigation
1ETH Zurich · 2Microsoft Spatial AI Lab · University of Bonn
RAGNav: A Retrieval-Augmented Topological Reasoning Framework for Multi-Goal Visual-Language Navigation
Mar 4, 2026Ling Luo, Qiangian BaiVision-Language Navigation Visual Spatial Reasoning
Southwestern University of Finance and Economics, Chengdu, 611130, China
PROSPECT: Unified Streaming Vision-Language Navigation via Semantic--Spatial Fusion and Latent Predictive Representation
Mar 4, 2026Zehua Fan, Wenqi Lyu, Wenxuan Song +12Vision-Language Navigation Vision-Language-Action Models
Z. Fan and F. Gao are with Shanghai Jiao Tong University, Shanghai, China. · W. Lyu is with the University of Adelaide, Adelaide, Australia. · W. Song and J. He are with the Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. +5
MA-CoNav: A Master-Slave Multi-Agent Framework with Hierarchical Collaboration and Dual-Level Reflection for Long-Horizon Embodied VLN
Mar 3, 2026Ling Luo, Qianqian BaiVision-Language Navigation Multi-Agent Coordination
Southwestern University of Finance and Economics, Chengdu, 611130, China
TagaVLM: Topology-Aware Global Action Reasoning for Vision-Language Navigation
Mar 3, 2026Jiaxing Liu, Zexi Zhang, Xiaoyan Li +3Vision-Language Navigation Visual Navigation
School of Information Science and Technology, Beijing University of Technology, China. · Now at Imperial College London, U.K.
CoFL: Continuous Flow Fields for Language-Conditioned Navigation
Mar 3, 2026Haokun Liu, Zhaoqi Ma, Yicheng Chen +5Vision-Language Navigation Flow Map Learning
SFCo-Nav: Efficient Zero-Shot Visual Language Navigation via Collaboration of Slow LLM and Fast Attributed Graph Alignment
Mar 2, 2026Chaoran Xiong, Litao Wei, Xinhao Hu +5Vision-Language Navigation Robot Navigation
The authors are with the Shanghai Key Laboratory of Navigation and Location Based Services, Shanghai Jiao Tong University (SJTU), Shanghai 200240, China. · Litao Wei is also with Zhiyuan College, SJTU. · Ling Pei is also with the State Key Laboratory of Submarine Geoscience, SJTU.
ProFocus: Proactive Perception and Focused Reasoning in Vision-and-Language Navigation
Mar 1, 2026Wei Xue, Mingcheng Li, Xuecheng Wu +3Vision-Language Navigation
College of Intelligent Robotics and Advanced Manufacturing, Fudan University · Xi’an Jiaotong University · ByteDance +1
Enhancing Vision-Language Navigation with Multimodal Event Knowledge from Real-World Indoor Tour Videos
Feb 27, 2026Haoxuan Xu, Tianfu Li, Wenbo Chen +4Vision-Language Navigation Multimodal KGs
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China · Tsinghua University, Shenzhen 518055, China · Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi 31122, UAE +1
Global Commander and Local Operative: A Dual-Agent Framework for Scene Navigation
Feb 21, 2026Kaiming Jin, Yuefan Wu, Shengqiong Wu +3Vision-Language Navigation VLMs for Robotics
National University of Singapore · Simon Fraser University · University of Oxford
CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigation
Feb 20, 2026Xia Su, Ruiqi Chen, Benlin Liu +4Embodied Spatial Reasoning VLN Benchmarks
University of Washington Seattle, WA, USA · University of California, Santa Cruz Santa Cruz, CA, USA
Learning to Retrieve Navigable Candidates for Efficient Vision-and-Language Navigation
Feb 17, 2026Shutian Gu, Chengkai Huang, Ruoyu Wang +1Vision-Language Navigation Retrieval-Augmented Generation
University of New South Wales, Sydney, Australia · Macquarie University, Sydney, Australia · Data61, CSIRO, Sydney, Australia
One Agent to Guide Them All: Empowering MLLMs for Vision-and-Language Navigation via Explicit World Representation
Feb 17, 2026Zerui Li, Hongpei Zheng, Fangguo Zhao +5Vision-Language Navigation VLMs for Robotics
Australian Institute for Machine Learning, Adelaide University · The University of Manchester · Zhejiang University +1
pFedNavi: Structure-Aware Personalized Federated Vision-Language Navigation for Embodied AI
Feb 16, 2026Qingqian Yang, Hao Wang, Sai Qian Zhang +6Federated Learning Aggregation Non-IID Federated Learning
Stevens Institute of Technology · Hoboken, NJ, USA · New York University +9
LongNav-R1: Horizon-Adaptive Multi-Turn RL for Long-Horizon VLA Navigation
Feb 12, 2026Yue Hu, Avery Xi, Qixin Xiao +4Robot Navigation Vision-Language Navigation
University of Michigan, Ann Arbor

Mar 14, 2026J/K move · Enter open · S save

Your Vision-Language-Action Model Already Has Attention Heads For Path Deviation Detection

Jaehwan Jeong, Evelyn Zhu, Jinying Lin, Emmanuel Jaimes+4

Korea University · University of California, Los Angeles · NVIDIA

Robotic Anomaly Detection Failure Detection for VLA Models Vision-Language-Action Models Vision-Language Navigation

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have demonstrated strong potential for predicting semantic actions in navigation tasks, demonstrating the ability to reason over complex linguistic instructions and visual contexts. However, they are fundamentally hindered by visual-reasoning hallucinations that lead to trajectory deviations. Addressing this issue has conventionally required training external critic modules or relying on complex uncertainty heuristics. In this work, we discover that monitoring a few attention heads within a frozen VLA model can accurately detect path deviations without incurring additional computational overhead. We refer to these heads, which inherently capture the spatiotemporal causality between historical visual sequences and linguistic instructions, as Navigation Heads. Using these heads, we propose an intuitive, training-free anomaly-detection framework that monitors their signals to detect hallucinations in real time. Surprisingly, among over a thousand attention heads, a combination of just three is sufficient to achieve a 44.6 % deviation detection rate with a low false-positive rate of 11.7 %. Furthermore, upon detecting a deviation, we bypass the heavy VLA model and trigger a lightweight Reinforcement Learning (RL) policy to safely execute a shortest-path rollback. By integrating this entire detection-to-recovery pipeline onto a physical robot, we demonstrate its practical robustness. All source code will be publicly available.