cs.ROMar 1, 2026

ProFocus: Proactive Perception and Focused Reasoning in Vision-and-Language Navigation

Authors: Wei Xue, Mingcheng Li, Xuecheng Wu, Jingqun Tang, Dingkang Yang, Lihua Zhang

Organizations: College of Intelligent Robotics and Advanced Manufacturing, Fudan University · Xi’an Jiaotong University · ByteDance · Fysics Intelligence Technologies Co., Ltd. (Fysics AI)

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Vision-and-Language Navigation (VLN) requires agents to accurately perceive complex visual environments and reason over navigation instructions and histories. However, existing methods passively process redundant visual inputs and treat all historical contexts indiscriminately, resulting in inefficient perception and unfocused reasoning. To address these challenges, we propose ProFocus, a training-free progressive framework that unifies Proactive Perception and Focused Reasoning through collaboration between large language models (LLMs) and vision-language models (VLMs). For proactive perception, ProFocus transforms panoramic observations into structured ego-centric semantic maps, enabling the orchestration agent to identify missing visual information needed for reliable decision-making, and to generate targeted visual queries with corresponding focus regions that guide the perception agent to acquire the required observations. For focused reasoning, we propose Branch-Diverse Monte Carlo Tree Search (BD-MCTS) to identify top-kk high-value waypoints from extensive historical candidates. The decision agent focuses reasoning on the historical contexts associated with these waypoints, rather than considering all historical waypoints equally. Extensive experiments validate the effectiveness of ProFocus, achieving state-of-the-art performance among zero-shot methods on R2R and REVERIE benchmarks.

Figures & tables

Figure 1: Overview of the ProFocus framework. ProFocus consists of two core mechanisms: Proactive Perception , which selectively acquires instruction-relevant visual evidence through a closed perception–reasoning loop, and Focused Reasoning , which leverages BD-MCTS to highlight top- kk high-value waypoints for effective focused reasoning.
MethodsNE ↓\downarrowOSR ↑\uparrowSR ↑\uparrowSPL ↑\uparrow
Training-based VLN specialists
Seq2Seq [ 2 ]7.812821–
Speaker-Follower [ 9 ]6.624536–
EnvDrop [ 31 ]5.22–5248
Pretraining-based VLN specialists
PREVALENT [ 10 ]4.71–5853
Table 1: Results on R2R validation unseen set. † denotes our re-implementation with different foundation models. Bold best, underline second-best within each method group .
MethodsOSR ↑\uparrowSR ↑\uparrowSPL ↑\uparrow
Training-based VLN specialists
Seq2Seq [ 2 ]8.14.22.8
FAST-MATTN [ 25 ]28.214.47.2
NavQ [ 34 ]62.054.139.2
Pretraining-based VLN specialists
HAMT [ 7 ]35.431.629.6
Table 2: Results on REVERIE validation unseen set. † denotes our re-implementation with different foundation models. Bold best, underline second-best within each method group .
Figure 2: Performance comparison on the 30 longest navigation episodes from R2R validation unseen set.
MethodsNE ↓\downarrowOSR ↑\uparrowSR ↑\uparrowSPL ↑\uparrow
DS3+GLM
ProFocus5.2163.050.041.2
ProFocus w/o BD-MCTS5.7554.246.940.0
ProFocus w/o PP6.4550.042.030.0
Q3+Q3VL
ProFocus4.9265.052.039.8
Table 3: Ablation study on R2R validation unseen set. BD-MCTS: Branch-Diverse MCTS for focused reasoning, PP: Proactive Perception mechanism.
MethodsOSR ↑\uparrowSR ↑\uparrowSPL ↑\uparrow
DS3+GLM
ProFocus57.1436.9025.85
ProFocus w/o BD-MCTS42.6733.3326.73
ProFocus w/o PP36.0032.5023.05
Q3+Q3VL
ProFocus51.6740.0024.83
Table 4: Ablation study on REVERIE validation unseen set. BD-MCTS: Branch-Diverse MCTS for focused reasoning, PP: Proactive Perception mechanism.
Figure 3: Qualitative case study demonstrating proactive perception and focused reasoning. Left : Proactive perception enables accurate semantic value estimation by acquiring fine-grained visual details. Right : BD-MCTS-guided top- kk focused reasoning enables error correction through global historical awareness.
More in this topic

Vision-Language Navigation

Vision-Language Navigation studies how embodied agents use natural-language instructions and visual observations to plan and execute routes.

See all 310 papers

Explore similar work

CardsList
Mar 18, 2026

P3^{3}Nav: End-to-End Perception, Prediction and Planning for Vision-and-Language Navigation

In Vision-and-Language Navigation (VLN), an agent is required to plan a path to the target specified by the language instruction, using its visual observations. Consequently, prevailing VLN methods primarily focus on building powerful planners through visual-textual alignment. However, these approaches often bypass the imperative of comprehensive scene understanding prior to planning, leaving the agent with insufficient perception or prediction capabilities. Thus, we propose P3^{3}Nav, a novel end-to-end framework integrating perception, prediction, and planning in a unified pipeline to strengthen the VLN agent's scene understanding and boost navigation success. Specifically, P3^{3}Nav augments perception by extracting complementary cues from object-level and map-level perspectives. Subsequently, our P3^{3}Nav predicts waypoints to model the agent's potential future states, endowing the agent with intrinsic awareness of candidate positions during navigation. Conditioned on these future waypoints, P3^{3}Nav further forecasts semantic map cues, enabling proactive planning and reducing the strict reliance on purely historical context. Integrating these perceptual and predictive cues, a holistic planning module finally carries out the VLN tasks. Extensive experiments demonstrate that our P3^{3}Nav achieves new state-of-the-art performance on the REVERIE, R2R-CE, and RxR-CE benchmarks.

Tianfu Li, Wenbo Chen, Haoxuan Xu +2

HKUST(GZ), Guangzhou, China

Vision-Language Navigation
AskPDF
May 21, 2026

AwareVLN: Reasoning with Self-awareness for Vision-Language Navigation

Vision-and-Language Navigation (VLN) requires an agent to ground language instructions to its own movement within a visual environment. While state-of-the-art methods leverage the reasoning capabilities of Vision-Language Models (VLMs) for end-to-end action prediction, they often lack an explicit and explainable understanding of the relationships between the agent, the instruction, and the scene. Conversely, explicitly building a scene map for heuristic planning is intuitively appealing but relies on additional 3D sensors and hinders large-scale vision-language pre-training. To bridge this gap, we propose AwareVLN, a novel framework that equips the navigation model with a self-aware reasoning mechanism, enabling it to understand the agent's state and task progress in a fully end-to-end and data-driven manner. Our approach features two key innovations: (1) a structural reasoning module that fosters spatial and task-oriented self-awareness, and (2) an automatic data engine with progress division for effective training. Extensive experiments on various datasets in Habitat simulator show our AwareVLN significantly outperforms previous state-of-the-art vision-language navigation methods. Project page: https://gwxuan.github.io/AwareVLN/.

Wenxuan Guo, Xiuwei Xu, Yichen Liu +7

Tsinghua University, China

Vision-Language NavigationVLM ReasoningEmbodied AISpatial ReasoningReasoning for VLA Models
AskPDF
Apr 2, 2026

Stop Wandering: Efficient Vision-Language Navigation via Metacognitive Reasoning

Training-free Vision-Language Navigation (VLN) agents powered by foundation models can follow instructions and explore 3D environments. However, existing approaches rely on greedy frontier selection and passive spatial memory, leading to inefficient behaviors such as local oscillation and redundant revisiting. We argue that this stems from a lack of metacognitive capabilities: the agent cannot monitor its exploration progress, diagnose strategy failures, or adapt accordingly. To address this, we propose MetaNav, a metacognitive navigation agent integrating spatial memory, history-aware planning, and reflective correction. Spatial memory builds a persistent 3D semantic map. History-aware planning penalizes revisiting to improve efficiency. Reflective correction detects stagnation and uses an LLM to generate corrective rules that guide future frontier selection. Experiments on GOAT-Bench, HM3D-OVON, and A-EQA show that MetaNav achieves state-of-the-art performance while reducing VLM queries by 20.7%, demonstrating that metacognitive reasoning significantly improves robustness and efficiency.

Xueying Li, Feng Lyu, Hao Wu +3

Central South University, Changsha, China · Nanjing University, Nanjing, China · State Grid Hubei Electric Power Research Institute, Wuhan, China +1

Vision-Language NavigationMemory-Augmented VLMs
AskPDF