cs.ROApr 2, 2026

Stop Wandering: Efficient Vision-Language Navigation via Metacognitive Reasoning

Authors: Xueying Li, Feng Lyu, Hao Wu, Mingliu Liu, Jia-Nan Liu, Guozi Liu

Organizations: Central South University, Changsha, China · Nanjing University, Nanjing, China · State Grid Hubei Electric Power Research Institute, Wuhan, China · Dongguan University of Technology, Dongguan, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Training-free Vision-Language Navigation (VLN) agents powered by foundation models can follow instructions and explore 3D environments. However, existing approaches rely on greedy frontier selection and passive spatial memory, leading to inefficient behaviors such as local oscillation and redundant revisiting. We argue that this stems from a lack of metacognitive capabilities: the agent cannot monitor its exploration progress, diagnose strategy failures, or adapt accordingly. To address this, we propose MetaNav, a metacognitive navigation agent integrating spatial memory, history-aware planning, and reflective correction. Spatial memory builds a persistent 3D semantic map. History-aware planning penalizes revisiting to improve efficiency. Reflective correction detects stagnation and uses an LLM to generate corrective rules that guide future frontier selection. Experiments on GOAT-Bench, HM3D-OVON, and A-EQA show that MetaNav achieves state-of-the-art performance while reducing VLM queries by 20.7%, demonstrating that metacognitive reasoning significantly improves robustness and efficiency.

Figures & tables

Figure 1. Qualitative trajectory comparison. Baselines suffer from local oscillation or failure when trapped by spatial ambiguities; MetaNav leverages episodic reflection to break deadlocks and generate efficient paths.
Figure 2. System overview of MetaNav . Spatial Memory Construction (D1) builds a persistent 3D semantic map and extracts frontiers from RGB-D input. History-Aware Heuristic Planning (D2) selects a frontier via a utility function combining semantic relevance, geometric cost, and episodic penalty, then executes it for a fixed replanning interval. Reflection and Correction (D3) maintains episodic memory, detects stagnation via information gain, and invokes an LLM to inject corrective rules into D2. Arrows indicate data flow; dashed arrows denote LLM/VLM queries.
MethodSR ↑\uparrowSPL ↑\uparrow
Supervised methods
SenseAct-NN Monolithic ( Khanna et al., 2024 )12.36.8
SenseAct-NN Skill Chain ( Khanna et al., 2024 )29.511.3
MTU3D ( Zhu et al., 2025 )47.227.7
Training-free methods
Modular CLIP on Wheels ( Gadre et al., 2023 )16.110.4
Table 1. Results on the Val Unseen split of GOAT-Bench. Best and second-best results are in bold and underline .
Figure 3. Performance across instruction modalities on GOAT-Bench.
MethodSR ↑\uparrowSPL ↑\uparrow
Supervised methods
BC ( Pomerleau, 1988 )5.41.9
DAgger ( Ross et al., 2011 )10.24.7
DAgRL ( Yokoyama et al., 2024b )18.37.9
RL ( Schulman et al., 2017 )18.67.5
DAgRL+OD ( Yokoyama et al., 2024b )37.119.8
Table 2. Results on the Val Unseen split of HM3D-OVON. Best and second-best results are in bold and underline .
MethodLLM-Match ↑\uparrowLLM-SPL ↑\uparrow
Blind LLMs
LLaMA-2 ( Touvron et al., 2023 )29.0N/A
GPT-4 ( Achiam et al., 2023 )35.5N/A
LLM with captions
LLaMA-2 w/ LLaVA-1.5 ( Liu et al., 2024a )30.95.9
GPT-4 w/ LLaVA-1.5 ( Liu et al., 2024a )38.17.0
Table 3. Results on A-EQA. Best and second-best results are in bold and underline .
Architecture VariantSR ↑\uparrowSPL ↑\uparrow
MetaNav (Full Architecture)71.451.8
Ablating D3: Reflection and Correction
w/o Long-Term Summary68.749.6
w/o Episodic Memory69.249.9
w/o Reflection66.347.1
Ablating D2: History-Aware Heuristic Planning
Table 4. Ablation study on GOAT-Bench Val Unseen.
Figure 4. Effect of replanning interval on GOAT-Bench and HM3D-OVON.
Figure 5. Effect of short-term memory capacity KK on GOAT-Bench.
DesignComponentLatency
Design 1Point Cloud Projection54.0 ms
Memory Fusion359.9 ms
Design 2Unified Scoring & Penalty2.0 ms
LLM Inference5.40 s
Design 3Reflection & Summary5.75 s
Table 5. Average latency of MetaNav components.
Figure 6. Trajectory comparison across four goal modalities. 3D-Mem (red) shows local oscillation and target confusion; MetaNav (green) produces efficient paths.
MethodBase Nav.Reflect.SummaryTotal ↓\downarrow
3D-Mem31.6N/AN/A31.6
MetaNav (Ours)21.31.91.825.1
Table 6. Average VLM queries per episode on GOAT-Bench (evaluated on RTX 3080 GPU).
More in this topic

Vision-Language Navigation

Vision-Language Navigation studies how embodied agents use natural-language instructions and visual observations to plan and execute routes.

See all 310 papers

Explore similar work

CardsList

  1. SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoning

    Jun 8, 2026Yucheng Deng, Pingrui Lai, Xinhai Li +5Visual Spatial ReasoningEmbodied Navigation

    School of Information Science and Electronic Engineering & School of Integrated Circuits, Shanghai Jiao Tong University · Institute of Artificial Intelligence, China Telecom · 3Central South University +1

  2. SeeNav-Agent: Enhancing Vision-Language Navigation with Visual Prompt and Step-Level Policy Optimization

    Dec 2, 2025Zhengcheng Wang, Zichuan Lin, Yijun Yang +2Vision-Language Navigation

    Tencent AI Lab

  3. ReflectVLN: Training Vision-Language Navigation Agents with Reflective Reasoning

    Jul 14, 2026Jiahang Wang, Yirong Yang, Yanqing Zhu +4Hierarchical PlanningVision-Language Navigation

    Amap, Alibaba Group, China · Beihang University