Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation
Organizations: School of Software Technolog,Zhejiang University · Zhejiang Humanoid Robot Innovation Center Co., Ltd.
Abstract
Although Large Vision-Language Models (VLMs) have significantly advanced embodied navigation, their direct deployment remains challenging, as existing methods often force VLMs into unnatural action spaces that misalign with their 2D pre-training priors, compounded by rigid reasoning schedules and inefficient memory management. To overcome these limitations, we propose TAMP-Nav, a unified framework for efficient embodied navigation. First, we introduce a Pixel-to-3D Action Formulation (Point) that reformulates navigation into 2D visual prompting. Specifically, the VLM merely selects 2D pixels, which are then projected into 3D coordinates for a low-level SLAM controller. This design naturally aligns embodied execution with the VLM's inherent 2D visual capabilities. Second, we propose an integrated Selective Reasoning and Anchor-Trajectory Memory mechanism (Think and Memorize), which dynamically triggers Chain-of-Thought and retains high-fidelity memory only at critical nodes, compressing redundant trajectories into lightweight Space-Time Indicators, thereby preserving critical historical information and enhancing spatio-temporal perception. Finally, we design an efficient Two-Level Alignment Paradigm (Align) via Group Relative Policy Optimization (GRPO). By superimposing global outcome rewards with fine-grained process rewards, this dense supervision tightly aligns the agent's cognitive planning with physical environmental feedback, endowing the model with adaptive reasoning capabilities. Experiments demonstrate that TAMP-Nav achieves state-of-the-art performance (e.g., 66.2% SR on R2R-CE) with high runtime and sample efficiency (requiring only 90k training trajectories).
Figures & tables
| R2R-CE Val-Unseen | RxR-CE Val-Unseen | |||||||
| Method | NE | OS | SR | SPL | NE | SR | SPL | nDTW |
| HPN+DN* Krantz et al. (2021) | 6.31 | 40.0 | 36.0 | 34.0 | - | - | - | - |
| CMA* Hong et al. (2022) | 6.20 | 52.0 | 41.0 | 36.0 | 8.76 | 26.5 | 22.1 | 47.0 |
| GridMM* Wang et al. (2023a) | 5.11 | 61.0 | 49.0 | 41.0 | - | - | - | - |
| ETPNav* An et al. (2024) | 4.71 | 65.0 | 57.0 | 49.0 | 5.64 | 54.7 | 44.8 | 61.9 |
| ScaleVLN* Wang et al. (2023b) | 4.80 | - | 55.0 | 51.0 | - | - | - | - |
| Block | Variant | R2R-CE: NE / OS / SR / SPL | RxR-CE: NE / SR / SPL / nDTW |
|---|---|---|---|
| Action (SFT only) | NavFoM-style metric waypoint | 7.18 / 42.0 / 30.9 / 24.7 | 8.47 / 27.2 / 20.6 / 43.1 |
| Pixel-to-3D | 4.88 / 62.0 / 55.7 / 50.3 | 6.10 / 52.4 / 46.2 / 62.1 | |
| GRPO | Global trajectory advantage only | 4.52 / 67.0 / 59.3 / 52.1 | 5.30 / 58.4 / 49.7 / 65.5 |
| Two-Level without annealed guidance | 4.12 / 71.5 / 63.4 / 56.0 | 4.72 / 62.6 / 53.8 / 69.4 | |
| Full Two-Level GRPO | 3.85 / 74.5 / 66.2 / 58.8 | 4.32 / 65.7 / 56.9 / 72.4 | |
| Memory | Uniform sampling | 4.43 / 68.2 / 60.8 / 53.6 | 5.03 / 59.4 / 50.8 / 66.8 |
| SFT supervision | R2R-CE: NE / OS / SR / SPL | RxR-CE: NE / SR / SPL / nDTW |
|---|---|---|
| Action only (no CoT) | 5.31 / 57.0 / 50.6 / 45.0 | 6.71 / 46.1 / 39.8 / 56.2 |
| Qwen2.5-VL-7B CoT | 5.46 / 55.7 / 49.4 / 43.1 | 6.83 / 44.8 / 38.2 / 54.9 |
| GLM-4.5V-108B CoT | 4.99 / 60.8 / 54.4 / 48.9 | 6.24 / 51.2 / 44.9 / 60.8 |
| Gemini 2.5 Flash CoT | 4.88 / 62.0 / 55.7 / 50.3 | 6.10 / 52.4 / 46.2 / 62.1 |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| System Prompt |
| You are a strict filter for navigation instructions. |
| - Pure action = a movement-only command without a target object/landmark/destination, e.g., ”go straight”, ”turn left”. |
| - Not pure action = includes a target or landmark, e.g., ”go straight to the bed”. |
| If there are 4 or more consecutive pure actions anywhere in the instruction, output DROP. Otherwise output KEEP. Consecutive means back-to-back pure actions with no non-pure-action instruction between them. |
| User Prompt Template |
| Instruction: {task} |
| System Prompt |
| You are an expert at summarizing navigation trajectories. Given a task and image sequences, your job is to summarize the trajectory from image in ONE continuous paragraph. Explicitly state which part of the task has been completed. Do NOT mention future parts. Return ONLY a JSON object. |
| User Prompt Template |
| Task: {task} |
| Summary ranges: {segment_list_text} |
| For EACH range [0 k], write ONE paragraph in first person. Describe what happened and which part of the task is completed by that point. |
| System Prompt |
| You are a controlled reasoning module. Describe the indoor scene layout (open paths, obstacles, landmarks) based on the attached front, left, back, and right views. Only describe visible objects. Do NOT mention targets, goals, history, or next steps. |
| Output Format |
| "Analyze the current observations": [your detailed description] |
| System Prompt |
| IRON RULE: IF gt_action == "stop" , declare arrival. ELSE, reason step-by-step how the move advances the task. The final sentence MUST name the target view and an object-defined region. FORBIDDEN: coordinates, pixels, red circles, proportions. |
| User Prompt Template |
| Task: {task} History: {summary} Analysis: {analysis} |
| Target Guidance: {target_guidance} (Use this to locate the region, but do NOT mention the guidance itself). |
| System Prompt |
| Rewrite the three-stage reasoning into a single, coherent English paragraph. Preserve spatial logic and the target view. Remove redundancies. Do NOT include stage titles. Output ONLY the polished paragraph. |
Embodied Navigation
Embodied navigation studies how robots perceive, map, and move through environments to reach goals and follow instructions.
See all 134 papers