Organizations: MoE Key Lab of Artificial Intelligence, Institute of AI, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Beihang University, Beijing, China · The Hong Kong University of Science and Technology, Hong Kong, China · Tsinghua University, Beijing, China · University of Cambridge, Cambridge, UK · Intelligent Game and Decision Lab (IGDL), Beijing, China
Urban navigation requires embodied agents to pursue long-horizon goals through local decisions based on egocentric observations. However, existing agentic navigation methods often struggle to translate distant goals into coherent local decisions in large-scale physical environments. Their reliance on linguistic reasoning over transient observations or limited history constrains anticipation of the consequences of actions and future conditions, despite the importance of such foresight for navigating long and complex urban routes. To bridge this gap, we propose PreAct-Nav, an agentic navigation framework that equips frozen policies with anticipatory reasoning for robust urban navigation. Our central idea is to anchor local decisions in persistent medium-horizon subgoals, assess the consequences of predicted actions before execution, and continually update the reasoning context using actual outcomes. At its core, a navigation memory module maintains the active subgoal and relevant experience across decisions, translating distant goals into actionable intermediate objectives. We further introduce a predictive world sandbox that uses an action-conditioned world model (AC-WM) to forecast world dynamics conditioned on candidate movements. A vision-language model (VLM) reasoner interprets these predictions under the current subgoal to retain or revise actions. After execution, real observations are used to assess outcomes, correct inconsistent assumptions, and update memory to continue or reformulate the subgoal. Extensive evaluations demonstrate that the proposed PreAct-Nav improves action selection through memory updates and visual prediction, with more pronounced gains on longer routes and routes with more turns.
Figures & tables
Figure 1: Reasoning before action with PreAct-Nav. Navigating across an intersection requires local decisions to serve a long-horizon goal. Compared with reasoning from observed history and language-based future thoughts, PreAct-Nav maintains medium-horizon subgoals to decompose long-horizon navigation tasks and uses action-conditioned future visual cues to preview, review, and revise local actions before execution. Real feedback updates memory for subsequent decisions.
Figure 2: Overview of PreAct-Nav. A shared VLM coordinates Navigation Memory and the Predictive Sandbox through Action Preview, Action Review, Action Execute, and Memory Update. Memory provides observed experience and the current subgoal; the sandbox supplies predicted visual cues. Revised actions return to preview. After execution, real observations are checked against expected cues to update subgoal status and open-question progress.
Test-seen
Test-unseen
Method
SR ↑
SPL ↑
NE ↓
nDTW ↑
SDTW ↑
OSR ↑
SR ↑
SPL ↑
NE ↓
nDTW ↑
SDTW ↑
OSR ↑
UrbanNav
82.79
81.16
3.3596
94.10
81.73
91.91
78.14
74.73
5.9659
87.73
76.13
84.92
Reasoner: Qwen3.8-27B
Direct VLM
83.13
80.41
3.3192
94.31
82.02
92.49
79.05
75.58
5.9187
88.02
76.99
85.63
NavGPT
83.23
80.54
3.3090
94.35
82.12
92.49
79.11
75.65
5.9017
88.09
77.06
85.67
Open-Nav
83.15
80.45
3.3175
94.31
82.04
92.44
79.24
75.77
5.8964
88.11
77.18
85.74
Table 1: Comparison on UrbanNav. All methods use the UrbanNav policy, with rows grouped by VLM reasoner. NE is in metres; other metrics are percentages. Best results are bold.
Method
SR ↑
SPL ↑
NE ↓
nDTW ↑
SDTW ↑
OSR ↑
Direct VLM
43.51
43.11
18.28
37.44
32.33
43.51
NavGPT
73.38
70.19
10.12
51.00
48.32
73.38
Open-Nav
64.29
60.98
12.21
45.08
41.41
64.29
EvoNav
63.64
61.05
11.97
46.45
42.34
63.64
Ours
79.22
78.94
8.31
68.77
65.85
79.22
Table 2: Comparison on CityWalker. All methods use the frozen CityWalker policy and Qwen reasoner. NE is in metres; other metrics are percentages. Baseline adaptations and evaluation settings are detailed in Appendix D.5 .
Policy
SR ↑
SPL ↑
NE ↓
nDTW ↑
SDTW ↑
OSR ↑
UrbanNav
78.14
74.73
5.9659
87.73
76.13
84.92
+ harness
83.38
(+5.24)
77.87
(+3.13)
5.3353
(-0.6306)
90.47
(+2.74)
81.30
(+5.17)
90.12
(+5.20)
NoMaD
76.33
69.05
7.2776
82.73
73.55
81.20
+ harness
79.11
(+2.78)
71.81
(+2.76)
7.2727
(-0.0050)
82.74
(+0.01)
76.13
(+2.58)
81.20
(0.00)
OmniVLA
82.65
78.06
5.8666
87.77
79.98
86.90
+ harness
86.23
(+3.58)
81.44
(+3.39)
5.7795
(-0.0871)
88.36
(+0.58)
83.33
(+3.35)
88.07
(+1.17)
Table 3: Comparison across navigation policies on test-unseen. All pairs use Qwen. Green parentheses show changes from the corresponding frozen policy: percentage points for rate metrics and metres for NE. Changes are computed before rounding.
Reasoning Harness
Navigation Memory Tool
Predictive Sandbox Tool
SR ↑
SPL ↑
NE ↓
nDTW ↑
SDTW ↑
OSR ↑
×
×
×
78.14
74.73
5.9659
87.73
76.13
84.92
✓
×
×
80.56
76.74
5.6410
89.60
78.65
88.37
✓
✓
×
82.37
76.91
5.3986
90.34
80.34
89.90
✓
×
✓
81.48
77.24
5.4791
89.87
79.47
88.89
✓
✓
✓
83.38
77.87
5.3353
90.47
81.30
90.12
Table 4: Component ablation on test-unseen. ✓ / × indicate enabled/disabled components. The first row is the UrbanNav policy. Without either tool, the Reasoning Harness reviews and revises policy actions. NE is in metres; other metrics are percentages.
Figure 4: Action revision and memory update. (a) Predicted cues prompt a revision from the storefront toward the sidewalk, supported by the next real observation. (b) Pedestrians remain in the passage, prompting a new subgoal to pass on the right.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Real-world navigation case. A robot follows the building exterior toward a delivery locker. The panels connect the current subgoal, visual previews, action revision, and memory update.
Placeholder
Value supplied to the operation
destination_view
The unmodified destination-view description g .
observed_experience
Recent executed-action records: source step, action in its source frame, measured progress, expected cues, and observed feedback.
subgoal
Current objective, visual anchor, completion cue, open question, semantic status, and runtime metadata; null before initialization.
current_action_m
The proposed waypoint sequence, expressed in the current frame.
endpoint_x_m , endpoint_y_m
Coordinates of the final waypoint, formatted to three decimal places.
earlier_reviews
Ordered action indices, waypoint sequences, and structured reviews; an empty list at the start of a decision.
Appendix
Table 5: Runtime substitutions in the prompt templates. Placeholders are replaced by typed values before serialization; they are not additional model instructions.
Observed feedback
Subgoal status
Next operation
SUPPORTED
ACTIVE
Retain the objective and append evidence of compatible progress.
UNRESOLVED
ACTIVE
Retain the objective and unresolved question.
ACHIEVED
ACHIEVED
Close the completed subgoal; establish the next objective.
CONTRADICTED
CONTRADICTED
Reconsider the objective using the conflicting observation.
Appendix
Table 6: From observed feedback to memory state. Feedback is generated by the VLM; storage and semantic-status updates are performed by the harness.
Displayed field
Runtime field
Role in the method
Objective
intent , objective
Medium-horizon intent zt in Equation 2 .
Visual cues
anchor , completion
Observable anchors and progress/completion cues ct .
Open question
question
Unresolved question qt in memory; review can specify a query about the current action.
Action assessment
review verdict
ωtj in Equation 8 .
Preview useful
query_useful
κtj in Equation 8 .
Prediction quality
prediction_quality
Video assessment χtj .
Appendix
Table 7: Prompt outputs in the method and qualitative figures. Descriptive figure labels expose the semantics of the serialized fields without adding new model outputs.
Current action, its review, shared context, available images
Revised path or abstention, confidence, grounding, rationale, expected cues.
Memory Update
Subgoal, executed movement, expected cues, before/after real views
Observed-feedback verdict and rationale.
Appendix
Table 8: Information exchanged through the harness. Operations use the same vision–language reasoner. The table lists the inputs and outputs of each prompted operation.
Group
Episodes
SR
Δ SR interval
SPL
Δ SPL interval
Short
3,657
94.26 / 94.56
[0.10,0.51]
87.65 / 87.91
[0.08,0.47]
Long
977
37.87 / 41.56
[1.82,5.57]
36.71 / 40.26
[1.72,5.39]
Low + medium turn
3,089
94.92 / 95.40
[0.27,0.72]
88.48 / 88.92
[0.24,0.66]
High turn
1,545
57.28 / 59.35
[1.04,3.07]
53.77 / 55.77
[0.98,2.98]
Appendix
Table 9: Route-group results. SR/SPL show harness with Navigation Memory Tool / full method from Table 3 . Each partition covers all 4,634 episodes. Intervals describe the paired gain in percentage points.
Changed input
Measurement
Pairs
Result
Updated memory
Policy-action assessment changed
1,501
13.3%
Updated memory
Retain/revise decision changed
1,501
8.9%
Predicted video
States requesting a revision
525
147→214
Predicted video
Endpoint change when both revise
133
0.28 m
Appendix
Table 10: Paired effects on reasoning. Each comparison holds observed images, destination, and policy proposal fixed. Only the memory update or predicted-video input changes. These are state-level probes, separate from the navigation rollouts.
Measurement
Pairs
Estimate
95% interval
Memory: changed policy-action assessment
1,501
13.26
[11.90,14.70]
Memory: changed retain/revise request
1,501
8.93
[7.62,10.18]
Revision rate without prediction
525
28.00
[21.56,35.44]
Revision rate with prediction
525
40.76
[34.63,47.30]
Paired revision-rate difference
525
12.76
[8.90,16.80]
Endpoint change when both revise
133
0.278
[0.215,0.342]
Appendix
Table 11: Paired reasoning measurements. Rates are percentages; rate differences are percentage points. The last row is in metres.
Mean (s)
Median (s)
P90 (s)
Peak GPU memory (GiB)
11.37
11.35
11.47
34.06
Appendix
Table 12: Predictive sandbox cost on one NVIDIA H200 GPU. Statistics cover 1,700 rollouts, each containing 60 predicted frames. P90 denotes the 90th percentile.
Operation
Observed / predicted frames
Mean (s)
Median (s)
P90 (s)
Subgoal establishment
3 / 0
5.76
5.21
6.49
Action Preview
3 / 0
5.06
5.04
5.41
Action Review
3 / 4
5.19
5.05
5.65
Action Revision
3 / 0
8.65
9.82
10.77
Memory Update
2 / 0
2.57
2.35
2.91
Appendix
Table 13: Per-request latency on one NVIDIA H200 GPU at batch size one. Predicted inputs use frame indices 10, 20, 40, and 60, with index 0 denoting the input observation. Each row times only the named operation; video generation is reported in Table 12 .
Vision-Language Navigation (VLN) requires embodied agents to generate actions based on instructions and observations. General-purpose multimodal agents offer a promising basis for this task, but selecting plausible local actions does not ensure that execution remains consistent with the intended route, particularly in long-horizon tasks. Moreover, the accumulated interaction history increases the input required for subsequent decisions, resulting in a significant inference overhead. To this end, we introduce \method, an Agentic VLN framework that includes a Goal Agent that sets adaptive goals for local actions, a Verify Agent that dynamically verifies whether a goal has been completed, a Memory Agent for multimodal context compression, and a Visuomotor Agent to execute adaptive goals. Specifically, the Goal Agent formulates adaptive goals based on the instruction, current observation, and execution history. Then the Visuomotor Agent executes navigation actions to achieve each goal, while the Verify Agent uses a goal-specific verification question to dynamically assess whether the observed outcomes satisfy the intended completion condition. Verified goal completion then marks a boundary for the Memory Agent to compress the corresponding multimodal interaction history while preserving information needed for subsequent navigation. We evaluate navigation on R2R-CE and RxR-CE, examine framework variants across three model backbones, and study context evolution during execution. For Real-World evaluation, \method achieves 83.3% success and 1.51,m navigation error across eight challenging routes evaluated three times each.
Haoxiang Shi, Zaijing Li, Muhe Ding +3
Harbin Institute of Technology (Shenzhen) · Pengcheng Laboratory
City-scale outdoor navigation is currently hindered by the heavy reliance on dense maps or costly navigation supervision. In this work, we introduce a novel paradigm for leveraging directional instructions from commercial navigation tools (e.g., Google Maps). To bridge the gap between commercial instructions and executable navigation actions, while mitigating long-horizon error accumulation through robust trajectory recovery, we propose DA-Nav, a Direction-Aware vision-language Navigation framework that reformulates navigation as a discrete spatial grounding problem on the egocentric 2D image plane. To achieve trajectory recovery, DA-Nav employs a Chain-of-Thought (CoT) reasoning process encompassing deviation assessment, action prediction, and target grid selection. We further introduce ReDA, a dataset that provides direction-aware instructions and recovery trajectories to enhance spatial grounding and support CoT recovery reasoning. Extensive experiments in CARLA demonstrate that DA-Nav achieves a high success rate of 56.16% in unseen urban environments, outperforming existing State-of-The-Art (SoTA) methods while maintaining a substantially stronger recovery capability. Furthermore, without fine-tuning, DA-Nav seamlessly adapts to both quadruped and humanoid robots, enabling stable kilometer-scale closed-loop outdoor navigation in complex real world environments.
Ye Yuan, Kehan Chen, Xinqiang Yu +7
School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China · National Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences, Beijing, China +3
Embodied navigation agents built upon large reasoning models (LRMs) can handle complex, multimodal environmental input and perform grounded reasoning per step to improve sequential decision-making for long-horizon tasks. However, a critical question remains: how can the reasoning capabilities of LRMs be harnessed intelligently and efficiently for long-horizon navigation tasks? In simple scenes, agents are expected to act reflexively, while in complex ones they should engage in deliberate reasoning before acting.To achieve this, we introduce Hybrid ReasOning Navigation (HiRO-Nav) agent, the first kind of agent capable of adaptively determining whether to perform thinking at every step based on its own action entropy. Specifically, by examining how the agent's action entropy evolves over the navigation trajectories, we observed that only a small fraction of actions exhibit high entropy, and these actions often steer the agent toward novel scenes or critical objects. Furthermore, studying the relationship between action entropy and task completion (i.e., Q-value) reveals that improving high-entropy actions contributes more positively to task success.Hence, we propose a tailored training pipeline comprising hybrid supervised fine-tuning as a cold start, followed by online reinforcement learning with the proposed hybrid reasoning strategy to explicitly activate reasoning only for high-entropy actions, significantly reducing computational overhead while improving decision quality. Extensive experiments on the CHORES-S ObjectNav benchmark showcases that HiRO-Nav achieves a better trade-off between success rates and token efficiency than both dense-thinking and no-thinking baselines.
He Zhao, Yijun Yang, Zichuan Lin +2
Nanyang Technological University · Tencent · Independent Researcher