PreAct-Nav: Agentic Reasoning Before Action for Urban Navigation

Jing Xie; Shouwei Ruan; Yubin Wang; Yuxiang Zhang; Junwei Yang; Songchang Jin; Dianxi Shi

cs.ROOct 4, 2026

PreAct-Nav: Agentic Reasoning Before Action for Urban Navigation

Authors: Jing Xie, Shouwei Ruan, Yubin Wang, Yuxiang Zhang, Junwei Yang, Songchang Jin, Dianxi Shi

Organizations: MoE Key Lab of Artificial Intelligence, Institute of AI, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Beihang University, Beijing, China · The Hong Kong University of Science and Technology, Hong Kong, China · Tsinghua University, Beijing, China ·

Abstract

Urban navigation requires embodied agents to pursue long-horizon goals through local decisions based on egocentric observations. However, existing agentic navigation methods often struggle to translate distant goals into coherent local decisions in large-scale physical environments. Their reliance on linguistic reasoning over transient observations or limited history constrains anticipation of the consequences of actions and future conditions, despite the importance of such foresight for navigating long and complex urban routes. To bridge this gap, we propose PreAct-Nav, an agentic navigation framework that equips frozen policies with anticipatory reasoning for robust urban navigation. Our central idea is to anchor local decisions in persistent medium-horizon subgoals, assess the consequences of predicted actions before execution, and continually update the reasoning context using actual outcomes. At its core, a navigation memory module maintains the active subgoal and relevant experience across decisions, translating distant goals into actionable intermediate objectives. We further introduce a predictive world sandbox that uses an action-conditioned world model (AC-WM) to forecast world dynamics conditioned on candidate movements. A vision-language model (VLM) reasoner interprets these predictions under the current subgoal to retain or revise actions. After execution, real observations are used to assess outcomes, correct inconsistent assumptions, and update memory to continue or reformulate the subgoal. Extensive evaluations demonstrate that the proposed PreAct-Nav improves action selection through memory updates and visual prediction, with more pronounced gains on longer routes and routes with more turns.

Figures & tables

Figure 1: Reasoning before action with PreAct-Nav. Navigating across an intersection requires local decisions to serve a long-horizon goal. Compared with reasoning from observed history and language-based future thoughts, PreAct-Nav maintains medium-horizon subgoals to decompose long-horizon navigation tasks and uses action-conditioned future visual cues to preview, review, and revise local actions before execution. Real feedback updates memory for subsequent decisions.

Figure 2: Overview of PreAct-Nav. A shared VLM coordinates Navigation Memory and the Predictive Sandbox through Action Preview, Action Review, Action Execute, and Memory Update. Memory provides observed experience and the current subgoal; the sandbox supplies predicted visual cues. Revised actions return to preview. After execution, real observations are checked against expected cues to update subgoal status and open-question progress.

Reasoner: Qwen3.8-27B
	Test-seen						Test-unseen
Method	SR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	nDTW $\uparrow$	SDTW $\uparrow$	OSR $\uparrow$	SR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	nDTW $\uparrow$	SDTW $\uparrow$	OSR $\uparrow$
UrbanNav	82.79	81.16	3.3596	94.10	81.73	91.91	78.14	74.73	5.9659	87.73	76.13	84.92
Direct VLM	83.13	80.41	3.3192	94.31	82.02	92.49	79.05	75.58	5.9187	88.02	76.99	85.63
NavGPT	83.23	80.54	3.3090	94.35	82.12	92.49	79.11	75.65	5.9017	88.09	77.06	85.67
Open-Nav	83.15	80.45	3.3175	94.31	82.04	92.44	79.24	75.77	5.8964	88.11	77.18	85.74

Table 1: Comparison on UrbanNav. All methods use the UrbanNav policy, with rows grouped by VLM reasoner. NE is in metres; other metrics are percentages. Best results are bold.

Method	SR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	nDTW $\uparrow$	SDTW $\uparrow$	OSR $\uparrow$
Direct VLM	43.51	43.11	18.28	37.44	32.33	43.51
NavGPT	73.38	70.19	10.12	51.00	48.32	73.38
Open-Nav	64.29	60.98	12.21	45.08	41.41	64.29
EvoNav	63.64	61.05	11.97	46.45	42.34	63.64
Ours	79.22	78.94	8.31	68.77	65.85	79.22

Table 2: Comparison on CityWalker. All methods use the frozen CityWalker policy and Qwen reasoner. NE is in metres; other metrics are percentages. Baseline adaptations and evaluation settings are detailed in Appendix D.5 .

Policy	SR $\uparrow$		SPL $\uparrow$		NE $\downarrow$		nDTW $\uparrow$		SDTW $\uparrow$		OSR $\uparrow$
UrbanNav	78.14		74.73		5.9659		87.73		76.13		84.92
+ harness	83.38	(+5.24)	77.87	(+3.13)	5.3353	(-0.6306)	90.47	(+2.74)	81.30	(+5.17)	90.12	(+5.20)
NoMaD	76.33		69.05		7.2776		82.73		73.55		81.20
+ harness	79.11	(+2.78)	71.81	(+2.76)	7.2727	(-0.0050)	82.74	(+0.01)	76.13	(+2.58)	81.20	(0.00)
OmniVLA	82.65		78.06		5.8666		87.77		79.98		86.90
+ harness	86.23	(+3.58)	81.44	(+3.39)	5.7795	(-0.0871)	88.36	(+0.58)	83.33	(+3.35)	88.07	(+1.17)

Table 3: Comparison across navigation policies on test-unseen. All pairs use Qwen. Green parentheses show changes from the corresponding frozen policy: percentage points for rate metrics and metres for NE. Changes are computed before rounding.

Reasoning Harness	Navigation Memory Tool	Predictive Sandbox Tool	SR $\uparrow$	SPL $\uparrow$	NE $\downarrow$	nDTW $\uparrow$	SDTW $\uparrow$	OSR $\uparrow$
$\times$	$\times$	$\times$	78.14	74.73	5.9659	87.73	76.13	84.92
$\checkmark$	$\times$	$\times$	80.56	76.74	5.6410	89.60	78.65	88.37
$\checkmark$	$\checkmark$	$\times$	82.37	76.91	5.3986	90.34	80.34	89.90
$\checkmark$	$\times$	$\checkmark$	81.48	77.24	5.4791	89.87	79.47	88.89
$\checkmark$	$\checkmark$	$\checkmark$	83.38	77.87	5.3353	90.47	81.30	90.12

Table 4: Component ablation on test-unseen.

\checkmark

\times

indicate enabled/disabled components. The first row is the UrbanNav policy. Without either tool, the Reasoning Harness reviews and revises policy actions. NE is in metres; other metrics are percentages.

Figure 4: Action revision and memory update. (a) Predicted cues prompt a revision from the storefront toward the sidewalk, supported by the next real observation. (b) Pedestrians remain in the passage, prompting a new subgoal to pass on the right.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Real-world navigation case. A robot follows the building exterior toward a delivery locker. The panels connect the current subgoal, visual previews, action revision, and memory update.

Placeholder	Value supplied to the operation
destination_view	The unmodified destination-view description $g$ .
observed_experience	Recent executed-action records: source step, action in its source frame, measured progress, expected cues, and observed feedback.
subgoal	Current objective, visual anchor, completion cue, open question, semantic status, and runtime metadata; null before initialization.
current_action_m	The proposed waypoint sequence, expressed in the current frame.
endpoint_x_m , endpoint_y_m	Coordinates of the final waypoint, formatted to three decimal places.
earlier_reviews	Ordered action indices, waypoint sequences, and structured reviews; an empty list at the start of a decision.

Appendix

Table 5: Runtime substitutions in the prompt templates. Placeholders are replaced by typed values before serialization; they are not additional model instructions.

Observed feedback	Subgoal status	Next operation
SUPPORTED	ACTIVE	Retain the objective and append evidence of compatible progress.
UNRESOLVED	ACTIVE	Retain the objective and unresolved question.
ACHIEVED	ACHIEVED	Close the completed subgoal; establish the next objective.
CONTRADICTED	CONTRADICTED	Reconsider the objective using the conflicting observation.

Appendix

Table 6: From observed feedback to memory state. Feedback is generated by the VLM; storage and semantic-status updates are performed by the harness.

Displayed field	Runtime field	Role in the method
Objective	intent , objective	Medium-horizon intent $z_{t}$ in Equation 2 .
Visual cues	anchor , completion	Observable anchors and progress/completion cues $c_{t}$ .
Open question	question	Unresolved question $q_{t}$ in memory; review can specify a query about the current action.
Action assessment	review verdict	$\omega_{t}^{j}$ in Equation 8 .
Preview useful	query_useful	$\kappa_{t}^{j}$ in Equation 8 .
Prediction quality	prediction_quality	Video assessment $\chi_{t}^{j}$ .

Appendix

Table 7: Prompt outputs in the method and qualitative figures. Descriptive figure labels expose the semantics of the serialized fields without adding new model outputs.

Operation	Principal inputs	Returned information
Subgoal establishment	Destination, observed images, experience, previous objective	Route intent, anchor, objective, completion cue, open question.
Action Preview / Review	Current action, context, earlier reviews, optional predicted rollout	Action assessment, query usefulness, prediction quality, rationale, expected cues.
Action revision	Current action, its review, shared context, available images	Revised path or abstention, confidence, grounding, rationale, expected cues.
Memory Update	Subgoal, executed movement, expected cues, before/after real views	Observed-feedback verdict and rationale.

Appendix

Table 8: Information exchanged through the harness. Operations use the same vision–language reasoner. The table lists the inputs and outputs of each prompted operation.

Group	Episodes	SR	$\Delta$ SR interval	SPL	$\Delta$ SPL interval
Short	3,657	94.26 / 94.56	$[0.10,0.51]$	87.65 / 87.91	$[0.08,0.47]$
Long	977	37.87 / 41.56	$[1.82,5.57]$	36.71 / 40.26	$[1.72,5.39]$
Low + medium turn	3,089	94.92 / 95.40	$[0.27,0.72]$	88.48 / 88.92	$[0.24,0.66]$
High turn	1,545	57.28 / 59.35	$[1.04,3.07]$	53.77 / 55.77	$[0.98,2.98]$

Appendix

Table 9: Route-group results. SR/SPL show harness with Navigation Memory Tool / full method from Table 3 . Each partition covers all 4,634 episodes. Intervals describe the paired gain in percentage points.

Changed input	Measurement	Pairs	Result
Updated memory	Policy-action assessment changed	1,501	13.3%
Updated memory	Retain/revise decision changed	1,501	8.9%
Predicted video	States requesting a revision	525	$147\to 214$
Predicted video	Endpoint change when both revise	133	0.28 m

Appendix

Table 10: Paired effects on reasoning. Each comparison holds observed images, destination, and policy proposal fixed. Only the memory update or predicted-video input changes. These are state-level probes, separate from the navigation rollouts.

Measurement	Pairs	Estimate	95% interval
Memory: changed policy-action assessment	1,501	13.26	$[11.90,14.70]$
Memory: changed retain/revise request	1,501	8.93	$[7.62,10.18]$
Revision rate without prediction	525	28.00	$[21.56,35.44]$
Revision rate with prediction	525	40.76	$[34.63,47.30]$
Paired revision-rate difference	525	12.76	$[8.90,16.80]$
Endpoint change when both revise	133	0.278	$[0.215,0.342]$

Appendix

Table 11: Paired reasoning measurements. Rates are percentages; rate differences are percentage points. The last row is in metres.

Mean (s)	Median (s)	P90 (s)	Peak GPU memory (GiB)
11.37	11.35	11.47	34.06

Appendix

Table 12: Predictive sandbox cost on one NVIDIA H200 GPU. Statistics cover 1,700 rollouts, each containing 60 predicted frames. P90 denotes the 90th percentile.

Operation	Observed / predicted frames	Mean (s)	Median (s)	P90 (s)
Subgoal establishment	3 / 0	5.76	5.21	6.49
Action Preview	3 / 0	5.06	5.04	5.41
Action Review	3 / 4	5.19	5.05	5.65
Action Revision	3 / 0	8.65	9.82	10.77
Memory Update	2 / 0	2.57	2.35	2.91

Appendix

Table 13: Per-request latency on one NVIDIA H200 GPU at batch size one. Predicted inputs use frame indices 10, 20, 40, and 60, with index 0 denoting the input observation. Each row times only the named operation; video generation is reported in Table 12 .

Embodied Navigation

Embodied navigation studies how robots perceive, map, and move through environments to reach goals and follow instructions.

See all 131 papers

NavHarness: Adaptive Goals for Agentic Vision-Language Navigation
Sep 30, 2026Haoxiang Shi, Zaijing Li, Muhe Ding +3Vision-Language Navigation Multimodal Agents
Harbin Institute of Technology (Shenzhen) · Pengcheng Laboratory
DA-Nav: Direction-Aware City-Scale Vision-Language Navigation
Jul 13, 2026Ye Yuan, Kehan Chen, Xinqiang Yu +7Embodied Navigation Vision-Language Navigation
School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China · National Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences, Beijing, China +3
HiRO-Nav: Hybrid ReasOning Enables Efficient Embodied Navigation
Apr 9, 2026He Zhao, Yijun Yang, Zichuan Lin +2Adaptive Reasoning in Language Models Embodied Navigation
Nanyang Technological University · Tencent · Independent Researcher

Sep 30, 2026J/K move · Enter open · S save

NavHarness: Adaptive Goals for Agentic Vision-Language Navigation

Haoxiang Shi, Zaijing Li, Muhe Ding, Xiang Deng+2

Harbin Institute of Technology (Shenzhen) · Pengcheng Laboratory

Vision-Language Navigation Multimodal Agents

Open paper AskPDF arXiv

Vision-Language Navigation (VLN) requires embodied agents to generate actions based on instructions and observations. General-purpose multimodal agents offer a promising basis for this task, but selecting plausible local actions does not ensure that execution remains consistent with the intended route, particularly in long-horizon tasks. Moreover, the accumulated interaction history increases the input required for subsequent decisions, resulting in a significant inference overhead. To this end, we introduce \method, an Agentic VLN framework that includes a Goal Agent that sets adaptive goals for local actions, a Verify Agent that dynamically verifies whether a goal has been completed, a Memory Agent for multimodal context compression, and a Visuomotor Agent to execute adaptive goals. Specifically, the Goal Agent formulates adaptive goals based on the instruction, current observation, and execution history. Then the Visuomotor Agent executes navigation actions to achieve each goal, while the Verify Agent uses a goal-specific verification question to dynamically assess whether the observed outcomes satisfy the intended completion condition. Verified goal completion then marks a boundary for the Memory Agent to compress the corresponding multimodal interaction history while preserving information needed for subsequent navigation. We evaluate navigation on R2R-CE and RxR-CE, examine framework variants across three model backbones, and study context evolution during execution. For Real-World evaluation, \method achieves 83.3% success and 1.51,m navigation error across eight challenging routes evaluated three times each.

Abstract

Embodied Navigation

Explore similar work

NavHarness: Adaptive Goals for Agentic Vision-Language Navigation

DA-Nav: Direction-Aware City-Scale Vision-Language Navigation

HiRO-Nav: Hybrid ReasOning Enables Efficient Embodied Navigation