NavHarness: Towards Lifelong Embodied Navigation

Xunyi Zhao; Jian Zhou; Sihao Lin; Gengze Zhou; Zerui Li; Xinyu Yan; Jiajun Liu; Anton van den Hengel

cs.ROSep 28, 2026

NavHarness: Towards Lifelong Embodied Navigation

Authors: Xunyi Zhao, Jian Zhou, Sihao Lin, Gengze Zhou, Zerui Li, Xinyu Yan, Jiajun Liu, Anton van den Hengel, +1 more

Organizations: Adelaide University · Responsible AI Research Centre, AIML · CSIRO Data61

Abstract

Frontier models can now perform well on individual embodied navigation tasks through multi-round multimodal reasoning with simple tools. Across successive tasks, however, an agent must also rely on an evolving map and earlier search records, both of which may be incomplete or conflict with new observations. We present NavHarness, a training-free embodied harness towards lifelong navigation that makes memory processing part of the navigation loop. During navigation, its multi-round agentic session draws on maps, task records, and house knowledge, checking them against observations and recording corrections to guide its actions. NavHarness preserves this experience across fresh conversations for new tasks or recovery attempts, while outcome verification and run-end summaries support its later reuse. On GOAT-Bench, NavHarness improves s-SR over context-only independent sessions by 18.6 points with Astra and 22.6 with Opus 5. Using SLAM-estimated poses, NavHarness with GPT-6 Astra achieves state-of-the-art task success of 83.7 s-SR with 36.9 e-SR on GOAT-Bench and 85.9 s-SR on IR2R-CE. To understand these gains, we examine how experience is carried between sessions and find that structured recovery handovers outperform length-matched summaries. In extended deployments across houses, consolidation improves navigation beyond retaining maps and task records, with case studies showing how agents use earlier experience to interpret new goals, investigate unresolved questions, and resume failed searches. We suggest that progress towards lifelong navigation depends on how successive reasoning sessions build on prior experience, alongside improvements in single-task capability.

Figures & tables

Figure 1 : NavHarness execution flow. The orchestrator manages navigation sessions and persistent memory. Verify and Certify denote pre-stop verification and post-stop certification, respectively.

Method	Base Model	Type	GT pose	s-SR	SPL	e-SR
MemoryExplorer † ( Wang et al., 2026c )	Qwen2.5-VL-7B	trained	✓	46.4	28.0	–
SSMG-Nav ( Niu et al., 2026 )	Qwen-VL-Plus	zero-shot	✓	46.5	34.1	8.6
EvoMemNav ( Ge et al., 2026 )	Qwen3-VL-8B	zero-shot	✓	59.6	38.9	–
ReEXplore † ( Zhang et al., 2025a )	GPT-4o	zero-shot	✓	59.8	42.5	–
AstraNav-Memory ( Ren et al., 2025 )	Qwen2.5-VL-3B	trained	✓	62.7	56.9	–
STEGNav ( Chen et al., 2026c )	GPT-5.4-mini	zero-shot	✓	66.3	39.7	–

Table 1 : Comparison with SOTA methods on GOAT-Bench Val-Unseen. GT pose: simulator ground-truth pose. † : 278-subtask evaluation. Our scores: mean

\pm

s.d. over three seeds.

Method	Type	GT pose	s-SR	SPL	e-SR	t-nDTW
CMA ( Krantz et al., 2023 )	trained	$\times$	19	18	–	38
MAP-CMA ( Krantz et al., 2023 )	trained	✓	35	32	–	47
ETPNav ( Han et al., 2026 )	trained	✓	28	27	–	41
HNR ( Han et al., 2026 )	trained	✓	30	28	–	44
OVER-NAV ( Zhao et al., 2024 )	hybrid	✓	35	33	–	50
SeqWalker ( Han et al., 2026 )	trained	✓	36	34	–	52

Table 2 : IR2R-CE Val-Unseen. GT pose: simulator pose. Ours: three-seed mean

\pm

s.d.

A. Session policies
Configuration / intervention	s-SR	SPL	$\Delta$ s-SR	95% CI
NavHarness (full reference)	$81.5\pm 0.7$	$55.0\pm 0.6$	–	–
Independent sessions	$58.9\pm 0.9$	$34.9\pm 0.8$	$-22.6$	$[-26.3,-19.0]$
Single session	$54.9\pm 1.1$	$32.8\pm 0.9$	$-26.6$	$[-30.5,-22.8]$
Independent sessions + pre-stop verification	$62.4\pm 0.8$	$36.5\pm 0.7$	$-19.1$	$[-22.4,-16.4]$
B. Cross-task memory

Table 3 : Opus 5 comparisons on the full GOAT-Bench Val-Unseen split. Scores are mean

\pm

s.d. over three seeds (%).

\Delta

s-SR denotes the paired change from full.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Call	Arguments	Returns or effect
observe	none	the forward RGB view and remaining step budget
step	a list of actions	executes them and reports motion, collision, and budget status
get_goal	none	the current goal, as text or as the goal image
get_map	floor, zoom	the top-down occupancy map with the markers on it
mark	a place name	drops or moves a named marker at the current position
preview_path	a map point	the length of a route over known free space; nothing moves

Appendix

Table 4 : The tool registry. Only step consumes the task’s physical budget, while free calls cost no steps.

Position	1	2	3	4	5	6	7	8	9	10
s-SR (%)	72.2	72.2	83.3	83.3	77.8	82.1	73.1	85.7	81.2	85.7

Appendix

Table 5: Observed task-position success in the NavHarness Val-Unseen diagnostic run.

Path	Writer	Content
run/ledger.md	Orchestrator	Task-status index
run/tasks/ $k$ /goal.md	Orchestrator	Goal and modality
run/tasks/ $k$ /handover.md	Acting session	Task experience
run/tasks/ $k$ /lambda.md	Acting session	Recovery handover
run/tasks/ $k$ /verdict.json	Orchestrator	Judge’s assessment
house/	Consolidation session	House notes: index, overview, room notes, skills

Appendix

Table 6: Textual memory in the workspace. Each file has a designated writer.

Scope	State reconsidered
continue	No reset
attempt	Active conversation
place	Conversation and current location
room	Conversation, location, and route
floor	Conversation, location, route, and floor association
house	The same beliefs as floor, with house-level reassessment

Appendix

Table 7 : Recovery scopes. All retain the map, markers, ledger, and archive.

Benchmark	Scenes	Episodes / tours	Tasks	Tasks per sequence
GOAT-Bench	36	360 episodes	2,669	5–10
IR2R-CE	11	36 tours	1,824	3–100

Appendix

Table 8 : Evaluation scope on validation-unseen data.

Method	Type	s-SR	SPL
SenseAct-NN Monolithic ( Khanna et al., 2024 )	trained	12.3	6.8
Modular CLIP on Wheels ( Gadre et al., 2023 ; Khanna et al., 2024 )	zero-shot	16.1	10.4
VLMnav ( Goetting et al., 2024 ; Ji et al., 2025 )	zero-shot	20.1	9.6
Modular GOAT ( Khanna et al., 2024 )	zero-shot	24.9	17.2
DyNaVLM ( Ji et al., 2025 )	zero-shot	25.5	10.2
SenseAct-NN Skill Chain ( Khanna et al., 2024 )	trained	29.5	11.3

Appendix

Table 9 : Additional published GOAT-Bench Val-Unseen results. All listed methods use simulator-provided pose.

Method	Type	GT pose	s-SR	SPL	e-SR	t-nDTW
TourCMA	trained	$\times$	18	17	–	36
PoolCMA	trained	$\times$	16	15	–	36
PoolEndCMA	trained	$\times$	18	16	–	38

Appendix

Table 10 : Additional published IR2R-CE Val-Unseen results ( Krantz et al., 2023 ) .

Quantity	Limit or counting rule
Movement budget	500 executed forward or turning actions per task
Model-turn budget	200 navigation-model turns per task
Action batching	Each executed action is charged individually
Recovery allowance	At most one recovery, sharing the remaining task budgets
Fallback trigger	80 turns, without resetting either budget
Observation and memory	No movement-action charge

Appendix

Table 11 : Budget accounting for the controlled GOAT comparisons.

Metric	NavHarness	Fixed access
s-SR (%)	81.5 $\pm$ 0.7	74.5 $\pm$ 0.8
SPL	55.0 $\pm$ 0.6	49.0 $\pm$ 0.7
Mean time / task (s)	129.5	137.2
Paired $\Delta$ s-SR (pp)	$+7.0$
95% CI (pp)	$[+5.9,+8.2]$

Appendix

Table 12: Agent-directed and fixed-access historical memory on GOAT-Bench with Opus 5. Scores are mean

\pm

standard deviation over three runs. The paired difference is NavHarness minus fixed access. The reference wall time comes from the profiling runs in Appendix B.3 .

Navigator	Judge	s-SR (%)	SPL
Qwen3.8-27B	Qwen3.8-27B	71.7 $\pm$ 0.5	48.2 $\pm$ 0.4
Qwen3.8-27B	Opus 5	71.3 $\pm$ 0.4	48.5 $\pm$ 0.5
Opus 5	Opus 5	81.5 $\pm$ 0.7	55.0 $\pm$ 0.6
Opus 5	Qwen3.8-27B	81.7 $\pm$ 0.6	55.2 $\pm$ 0.5

Appendix

Table 13: Same-model and cross-model judging. Both completion checks use the listed judge, with all other roles unchanged. Scores are mean

\pm

standard deviation over three runs.

Goal	s-SR	SPL
Object category	86.5	62.4
Language description	75.2	48.8
Image	82.2	53.0

Appendix

Table 14 : The full harness by goal modality on GOAT-Bench. Percentages.

Diagnostic	Full	Independent + pre-stop verification
Tasks with a refused first stop	11	20
Rejection precision	73	75
Rescue among correctly refused stops	45	35
Rescues / all tasks	3.6	5.0
Harms / all tasks	0.7	1.1
Net local success change	$+2.9$	$+4.0$

Appendix

Table 15 : Pre-stop diagnostics. Values are percentages except the net change, which is in percentage points.

Evidence	Certified, true	Certified, false	Rejected, true	Rejected, false	Acc. (accept-all)	$\kappa$
Four stop views	78.5	5.2	4.1	12.2	90.7 (82.6)	0.67
One forward frame	74.2	10.7	6.4	8.7	82.9 (80.6)	0.40

Appendix

Table 16 : Post-stop certification outcomes and accuracy (% of claimed completions). Parentheses give accept-all accuracy.

Component	$\Delta$ s-SR	Absorbs	What shows when it is removed
Map and markers	$-12.6$	revisits and displacement	steps $+32\%$ , markers written about $2.6\times$
Ledger	$-9.7$	repeated goals	steps $+25\%$ despite retaining the map
Recovery note	$-13.0$	a stalled search	recovery success 35% instead of 56%
Recovery	$-10.4$	an unproductive attempt	requests come long before the fallback
Four-view evidence	$-2.9$	an ambiguous closure	claimed-completion $\kappa$ : 0.40 vs. 0.67 in separate runs
Certification	$-2.5$	an unsupported claim	12.2% false and 4.1% true completions rejected (share of claims)

Appendix

Table 17 : Component effects and associated diagnostics. Changes in s-SR are aggregate interventions, not effects estimated within each stress condition. The note intervention substitutes another task’s note.

Behaviour	Instruction or tool support	Relation
Report the goal, outcome, and rooms passed	Handover instruction	Requested
Report which markers were dropped	Handover instruction	Requested
Write down unsuccessful searches	Handover instruction	Requested
Move a marker	Marker tool supports re-placement	Tool-supported
Question an earlier task account	Warning that accounts may be wrong	Requested
Correct a particular marker	General permission to maintain markers	Not scripted

Appendix

Table 18 : Deployment instructions and observed behaviors. A behavior not explicitly scripted may still be supported by general memory instructions.

Explore similar work

Sep 14, 2026cs.RO

HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness

Embodied navigation requires agents to ground instructions or object goals in spatial observations and translate plans into successful execution. As multimodal large language models (MLLMs) become increasingly capable, they offer stronger support for navigation without task-specific training; however, improved semantic reasoning alone does not ensure that proposed actions remain consistent with spatial evidence, task progress, and execution outcomes. We introduce HarnessVLN, a zero-shot, training-free framework that unifies instruction-following and object-goal navigation through a shared Agent Harness. The Harness coordinates perception, memory, and execution tools through a unified interface, validating planner proposals for evidential support, geometric feasibility, and subgoal consistency before dispatch. It jointly manages hierarchical event memory and a persistent Spatiotemporal Graph to track task progress, preserve spatial evidence, and contextualize failures. Structured execution feedback updates this shared state, guiding subsequent planning, recovery, and termination. Across R2R, RxR, HM3D-v2, and HM3D-OVON, HarnessVLN achieves success rates of 59.6%, 51.4%, 76.0%, and 59.3%, respectively, outperforming prior training-free methods. Humanoid robot deployment further demonstrates its applicability to both navigation tasks in real-world environments. The project page is available at [https://agibot-harnessvln.netlify.app/].

Yang Chen, Lirong Che, Zhenyu Huang +7

Nanjing University · AGIBOT · Tsinghua University

Vision-Language Navigation Embodied Agents Navigation Agent Harness

Ask PDF

Jun 9, 2026cs.RO

AllDayNav: Lifelong Navigation via Real-World Reinforcement Learning

Lifelong embodied navigation in dynamic environments requires robots to form persistent scene understanding from fragmentary observations, which remains difficult for existing methods that rely on explicit maps or scene graphs and struggle to generalize beyond structured settings. We propose AllDayNav, a lifelong self-learning navigation framework that implicitly encodes scene dynamics into the billion-scale parameters of a large model via reinforcement learning, powered by a self-evolving multimodal memory that maintains and updates visual keyframes, semantic descriptions, and temporal context while autonomously generating open-vocabulary instructions, image goals, and structured rewards. Experiments in both synthetic and real-world environments across cross-room, cross-episode, and cross-task scenarios show that AllDayNav achieves success rates approaching

100\%

and consistently surpasses strong map-based, VLM, and RL baselines in path efficiency and robustness, demonstrating implicit, memory-driven reinforcement learning as a scalable alternative to explicit mapping for reliable lifelong navigation.

Hang Yin, Yinan Liang, Jiazhao Zhang +4

Tsinghua University, Beijing 100084, China · Galbot Robotics, Beijing, China · Peking University, Beijing 100871, China +1

Navigation Offline Reinforcement Learning Scene Reasoning

Ask PDF

May 3, 2026cs.CV

TrajRAG: Retrieving Geometric-Semantic Experience for Zero-Shot Object Navigation

Existing zero-shot Object Goal Navigation (ObjectNav) methods often exploit commonsense knowledge from large language or vision-language models to guide navigation. However, such knowledge arises from internet-scale text rather than embodied 3D experience, and episodic observations collected during navigation are typically discarded, preventing the accumulation of lifelong experience. To this end, we propose Trajectory RAG (TrajRAG), a retrieval-augmented generation framework that enhances large-model reasoning by retrieving geometric-semantic experiences. TrajRAG incrementally accumulates episodic observations from past navigation episodes. To structure these observations, we propose a topological-polar (topo-polar) trajectory representation that compactly encodes spatial layouts and semantic contexts, effectively removing redundancies in raw episodic observations. A hierarchical chunking structure further organizes similar topo-polar trajectories into unified summaries, enabling coarse-to-fine retrieval. During navigation, candidate frontiers generate multiple trajectory hypotheses that query TrajRAG for similar past trajectories, guiding large-model reasoning for waypoint selection. New experiences are continually consolidated into TrajRAG, enabling the accumulation of lifelong navigation experience. Experiments on MP3D, HM3D-v1, and HM3D-v2 show that TrajRAG effectively retrieves relevant geometric-semantic experiences and improves zero-shot ObjectNav performance.

Yiyao Wang, Sixian Zhang, Keming Zhang +3

State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing · University of Chinese Academy of Sciences, Beijing · Institute of Computing Technology, Chinese Academy of Sciences, Beijing

Vision-Language Navigation Navigation Trajectory Generation Objects

Ask PDF