Giving embodied agents the freedom to act autonomously, initiate interactions, and understand their surroundings through experience is a step toward bringing them to life in the physical world. Realizing this vision requires action, interaction, and memory to work together, so that each encounter can inform current decisions and contribute experience for future tasks. We present EvoLife, a unified navigation harness that enables a multimodal embodied agent to choose how to move, when to ask, and what experience to carry forward. Its hybrid action space supports waypoint navigation, local discrete adjustments, pixel-based navigation, and object-directed navigation. Active interaction allows the agent to seek clarification or assistance when it encounters ambiguity or difficulty, while persistent spatial memory binds observations and dialogue to places, retaining useful experience to guide subsequent visits. We evaluate eight benchmarks across four task families: vision-language navigation, object navigation, interactive instance navigation, and continuous multi-task navigation. With an average of only 1.87 active interactions per episode, our method achieves a first-visit success rate (SR) of 100% across four 100-episode sets spanning vision-language navigation and object navigation. Complete-sequence SR reaches 98% on 50 LHPR-VLN sequences and 94% on 50 GOAT-Bench sequences. In the R2R revisit setting, success weighted by path length (SPL) increases from 82.23% to 93.31% with retained experience. Multi-day physical navigation further demonstrates exploration, proactive help-seeking, and revisiting remembered places. Together, these results advance the vision of bringing embodied agents to life through autonomous action, proactive interaction, and the accumulation and reuse of experience.
Figures & tables
Figure 1: EvoLife takes a step toward bringing embodied agents to life through autonomous action, proactive interaction, and the accumulation of experience. The left panel presents four task families: vision-language navigation [ 3 , 27 , 26 ] , object navigation [ 42 , 65 , 69 ] , interactive instance navigation [ 21 , 48 ] , and continuous multi-task navigation [ 47 , 24 ] . The center shows the unified harness, connecting a hybrid action space, active interaction, and persistent spatial memory. The right panel reports success rate (SR) across eight simulation benchmarks and illustrates multi-day physical navigation, demonstrating the agent’s ability to respond to environmental changes, proactively seek assistance, and revisit remembered places.
Figure 2: Overview of the unified EvoLife framework. A multimodal agent integrates hybrid actions, active interaction, and persistent memory within a unified decision loop. The agent adaptively selects navigation actions based on current observations and execution feedback, actively seeks external guidance to resolve navigation uncertainties, and retains spatially grounded experience for future tasks. Ci denotes a geometric candidate waypoint. Pi denotes a remembered place associated with its recorded pose and supporting evidence. Di denotes the i -th tool-level decision, encompassing perception, navigation, interaction, and memory operations. Qi and Ai denote the agent-generated question and the corresponding guide answer in the i -th interaction, respectively. The illustrated example demonstrates how hybrid actions, interaction, and memory jointly support lifelong embodied navigation.
Tool
Agent-selected input
Returned evidence and function
Perception
observe observe_panorama_candidates
Observation request
Return current four-view or panoramic observations and geometric waypoint candidates.
Return processed images and their processing records.
localize_object verify_object
View and box/point prompts; image version or object ID
Estimate object geometry from RGB-D observations and return visual overlays and depth-consistency evidence.
Hybrid Action Space
Table 1: Principal EvoLife tools organized into four functional groups. The agent autonomously selects tools and their parameters.
Figure 3: Online guide response generation. The navigation agent actively generates a question, while an independent multimodal guide generates a natural-language response based on the navigation instruction, current observations, target information, and reference route. The figure illustrates the current and target views, reference route, and spatial relationships between the agent and the destination. Only the generated natural-language response is returned to the navigation agent, which independently makes subsequent decisions. This process simulates natural human-to-human interaction, providing navigation guidance through directions, landmarks, and spatial relationships.
Benchmark
Task input
Episodes
Subtasks
Success criterion
Metrics
R2R-CE
Route instruction
100
–
STOP within 3m of the destination
SR, OSR, SPL, NE
RxR-CE
English route instruction
100
–
STOP within 3m of the destination
SR, SPL, NE, nDTW
HM3D-OVON
Open-vocabulary object category; Val-Unseen
100
–
STOP with dg<0.25m to an accepted goal viewpoint
SR, SPL, NE
HM3D v2
Object category; validation split
100
–
STOP with dg<0.1m to an accepted goal viewpoint
SR, SPL, NE
VL-LN
Category-only underspecified language goal; IIGN
100
–
STOP with dg<0.25m to an accepted target viewpoint
SR, SPL, NQ
CoIN
Category-only language goal; hidden target instance; Unseen categories
100
–
STOP with dg<0.25m to an accepted target-instance viewpoint
SR, SPL, NQ
Table 2: Evaluation protocols and sample sizes across eight navigation benchmarks. An episode denotes one complete task or continuous sequence.
Method
VL-LN
CoIN
LHPR-VLN
GOAT-Bench
SR ↑
SPL ↑
NQ ↓
SR ↑
SPL ↑
NQ ↓
SR ↑
ISR ↑
CSR ↑
CGT ↑
s-SR ↑
SPL ↑
e-SR ↑
VLFM [ 68 ]
8.8
5.6
–
0.2
0.1
–
–
–
MTU3D [ 85 ]
12.8
5.4
–
8.7
3.3
–
–
–
AIUTA [ 48 ]
4.0
2.2
5.42
6.7
2.3
1.13
–
–
ProCompNav [ 28 ]
–
17.0
6.2
2.30
–
–
InternVLA-N1 [ 23 , 84 ]
25.4
16.2
1.98
17.2
8.2
2.61
–
–
Table 4: Comparison on VL-LN, CoIN, LHPR-VLN, and GOAT-Bench.
Dataset
Model
Interaction
Memory
OSR ↑
SR ↑
SPL ↑
NE ↓
Steps ↓
ND ↓
NQ ↓
Time ↓
Cost ↓
(%)
(%)
(%)
(m)
task / sub.
task / sub.
(s)
($/ep.)
R2R
GPT-6 Astra
×
×
89
86
63.7
2.86
94.06
18.71
0.00
170.17
1.22
R2R
GPT-6 Astra
✓
×
100
100
82.2
0.95
70.12
15.57
1.66
164.08
1.06
R2R
GPT-6 Astra
×
✓
100
100
93.3
0.95
54.43
12.31
0.00
107.41
1.11
R2R
DeepSeek V4.1-Flash
×
×
68
55
39.5
5.23
136.64
43.02
0.00
361.21
0.03
R2R
DeepSeek V4.1-Flash
✓
×
99
98
70.8
1.21
118.44
44.72
3.21
459.12
0.03
Table 5: Navigation performance and efficiency across different benchmarks and interaction-memory settings. In the Interaction column, ✓ and × indicate whether active interaction is enabled or disabled, respectively. In the Memory column, × denotes a first-visit run in an unseen environment, while ✓ denotes a revisit run that reuses memory retained from the first visit. Memory accumulated during the current run remains available in both settings. ND denotes the mean number of completed tool-level decisions; NQ denotes the mean number of active interaction exchanges. For LHPR-VLN and GOAT-Bench, ND and NQ are reported as complete task / subtask, where one complete task is a continuous sequence. Subtask averages are computed by dividing the total counts by the total number of subtasks. Steps, time, and cost are averaged per episode, or per complete sequence for the continuous benchmarks.
Model
OSR ↑ (%)
SR ↑ (%)
SPL ↑ (%)
NE ↓ (m)
Steps ↓
ND ↓
NQ ↓
Time ↓ (s)
Cost ↓ ($/ep.)
GPT-6 Astra (ultra)
100
100
82.2
0.95
70.12
15.57
1.66
164.08
1.06
Qwen3.8-Max (xhigh)
93
91
70.2
1.71
104.90
72.98
1.59
906.20
0.46
DeepSeek V4.1-Flash (max)
99
98
70.8
1.21
118.44
44.72
3.21
459.12
0.03
Table 6: Backbone comparison of EvoLife on R2R-CE. ND denotes the mean number of completed tool-level decisions; NQ denotes the mean number of active interaction exchanges.
Figure 4: Qualitative example of active interaction and persistent memory in R2R-CE. The agent actively asks whether a nearby wooden door is the intended destination or whether it should continue through a narrow archway. Based on the guide’s response, the agent adjusts its navigation direction, proceeds through the archway, and records its entrance as place P2. Q2 and A2 denote the agent’s second interaction question and the guide’s corresponding answer, respectively, while P2 denotes the stored place record.
Figure 5: Failure case analysis on GOAT-Bench. Left: the agent successfully completes the picture-finding subtask. Right: the agent subsequently identifies the bookcase with diamond-shaped compartments through active interaction, but stops 0.73 m from the reference target, exceeding the required 0.25 m success threshold. Orange dots on the maps indicate the agent’s position. Pi denotes a saved place in persistent memory. Ci labels candidate waypoint i .
Figure 6: Three consecutive real-world robot navigation tasks across two days. In Task 1, the robot navigates from room 625 to the elevator on the opposite side of the floor, recording key locations along the route. In Task 2, the robot returns to room 625 using persistent memory and actively requests human assistance when encountering a closed door. In Task 3, the robot identifies and restores the matching saved map on the following morning and navigates to the glass office visited the previous day. The three trajectories are visualized in a shared coordinate frame, demonstrating active interaction, adaptation to environmental changes, and cross-day memory reuse.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Hybrid navigation on R2R-CE. The instruction requires passing through a bathroom to reach a gym. The observed map expands as the agent moves from the initial hallway through a bright workroom and the tiled bathroom. step and navigate_candidate requests provide complementary ways to expose and traverse the connecting doorways. The final view contains exercise equipment, and an explicit stop submits the completed task. Orange dots on the maps indicate the agent’s position after executing the corresponding decision. Pi denotes a saved place in persistent memory. In the panoramas, Ci labels candidate waypoint i , and green markers indicate the selected waypoint or pixel. Di denotes the i -th tool-level decision.
Figure S2: Pixel navigation and landmark recording on RxR-CE. The agent follows the passage between a couch and kitchen island, identifies the instructed right branch, and enters the room containing the sink. Pixel selections express local floor destinations, while a remember_place request records the eye-chart junction and its distinguishing geometry. The final observation shows the sink directly ahead before the task is submitted. Orange dots on the maps indicate the agent’s position after executing the corresponding decision. Pi denotes a saved place in persistent memory. In the panoramas, green markers indicate the selected waypoint or pixel. Di denotes the i -th tool-level decision.
Figure S3: Object grounding on HM3D v2. Starting from a living area, the agent approaches the hallway and bathroom entrance through visible free floor. Once the potted plant beside the vanity is visible, localize_object selects its image region with a bounding box. The subsequent approach ends beside the bathroom sink, where the agent submits an explicit stopping decision. Orange dots on the maps indicate the agent’s position after executing the corresponding decision. Pi denotes a saved place in persistent memory. In the panoramas, Ci labels candidate waypoint i , and green markers indicate the selected waypoint or pixel. Di denotes the i -th tool-level decision.
Figure S4: Active guidance during open-vocabulary object navigation. After entering the dining area, the agent asks where to look for a decorative plant. The guide identifies a tall white pot beneath artwork beside the blue fireplace wall and describes an aisle around the dining table. A subsequent candidate request follows that aisle, and the final view shows the agent beside the indicated plant. Orange dots on the maps indicate the agent’s position after executing the corresponding decision. Pi denotes a saved place in persistent memory. In the panoramas, Ci labels candidate waypoint i , and green markers indicate the selected waypoint or pixel. Di denotes the i -th tool-level decision.
Figure S5: Instance disambiguation on VL-LN. The category-level request to find a table is clarified through a description of a pale stone rectangular top on a central pedestal beneath an arched recess. The agent follows the narrow aisle, selects a point on the observed table for localization, and uses local movement to adjust its position. Its final summary refers to the same structural and spatial attributes. Orange dots on the maps indicate the agent’s position after executing the corresponding decision. Pi denotes a saved place in persistent memory. In the panoramas, Ci labels candidate waypoint i , and green markers indicate the selected waypoint or pixel. Di denotes the i -th tool-level decision.
Figure S6: Iterative clarification on CoIN. The agent first distinguishes the requested dresser from one beside a turquoise bed. A later question reveals that the furniture in a small pink-bed bedroom is also incorrect. Further guidance identifies a blue-runner passage through the kitchen. The final question verifies the long white dresser beneath a wide mirror in the larger bedroom. Orange dots on the maps indicate the agent’s position after executing the corresponding decision. Pi denotes a saved place in persistent memory. In the panoramas, Ci labels candidate waypoint i , and green markers indicate the selected waypoint or pixel. Di denotes the i -th tool-level decision.
Figure S7: Continuous two-stage navigation on LHPR-VLN. The agent first reaches the dining-room tray and submits that stage with stop . It then calls navigate_place with the saved start location to return toward the initial stair landing. A pixel request selects the bathroom opening, and the recorded execution continues with candidate movements before the final stop at the wash cabinet. Both destinations are visited in one continuous sequence. Orange dots on the maps indicate the agent’s position after executing the corresponding decision. Pi denotes a saved place in persistent memory. In the panoramas, green markers indicate the selected pixel. Di denotes the i -th tool-level decision.
Figure S8: Selected stages of a seven-subtask GOAT-Bench sequence. The goals combine language descriptions, reference images, and an object category. The execution panels show the initial mirror search, completion of the image-specified plant subtask, and completion of the final chest subtask. The observed map retains exploration across these stages. Only subtasks 1, 2, and 7 are shown in the execution panels. Orange dots on the maps indicate the agent’s position after executing the corresponding decision. Pi denotes a saved place in persistent memory. Di denotes the i -th tool-level decision.
Embodied navigation agents powered by large language models have shown strong performance on individual tasks but struggle to continually acquire new navigation skills, which suffer from catastrophic forgetting. We formalize this challenge as lifelong embodied navigation learning (LENL), where an agent is required to adapt to a sequence of navigation tasks spanning multiple scenes and diverse user instruction styles, while retaining previously learned knowledge. To tackle this problem, we propose Uni-Walker, a lifelong embodied navigation framework that decouples navigation knowledge into task-shared and task-specific components with Decoder Extension LoRA (DE-LoRA). To learn the shared knowledge, we design a knowledge inheritance strategy and an experts co-activation strategy to facilitate shared knowledge transfer and refinement across multiple navigation tasks. To learn the specific knowledge, we propose an expert subspace orthogonality constraint together and a navigation-specific chain-of-thought reasoning mechanism to capture specific knowledge and enhance instruction-style understanding. Extensive experiments demonstrate the superiority of Uni-Walker for building universal navigation agents with lifelong learning.
Xudong Wang, Jiahua Dong, Baichen Liu +3
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Mohamed bin Zayed University of Artificial Intelligence
Frontier models can now perform well on individual embodied navigation tasks through multi-round multimodal reasoning with simple tools. Across successive tasks, however, an agent must also rely on an evolving map and earlier search records, both of which may be incomplete or conflict with new observations. We present NavHarness, a training-free embodied harness towards lifelong navigation that makes memory processing part of the navigation loop. During navigation, its multi-round agentic session draws on maps, task records, and house knowledge, checking them against observations and recording corrections to guide its actions. NavHarness preserves this experience across fresh conversations for new tasks or recovery attempts, while outcome verification and run-end summaries support its later reuse. On GOAT-Bench, NavHarness improves s-SR over context-only independent sessions by 18.6 points with Astra and 22.6 with Opus 5. Using SLAM-estimated poses, NavHarness with GPT-6 Astra achieves state-of-the-art task success of 83.7 s-SR with 36.9 e-SR on GOAT-Bench and 85.9 s-SR on IR2R-CE. To understand these gains, we examine how experience is carried between sessions and find that structured recovery handovers outperform length-matched summaries. In extended deployments across houses, consolidation improves navigation beyond retaining maps and task records, with case studies showing how agents use earlier experience to interpret new goals, investigate unresolved questions, and resume failed searches. We suggest that progress towards lifelong navigation depends on how successive reasoning sessions build on prior experience, alongside improvements in single-task capability.
Xunyi Zhao, Jian Zhou, Sihao Lin +6
Adelaide University · Responsible AI Research Centre, AIML · CSIRO Data61
Lifelong navigation (LN) requires an embodied agent to solve a sequence of navigation subtasks in the same environment. Since solving each subtask from scratch incurs redundant exploration, an LN agent must consolidate experience from earlier stages and reuse it in later stages, often through persistent scene representations such as scene graphs or visual snapshots. However, existing approaches typically assume a stationary environment, whereas in real-world LN settings, human activities can cause the environment to evolve. With the stationary assumption violated, existing methods may fuse outdated prior observations with new observations, yet current benchmarks cannot reveal this failure mode. In this paper, we present EvoNav-Bench, which extends the GOAT-Bench style LN formulation in the context of evolving environments. Built on the ProcTHOR framework, EvoNav-Bench introduces environment modifications between navigation tasks, making prior experience useful but not fully reliable. This design enables controlled evaluation of how environment evolution affects LN agents that reuse prior scene observations. Using EvoNav-Bench, we benchmark three recent methods that build and reuse scene representations for navigation. We also compare three simple heuristic strategies for handling environment evolution: Frontier-Update, Fail-then-Update, and Stage-Reset. Our results show that existing methods are brittle under environment evolution, while the heuristic strategies enable a controlled analysis of how agents can adapt to scene changes and mitigate their impact.
Xilin Wang, Guoxi Zhang, Hongming Xu +3
State Key Laboratory of General Artificial Intelligence, BIGAI · 2Tsinghua University