AERR-Nav: Adaptive Exploration-Recovery-Reminiscing Strategy for Zero-Shot Object Navigation
Organizations: Hong Kong Polytechnic University · Institute of automation, Chinese Academy of Sciences · Hong Kong University of Science and Technology (Guangzhou)
Abstract
Zero-Shot Object Navigation (ZSON) in unknown multi-floor environments presents a significant challenge. Recent methods, mostly based on semantic value greedy waypoint selection, spatial topology-enhanced memory, and Multimodal Large Language Model (MLLM) as a decision-making framework, have led to improvements. However, these architectures struggle to balance exploration and exploitation for ZSON when encountering unseen environments, especially in multi-floor settings, such as robots getting stuck at narrow intersections, endlessly wandering, or failing to find stair entrances. To overcome these challenges, we propose AERR-Nav, a Zero-Shot Object Navigation framework that dynamically adjusts its state based on the robot's environment. Specifically, AERR-Nav has the following two key advantages: (1) An Adaptive Exploration-Recovery-Reminiscing Strategy, enables robots to dynamically transition between three states, facilitating specialized responses to diverse navigation scenarios. (2) An Adaptive Exploration State featuring Fast and Slow-Thinking modes helps robots better balance exploration, exploitation, and higher-level reasoning based on evolving environmental information. Extensive experiments on the HM3D and MP3D benchmarks demonstrate that our AERR-Nav achieves state-of-the-art performance among zero-shot methods. Comprehensive ablation studies further validate the efficacy of our proposed strategy and modules.
Figures & tables
| Method | Venue | Zero-shot | HM3D | MP3D | ||
|---|---|---|---|---|---|---|
| SR | SPL | SR | SPL | |||
| ZSON [ 23 ] | NeurIPS’22 | ✓ | 25.5 | 12.6 | 15.3 | 4.8 |
| ESC [ 49 ] | ICML’23 | ✓ | 39.2 | 22.3 | 28.7 | 14.2 |
| L3MVN [ 40 ] | IROS’23 | ✓ | 50.4 | 23.1 | 34.9 | 14.5 |
| PSL [ 31 ] | ECCV’24 | ✓ | 42.4 | 19.2 | 18.9 | 6.9 |
| VLFM [ 38 ] | ICRA’24 | ✓ | 52.5 | 30.4 | 36.4 | 17.5 |
| Method | Intra-Floor Scenarios | Inter-Floor Scenarios | All Episodes | |||
|---|---|---|---|---|---|---|
| SR | SPL | SR | SPL | SR | SPL | |
| VLFM [ 38 ] | 64.6 | 37.3 | 0.4 | 0.1 | 52.8 | 30.5 |
| MFNP [ 43 ] | 68.4 | 30.5 | 13.4 | 9.8 | 58.3 | 26.7 |
| ASCENT [ 11 ] | 72.6 | 37.7 | 33.3 | 14.9 | 65.4 | 33.5 |
| AERR-Nav | 75.8 | 38.1 | 62.5 | 27.3 | 72.3 | 35.6 |
| # | Strategy Combinations | HM3D | MP3D | ||||
|---|---|---|---|---|---|---|---|
| EXP | REC | REM | SR | SPL | SR | SPL | |
| 1 | ✓ | 68.9 | 34.5 | 45.1 | 17.2 | ||
| 2 | ✓ | ✓ | 70.1 | 35.2 | 46.3 | 17.4 | |
| 3 | ✓ | ✓ | 70.5 | 34.9 | 45.9 | 16.8 | |
| 4 | ✓ | ✓ | ✓ | 72.3 | 35.6 | 47.9 | 18.7 |
| Exploration Strategy | MLLM | HM3D | MP3D | ||
|---|---|---|---|---|---|
| SR | SPL | SR | SPL | ||
| (a) Semantic ( ) | 67.2 | 29.3 | 41.1 | 14.8 | |
| (b) Uncertainty ( ) | 66.8 | 31.1 | 41.8 | 15.5 | |
| (c) Semantic + Uncertainty ( ) | 68.5 | 33.4 | 43.3 | 16.3 | |
| (d) Semantic + Uncertainty ( ) | ✓ | 69.8 | 34.2 | 45.4 | 17.1 |
| (e) Semantic + Uncertainty (Dynamic ) | 69.3 | 33.8 | 45.9 | 17.3 | |
Robot Navigation
Robot navigation studies how robots perceive, localize, plan, and move safely toward goals in static or changing environments.
See all 554 papers