AERR-Nav: Adaptive Exploration-Recovery-Reminiscing Strategy for Zero-Shot Object Navigation

Jingzhi Huang; Junkai Huang; Haoyang Yang; Haoang Li; Yi Wang

cs.ROMar 18, 2026

AERR-Nav: Adaptive Exploration-Recovery-Reminiscing Strategy for Zero-Shot Object Navigation

Authors: Jingzhi Huang, Junkai Huang, Haoyang Yang, Haoang Li, Yi Wang

Organizations: Hong Kong Polytechnic University · Institute of automation, Chinese Academy of Sciences · Hong Kong University of Science and Technology (Guangzhou)

Abstract

Zero-Shot Object Navigation (ZSON) in unknown multi-floor environments presents a significant challenge. Recent methods, mostly based on semantic value greedy waypoint selection, spatial topology-enhanced memory, and Multimodal Large Language Model (MLLM) as a decision-making framework, have led to improvements. However, these architectures struggle to balance exploration and exploitation for ZSON when encountering unseen environments, especially in multi-floor settings, such as robots getting stuck at narrow intersections, endlessly wandering, or failing to find stair entrances. To overcome these challenges, we propose AERR-Nav, a Zero-Shot Object Navigation framework that dynamically adjusts its state based on the robot's environment. Specifically, AERR-Nav has the following two key advantages: (1) An Adaptive Exploration-Recovery-Reminiscing Strategy, enables robots to dynamically transition between three states, facilitating specialized responses to diverse navigation scenarios. (2) An Adaptive Exploration State featuring Fast and Slow-Thinking modes helps robots better balance exploration, exploitation, and higher-level reasoning based on evolving environmental information. Extensive experiments on the HM3D and MP3D benchmarks demonstrate that our AERR-Nav achieves state-of-the-art performance among zero-shot methods. Comprehensive ablation studies further validate the efficacy of our proposed strategy and modules.

Figures & tables

Figure 1: AERR-Nav pipeline: Robot adopts different strategies in different states. During exploration, the optimization function operates as the cerebellum to execute “fast thinking”, while the MLLM serves as the brain to engage in “slow thinking”. When entrapment occurs, two distinct recovery methods are employed depending on the distance to the target frontier point. After completing the exploration of a single floor, MLLM uses keypoint map to guide robot in autonomously revisiting potential target or stairway entrance.

Figure 2: Analysis of examples of exploration strategy: In Example 1, (a) it believes the region behind it has higher value scores and greater uncertainty; (b) after seeing a door, MLLM reasons that a bed is unlikely to be in a bathroom and therefore chooses the hallway; (c) after a period of rapid exploration, it encounters another door and MLLM infers that the bed is more likely to be in a bedroom than in the dining area; and (d) it eventually finds the target bed. In Example 2, (e) upon seeing a door, MLLM judges that the right-side door appears to lead to a bathroom and is thus more likely to contain a toilet; (f) in the area near the right-side door, it does not observe a toilet; fast thinking is triggered and, based on uncertainty, it goes to the left-side region; (g) after a period of fast thinking, the robot originally intended to return to the initial right-side area, but upon seeing a door, MLLM infers that the narrow room may be a bathroom; and (h) it finds the target toilet.

Figure 3: Explanation of the recovery state workflow: (a) and (d) illustrate abnormal behaviors caused by frontier points distance. (b) $\rightarrow$ (c) demonstrate how to handle overly distant frontier points using a segmented method. (e) $\rightarrow$ (f) show how MLLM progressively approaches the target frontier point via fine-grained action adjustments.

Figure 4: Explanation of how the reminiscing state facilitates progress in task: (a) visualizes the representation of the KeyPoint Map. (b) illustrates how the MLLM chose to keypoints and locate the staircase entrance. (c) shows the robot descending the stairs after locating the stairway entrance and then continuing to find the target.

Method	Venue	Zero-shot	HM3D		MP3D
Method	Venue	Zero-shot	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$
ZSON [ 23 ]	NeurIPS’22	✓	25.5	12.6	15.3	4.8
ESC [ 49 ]	ICML’23	✓	39.2	22.3	28.7	14.2
L3MVN [ 40 ]	IROS’23	✓	50.4	23.1	34.9	14.5
PSL [ 31 ]	ECCV’24	✓	42.4	19.2	18.9	6.9
VLFM [ 38 ]	ICRA’24	✓	52.5	30.4	36.4	17.5

Table 1: Comparison of AERR-Nav and other zero-shot methods on the HM3D and MP3D datasets.

Method	Intra-Floor Scenarios		Inter-Floor Scenarios		All Episodes
Method	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$
VLFM [ 38 ]	64.6	37.3	0.4	0.1	52.8	30.5
MFNP [ 43 ]	68.4	30.5	13.4	9.8	58.3	26.7
ASCENT [ 11 ]	72.6	37.7	33.3	14.9	65.4	33.5
AERR-Nav	75.8	38.1	62.5	27.3	72.3	35.6

Table 2: Performance comparison between intra and inter-floor navigation scenarios.

#	Strategy Combinations			HM3D		MP3D
#	EXP	REC	REM	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$
1	✓			68.9	34.5	45.1	17.2
2	✓	✓		70.1	35.2	46.3	17.4
3	✓		✓	70.5	34.9	45.9	16.8
4	✓	✓	✓	72.3	35.6	47.9	18.7

Table 3: Ablation study of different strategy combinations on the HM3D and MP3D.

Exploration Strategy	MLLM	HM3D		MP3D
Exploration Strategy	MLLM	SR $\uparrow$	SPL $\uparrow$	SR $\uparrow$	SPL $\uparrow$
(a) Semantic ( $\alpha=1,\beta=0$ )		67.2	29.3	41.1	14.8
(b) Uncertainty ( $\alpha=0,\beta=1$ )		66.8	31.1	41.8	15.5
(c) Semantic + Uncertainty ( $\alpha=0.5,\beta=0.5$ )		68.5	33.4	43.3	16.3
(d) Semantic + Uncertainty ( $\alpha=0.5,\beta=0.5$ )	✓	69.8	34.2	45.4	17.1
(e) Semantic + Uncertainty (Dynamic $\alpha,\beta$ )		69.3	33.8	45.9	17.3

Table 4: Ablation study of the adaptive exploration strategy. Performance comparison of semantic and uncertainty-based exploration, static versus dynamic weighting, and MLLM integration on the HM3D and MP3D datasets.

Table 9

Figure 5: Case Study1: (a)–(f)/(g)–(l) illustrate the complete process of AERR-Nav/ASCENT [ 11 ] searching for the sofa.

Figure 6: Case Study2: The visualization starts after each method has finished exploring the single-floor. (a)–(f)/(g)–(l) show the process of our AERR-Nav/ASCENT [ 11 ] searching for the potted plant.

Robot Navigation

Robot navigation studies how robots perceive, localize, plan, and move safely toward goals in static or changing environments.

See all 554 papers

CLUE: Adaptively Prioritized Contextual Cues by Leveraging a Unified Semantic Map for Effective Zero-Shot Object-Goal Navigation
May 19, 2026Taeyun Kim, Alvin Jinsung Choi, Dasol Hong +1Object-Goal Navigation
School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology), Daejeon, 34141, Republic of Korea.
ZONDA: Zero-shot Object Navigation with Dynamic Avoidance in Multi-floor Environments
Jul 23, 2026Shaomin Liang, Xuanhong Liao, Shiyao ZhangRobot Navigation Dynamic Obstacle Avoidance
School of Automation and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen, China · School of Automation Science and Engineering, South China University of Technology · School of Advanced Engineering, Great Bay University, Dongguan City, China
Schrödinger's Navigator: Imagining an Ensemble of Futures for Zero-Shot Object Navigation
Dec 24, 2025Yu He, Da Huang, Zhenyang Liu +53D Scene Prediction Robot Navigation
Fudan University · Shanghai Innovation Institute · ActiMind +3

May 19, 2026J/K move · Enter open · S save

CLUE: Adaptively Prioritized Contextual Cues by Leveraging a Unified Semantic Map for Effective Zero-Shot Object-Goal Navigation

Taeyun Kim, Alvin Jinsung Choi, Dasol Hong, Hyun Myung

School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology), Daejeon, 34141, Republic of Korea.

Object-Goal Navigation

Open paper AskPDF arXiv

Zero-shot object-goal navigation (ZSON) is a challenging problem in robotics that requires a comprehensive understanding of both language and visual observations. Contextual cues from rooms and objects are critical, but their relative importance depends on the target: some objects are strongly tied to specific room types, while others are better predicted by nearby co-located objects. Existing methods overlook this distinction, leading to inefficient and inaccurate exploration. We present CLUE, a novel navigation framework that adaptively balances the use of contextual rooms and objects by leveraging commonsense knowledge extracted from an offline large language model (LLM). By estimating a target's association with room types using LLM, the agent prioritizes room cues for predictable objects and object cues for those with weak room associations. Our framework constructs a unified semantic value map that integrates both types of contextual information, adaptively weighted by the target's ambiguity to guide exploration. Combined with multi-viewpoint verification and an exploration strategy informed by contextual cues, CLUE achieves robust and efficient navigation. Extensive experiments in simulation and real-world deployments show that our method consistently outperforms state-of-the-art baselines in both success rate (SR) and success weighted by path length (SPL), demonstrating its effectiveness and practicality for real-world navigation tasks.

Abstract

Robot Navigation

Explore similar work

CLUE: Adaptively Prioritized Contextual Cues by Leveraging a Unified Semantic Map for Effective Zero-Shot Object-Goal Navigation

ZONDA: Zero-shot Object Navigation with Dynamic Avoidance in Multi-floor Environments

Schrödinger's Navigator: Imagining an Ensemble of Futures for Zero-Shot Object Navigation