Improve How You Improve: Recursive Policy Self-Improvement for Zero-Shot Open-Vocabulary Multi-Object Navigation
Authors: Yang Li, Ziju Han, Aming WU, Zihao Zhang, Sijia Zhang, Yahong Han, Qinghua Hu
Organizations: School of Artificial Intelligence, Tianjin University, China · School of Computer Science and Information Engineering, Hefei University of Technology, China
To advance the development of embodied general navigation, zero-shot open-vocabulary multi-object navigation has attracted increasing attention, aiming to enable robots to efficiently search multiple objects in unseen environments without training. For this task, the key lies in improving navigation policies from earlier searches to locate subsequent goals more efficiently. Recent methods automate this process through trajectory reflection and rule revision, enabling policy self-improvement. However, the mechanisms for diagnosing problems and generating revisions are typically designed by humans and remain fixed across iterations. Consequently, the system's capacity for continued self-improvement remains constrained, even as its navigation policy evolves. To this end, we propose RPSI-Nav, a Recursive Policy Self-Improvement framework for zero-shot multi-object navigation that revises both the navigation program and its revision mechanism. RPSI-Nav diagnoses search behavior, requests diagnostic views when existing observations cannot distinguish competing causes, and generates targeted navigation revisions. Feedback from subsequent goal searches is linked to the rules that produced these revisions, guiding changes to diagnosis, observation acquisition, and revision planning. The updated improver governs subsequent navigation improvements, forming a recursive loop in which RPSI-Nav revises its own improvement process. Across goals, the robot conditionally inherits accepted program updates and spatial memory, enabling search experience to improve both future navigation and adaptation. Benchmark experiments and qualitative real-robot evaluations validate RPSI-Nav's effectiveness.
Figures & tables
Figure 1: Recursive improvement across goal searches. (a) S0 finds the lamp and pillow but leaves the plant hypothesis untested. (b) S2 selects a view that observes the lamp and plant. (c) Search experience guides navigation revisions, while revision outcomes drive the improver’s self-revision. Accepted improver updates govern subsequent revisions. Sk denotes the navigation program Pk in our method.
Figure 2: Overview of RPSI-Nav. Goals Q , RGB-D observations (It,Dt) , pose ξt , and memory Mt support navigation by Pk , yielding actions at , trajectory τ , and goal confirmations. A fixed harness executes actions with frozen-VLM perception. Rk=(Dk,Qk,Uk) guides diagnosis, observation acquisition, and revision through the shared agent workflow. Augmented records B+ and revision feedback support patches ΔXk , X∈{P,R} . Accepted patches apply under guard gk(x) . Revision outcomes drive Rk ’s self-revision, and accepted Rk+1 governs subsequent navigation revisions, closing the recursive loop.
Figure 3: Real-world multi-object navigation. The robot sequentially searches for a blue box, a TV, and a green table. First- and third-person views show the search process; colored boxes highlight targets, and arrows indicate motion directions.
Method
Type
HM3D
SR ↑
SPL ↑
PR ↑
PPL ↑
VLFM [ 40 ]
Sequential
44.92
19.90
57.34
24.87
OneMap [ 1 ]
54.20
27.80
65.50
32.70
WTRP [ 19 ]
57.20
33.40
67.10
38.20
RPSI-Nav (Ours)
63.14
35.81
69.49
40.07
OneMap [ 1 ]
Simultaneous
52.54
23.63
68.93
31.40
Table 1: Multi-object navigation on HM3D.
Method
SR ↑
SPL ↑
PR ↑
PPL ↑
OneMap [ 1 ]
16.83
6.79
33.69
13.95
LifelongCrossNav [ 16 ]
36.57
12.38
54.33
18.86
RPSI-Nav (Ours)
39.81
15.41
56.71
21.49
Table 2: Multi-object navigation on HM3D-MFMON.
Method
SR ↑
SPL ↑
PR ↑
PPL ↑
OneMap [ 1 ]
0.00
0.00
18.52
7.97
LifelongCrossNav [ 16 ]
12.50
3.11
33.68
10.86
RPSI-Nav (Ours)
15.97
4.87
34.95
12.31
Table 3: Multi-object navigation on the 288 Cross-Floor-Required HM3D-MFMON episodes.
Method
SR ↑
SPL ↑
Explore-EQA [ 24 ]
55.0
37.9
CG + Frontier [ 6 ]
61.5
41.3
3D-Mem [ 37 ]
69.1
48.9
HIMM [ 12 ]
72.8
56.1
SCOPE [ 30 ]
73.7
53.5
RPSI-Nav (Ours)
75.2
54.7
Table 4: Comparison on GOAT-Bench.
Variant
SR ↑
SPL ↑
PR ↑
PPL ↑
No program revision
54.2
27.8
65.5
32.7
Fixed improver R0
57.7
30.4
68.4
36.4
Recursive, fixed Q0
59.8
32.1
70.1
38.2
RPSI-Nav
63.1
35.8
69.5
40.1
Table 5: Effect of improver self-revision. Scores are means over independent runs (%).
Variant
SR ↑
SPL ↑
PR ↑
PPL ↑
Reset R across goals
60.9
33.8
68.8
38.7
RPSI-Nav
63.1
35.8
69.5
40.1
Table 6: Effect of cross-goal improver inheritance on HM3D. Scores are means over five independent runs (%).
Reference
Improver
Δ SR ↑
Δ SPL ↑
P0
R[0]
+2.2
+1.5
R[50]
+3.1
+2.3
R[100]
+4.7
+3.7
Pref
R[0]
+0.7
+0.5
R[50]
+1.7
+1.4
R[100]
+2.6
+2.2
Table 7: Mean navigation gains produced by improver snapshots, relative to each reference program.
Variant
SR ↑
SPL ↑
Frozen R∗
62.4±1.52
35.0±0.89
Adaptive R∗
64.2±1.48
36.2±0.87
Paired gain ( Δ )
+1.8±0.45
+1.2±0.25
Table 8: Frozen versus adaptive evolved improvers across five paired runs (100 episodes per run).
Figure 4: Navigation comparison and a representative RSI trace. Upper row: OneMap fails to complete the goal sequence. Lower row: RPSI-Nav completes the chair, toilet, and sofa searches. Top-down maps and RGB views show selected timesteps; boxes annotate visible targets. The three stages illustrate the recursive repair process: (a) R0 revises region ranking, producing P1 . (b) Repeated views prompt R0 to review the repair and add a viewpoint-coverage check, yielding R1 . (c) R1 invokes this check and redirects the next repair to viewpoint selection, producing P2 . Pk and Rk denote the navigation and improvement programs, respectively.
Event
Episodes
Rate (%)
Accepted R update
120/236
50.85
Cross-goal revision feedback
67/236
28.39
Completed recursive chain (total)
84/236
35.59
Cross-goal
58/236
24.58
Within-goal only
26/236
11.02
Table 9: Recursive execution statistics across all 236 sequential three-goal episodes.
Zero-Shot Object-Goal Navigation (ZS-OGN) requires embodied agents to explore and locate target objects without any prior training. To this end, recent methods leverage foundation models. But they typically rely on static priors and lack adaptation, which leads to repeated errors and costly trial and error. In this paper, we propose a self-evolving ZS-OGN framework that enables continuous test-time improvement. Specifically, we build an agentic rule memory by extracting actionable knowledge from past trajectories. Then, we propose a retrieval strategy based on upper confidence bound, selecting effective rules by balancing semantic relevance and historical success. In addition, we introduce a memory-guided preflection module that forecasts potential outcomes before action, reducing inefficient exploration. Extensive experiments show that our method outperforms existing zero-shot baselines, achieving a 10.1% improvement in success rate with fewer unnecessary steps.
Zero-Shot Object Navigation (ZSON) in unknown multi-floor environments presents a significant challenge. Recent methods, mostly based on semantic value greedy waypoint selection, spatial topology-enhanced memory, and Multimodal Large Language Model (MLLM) as a decision-making framework, have led to improvements. However, these architectures struggle to balance exploration and exploitation for ZSON when encountering unseen environments, especially in multi-floor settings, such as robots getting stuck at narrow intersections, endlessly wandering, or failing to find stair entrances. To overcome these challenges, we propose AERR-Nav, a Zero-Shot Object Navigation framework that dynamically adjusts its state based on the robot's environment. Specifically, AERR-Nav has the following two key advantages: (1) An Adaptive Exploration-Recovery-Reminiscing Strategy, enables robots to dynamically transition between three states, facilitating specialized responses to diverse navigation scenarios. (2) An Adaptive Exploration State featuring Fast and Slow-Thinking modes helps robots better balance exploration, exploitation, and higher-level reasoning based on evolving environmental information. Extensive experiments on the HM3D and MP3D benchmarks demonstrate that our AERR-Nav achieves state-of-the-art performance among zero-shot methods. Comprehensive ablation studies further validate the efficacy of our proposed strategy and modules.
Jingzhi Huang, Junkai Huang, Haoyang Yang +2
Hong Kong Polytechnic University · Institute of automation, Chinese Academy of Sciences · Hong Kong University of Science and Technology (Guangzhou)
Zero-shot object navigation requires agents to locate unseen targets in unfamiliar environments without prior maps or task-specific training. Despite the commonsense reasoning ability of vision-language models (VLMs), existing mapless navigators often suffer from repeated exploration and premature stopping caused by limited historical context and false-positive target predictions. We propose ReMemNav, a training-free framework that combines lightweight semantic grounding, memory-based decision correction, and target verification. RAM-derived semantic priors guide panoramic direction selection, while a geometry-triggered correction mechanism retrieves historical scene descriptions when the selected direction points toward previously visited regions. When a target is predicted to be visible, ReMemNav verifies the corresponding single-view observation before the final approach. Depth-based action sampling then converts the high-level decision into collision-free motion. Experiments on HM3D and MP3D show that ReMemNav achieves higher success rates and path efficiency than existing training-free zero-shot baselines. Specifically, ReMemNav achieves absolute SR/SPL gains of 1.7%/5.3% on HM3D v0.1, 18.2%/11.1% on HM3D v0.2, and 8.7%/6.9% on MP3D.