Depth and 3D foundation models can predict geometry from RGB images, allowing zero-shot object navigation without a depth sensor. However, when predicted depth is mapped as if it came from a sensor, view-dependent errors accumulate in the map, remove exploration goals, and end the search before the agent can observe and correct them. We propose ERA, a training-free object navigation system that replaces the depth sensor with depth predicted from RGB images and poses. ERA follows a paradigm we call Move with Error: movement provides new views that correct map errors, and the corrected map enables further movement. It realizes this paradigm with revisable occupancy and with exploration from reachable viewpoints, which continues the search when no frontier remains, and verifies the target across views before stopping. On three benchmarks, ERA achieves state-of-the-art performance among training-free methods without depth sensors. Its map and exploration components also raise VLFM's success from 10.8% to 61.7% under the same predicted depth. We further deploy ERA on a real robot.
Figures & tables
VLFM [ 37 ]
L3MVN [ 38 ]
Depth
SR
SPL
SR
SPL
sensor
68.2
34.5
63.2
27.5
predicted
10.8
7.5
29.2
13.7
Table 1: Replacing the depth sensor by predicted depth in two zero-shot navigation systems on HM3D v2. SR: success rate; SPL: success weighted by path length (both in %).
Figure 2: Map update rules under predicted depth , along the same 120 trajectories on HM3D v2. (a) Frames in which the map offers no reachable frontier, with sensor and with predicted depth. (b, c) Erroneous and real obstacles that remain in a map built from predicted depth as the agent moves on; VLFM, SemExp and SG-Nav keep every obstacle.
Figure 3: Overview of ERA. Depth predicted by a frozen model from posed RGB images can add erroneous obstacles to the map. Revisable occupancy (Move2Error) removes them when later views observe free space there. Exploration from reachable viewpoints (Error2Move) selects the next viewpoint in reachable free space, even when no frontier remains, and moving there yields new views. Target verification checks a target candidate in views from different positions before stopping. The panels are illustrations from real maps and views.
Figure 4: Free-space clearing. The same observations are fused into a map without (top) and with (bottom) free-space clearing. Predicted depth places an obstacle across the passage (step 60). Later views clear it in our map (step 85), and the agent walks through (step 96); without clearing, the obstacle stays and the passage remains closed.
HM3D v1
HM3D v2
MP3D
Method
Depth sensor
Training-free
SR
SPL
SR
SPL
SR
SPL
SemExp [ 6 ]
✓
✗
–
–
–
–
36.0
14.4
ZSON [ 16 ]
✗
✗
25.5
12.6
–
–
15.3
4.8
PixNav [ 3 ]
✗
✗
37.9
20.5
–
–
–
–
ESC [ 42 ]
✓
✓
39.2
22.3
–
–
28.7
14.2
L3MVN [ 38 ]
✓
✓
50.4
23.1
63.2 †
27.5 †
34.9
14.5
Table 2: Object navigation on the validation splits of HM3D v1, HM3D v2 and MP3D (SR / SPL in %). † Evaluated under our protocol. Bold: best among training-free methods without a depth sensor.
Sensor
DA3
VGGT
System
SR
SPL
SR
SPL
SR
SPL
VLFM [ 37 ]
68.2
34.5
10.8
7.5
8.3
6.1
+ revisable occupancy
–
–
50.7
25.8
52.4
28.5
+ reachable viewpoints
–
–
61.7
30.2
59.8
31.5
L3MVN [ 38 ]
63.2
27.5
29.2
13.7
26.5
13.2
+ revisable occupancy
–
–
54.8
24.3
56.3
25.3
Table 3: Generalization on HM3D v2 across systems and depth sources. The rows below each system are cumulative. Implementation details are given in Sec. D.3 .
Variant
SR
SPL
ERA (full)
70.1
30.6
w/o free-space clearing
13.9
6.4
frontier goals only
32.6
18.1
w/o target verification
67.6
29.9
With sensor depth
ERA (full)
70.7
33.7
Table 4: Ablation study on HM3D v2. Each variant changes one component of the full system.
Figure 5: Real-world deployment. ERA on a wheeled humanoid robot that uses only the RGB camera in its head. Top, from left to right: the robot at the goal, the view of its camera at the start and at the stop with the detection of the goal, and the map built from predicted depth with the path of the robot (black) and the goal (orange circle). Bottom: two runs from the same start, Base ( Tab. 5 ) and ERA, and their maps at the end.
Runs
System
Success
SR
Three categories
ERA
11 / 12
91.7
Same starts
Base
0 / 6
0
+ revisable occupancy
1 / 6
16.7
+ reachable viewpoints
6 / 6
100
Table 5: Real-robot runs. Success counts the runs that stop within 1 m of the target. The lower part compares runs from the same starts. Base uses the map update and frontier goals of VLFM; each row adds one component to the row above, and the last row is ERA.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: The map of VLFM at identical poses with sensor and with predicted depth. (a, b) The two maps of one episode after the initial turn. (c) Episodes in which the map has no frontier left, over the 120 episodes. (d) Obstacle cells that only the map from predicted depth contains, by the sensor reading at the pixels that produce them.
Figure 7: The seven update rules at the same frame. All maps except the first are built from the same predicted depth along the same trajectory; the first is the map of VLFM from sensor depth. Blue shows the region that the agent can reach and its reachable frontiers.
Map
Frames
Error
Real
Encl.
No frontier
ApexNav (released)
32
86.6
99.8
28.6
82.6
ApexNav
16
68.2
99.1
13.9
75.3
ApexNav
8
53.5
96.9
6.0
60.9
ApexNav
4
46.0
92.3
4.9
47.4
ApexNav
2
46.2
84.5
4.7
38.1
Revisable occ.
2
24.3
88.8
0.0
45.1
Appendix
Table 6: ApexNav with stronger free-space updates , built from predicted depth along the same 120 trajectories (%). Frames : free-space observations that remove an obstacle observed many times. Error , Real : erroneous and real obstacle cells left at the end of an episode, among the cells that each map marks. Encl. : frames in which the map encloses the agent. No frontier : frames without a reachable frontier, among those in which the map from sensor depth has one.
System
Depth
SR
SPL
Seen
Exh.
VLFM
sensor
68.2
34.5
88.4
8.0
VLFM
DA3
10.8
7.5
27.1
86.9
+ relaxed termination
DA3
17.7
11.0
36.9
74.2
+ revisable occupancy
DA3
50.7
25.8
70.0
24.8
+ reachable viewpoints
DA3
61.7
30.2
83.3
0.0
VLFM
VGGT
8.3
6.1
25.4
88.6
Appendix
Table 7: Exploration statistics on HM3D v2. Seen : episodes in which the goal becomes clearly visible (%). Exh. : episodes that end because the agent has no exploration goal left (%).
Parameter
Symbol
Value
Depth and reliability
Keyframes per query
–
up to 5
Keyframe window
–
40 steps
Confidence quantile
κ
0.15
Uncertainty offset and slope
α , β
0.03 m, 0.05
Conflict tolerance
–
0.10m+0.08D^t
Appendix
Table 8: Parameter values of ERA.
Figure 8: A second example of free-space clearing , in a corridor. The layout is the same as in Fig. 4 .
Figure 9: Depth error on the target surface while the agent approaches a candidate on HM3D v2. (a) Median error by the true distance to the target. (b) Correlation between the errors of two views of the same target, by the distance between the two camera positions; a correlation near one means that the second view repeats the error of the first.
Noise per action
Final pose error
Level
σd
σθ
Position
Heading
SR
SPL
None
–
–
–
–
70.1
30.6
Low
1%
0.1∘
0.10 m
1.2∘
67.3
30.1
Medium
2%
0.3∘
0.29 m
3.4∘
69.9
30.8
High
5%
1.0∘
0.96 m
11.4∘
66.9
29.2
Appendix
Table 9: Navigation with drifting poses on HM3D v2. The final pose error is the error at the end of an episode, averaged over the episodes.
Figure 10: Qualitative comparison on HM3D v2 with predicted depth. VLFM stops within a few steps of the start because its map has no frontier left. With revisable occupancy and exploration from reachable viewpoints, it explores the scene and reaches the goal, as does ERA.
Deployed
Ideal scale
Depth model
Mean
Median
P90
Mean
Median
P90
DA3
11.0
7.3
21.4
4.0
2.5
9.0
VGGT
16.7
9.8
30.3
3.3
2.6
5.2
Appendix
Table 10: Relative depth error (%) of the two depth sources as deployed, and with an ideal per-frame scale. The error of a frame is the mean absolute relative error over its pixels; we report the mean, the median and the 90th percentile over frames.
Variant
SR
SPL
Δ SR
ERA (full)
70.1
30.6
–
w/o target verification
67.6
29.9
−2.5
w/o position update
68.2
30.6
−1.9
w/o view-keeping approach
69.3
30.4
−0.8
w/o stopping checks
69.8
30.8
−0.3
Appendix
Table 11: Components of target verification on HM3D v2. w/o target verification removes all three parts; each of the last three rows removes one of them.
Figure 11: Two runs on the robot. Left: the robot at the goal of the first run; the RGB camera in its head is the only camera used. Right: the view of this camera at the start, during the approach and at the stop, where the boxes are the detections of the goal object, and the map built from predicted depth with the path of the robot (black) and the goal (orange circle). The robot stops 0.7 m from both the chair and the trash can; the trash can is not in view at the start.
Run
Target
Start
In view
Path
Stop
1
chair
6.5 m
✗
6.1 m
0.7 m
2
chair
4.3 m
✗
5.4 m
0.7 m
3
trash can
1.6 m
✓
1.7 m
0.7 m
4
trash can
1.6 m
✗
1.2 m
0.7 m
5
chair
3.7 m
✓
3.4 m
0.7 m
6
chair
3.6 m
✗
2.7 m
1.0 m
Appendix
Table 12: Real-robot runs. Start: distance from the start position to the target. In view: target visible in the first image. Stop: distance from the center of the robot to the target when it stops; in run 8 the stopping conditions were never met.