Schrödinger's Navigator: Imagining an Ensemble of Futures for Zero-Shot Object Navigation
Authors: Yu He, Da Huang, Zhenyang Liu, Zixiao Gu, Qiang Sun, Guangnan Ye, Yanwei Fu, Yu-Gang Jiang
Organizations: Fudan University · Shanghai Innovation Institute · ActiMind · Shanghai Jiao Tong University · Shanghai University of International Business and Economics · Shanghai Key Laboratory of Multimodal Embodied AI
Zero-shot object navigation (ZSON) requires robots to find target objects in unseen environments without task-specific fine-tuning or pre-built maps, a key capability for general-purpose service robots. Yet methods that perform well in simulation often degrade in cluttered real-world scenes with severe occlusion and latent hazards, where large unseen regions make single-scene inference brittle and unsafe. We propose Schrödinger's Navigator, a belief-aware framework that reasons at inference time over multiple trajectory-conditioned imagined 3D futures. Given candidate paths, a trajectory-conditioned 3D world model predicts hypothetical observations and maintains a superposition of plausible scene realizations rather than committing to one map. An adaptive occluder-aware sampler directs imagination to uncertainty-critical regions, while a Future-Aware Value Map (FAVM) aggregates imagined futures for robust, proactive action selection. Experiments in simulation and on a physical Go2 quadruped show that Schrödinger's Navigator outperforms strong ZSON baselines, improving hidden-target discovery and risk-aware waypoint selection in occlusion-heavy navigation scenarios. These results highlight imagined 3D futures as a scalable and generalizable strategy for zero-shot navigation in uncertain real-world environments.
Figures & tables
Figure 1: Real-world zero-shot object navigation often fails when the target (e.g., a cat) is hidden behind occlusions and surrounded by hazardous space. Conventional navigation systems perceive only the immediate occluder and are unable to infer what exists beyond it. Our Schrödinger’s Navigator addresses this challenge by modeling the unobserved regions as plausible futures.
Figure 2: Overview of the Schrödinger’s Navigator. Given the navigation context, the system adaptively samples candidate trajectories around occluded and uncertain regions. A trajectory-conditioned 3D world model then imagines future 3DGS observations along each path to expose hidden structures, target cues, and risks. These imagined futures are fused with current observations to construct a Future-Aware Value Map, aggregating trajectory-level utility and risk for robust waypoint selection. Finally, the selected navigation target is executed through the corresponding local controller.
Figure 3: Occluder-aware camera trajectory sampling.
Figure 4: Overview of trajectory-conditioned 3D imagination: Given the current observation and robot pose, an occluder-aware sampler generates candidate trajectories that condition a 3D world model to predict future 3DGS scenes. These scenes are depth-aligned to recover a consistent scale, transformed into the world frame, and fused into a unified 3DGS scene for future-aware planning.
Method
Training Free
HM3D
SR ↑
SPL ↑
DTG ↓
ZSON [ 26 ]
✗
0.255
0.126
–
PixNav [ 5 ]
✗
0.379
0.205
–
SPNet [ 52 ]
✗
0.312
0.101
–
SGM [ 50 ]
✗
0.602
0.308
–
ESC [ 55 ]
✓
0.392
0.223
–
Table 1: Quantitative Comparison on Simulation Results. Cell background colors indicate the method is the best , second best , or third best on this metric.
Variant
SR ↑
Trig. / Ep. ↓
Lat. / Trig. (s) ↓
Wait Ratio (%) ↓
Waypoint Change (%)
No Imagination
0.553
0.00
–
0.0
–
Single Traj. (N=1)
0.594
4.31
3.0
7.2
28.4
Dense Sampling (N=5)
0.622
4.18
15.0
30.8
51.6
Imagination-heavy (β=0.2)
0.497
4.36
9.0
22.9
67.2
Observation-heavy (β=0.8)
0.595
4.28
9.0
19.9
35.3
FAVM w/o Exploration E(p)
0.629
4.20
9.0
19.8
40.8
Table 2: Ablation and runtime analysis of Schrödinger’s Navigator on HM3D. We report success rate (SR), average number of imagination triggers per episode, per-trigger imagination latency, episode-level waiting ratio, and waypoint change rate.
Figure 5: Real-world object-navigation demonstrations of Schrödinger’s Navigator across three indoor scenarios: Office-Chair, Classroom-Plant, and Common Room-Trash Can. Each example shows the executed trajectory, egocentric observations with predicted target directions and target boxes, and world-model imagined views that reveal the target appearance.
Scene
Office
Classroom
Common Room
All
Search for static objects
InstructNav
7/10
7/10
8/10
22/30
Ours
8/10
8/10
7/10
23/30
Scenarios with Unknown Risks
InstructNav
4/10
3/10
4/10
11/30
Ours
5/10
6/10
6/10
17/30
Table 3: Real-world quantitative comparison against the InstructNav baseline. We report success counts (Success / Total Trials) across three indoor environments.
Parameter
Symbol
Value
Description
Trajectory ensemble size
N
3
Left, right, and over-the-top futures.
Camera poses per trajectory
Npose
16
Initial camera poses before interpolation.
Nominal radial offset
δ
0.20 m
Offset used to define the nominal orbit radius.
Minimum orbit radius
dcmin
1.0 m
Lower bound on the nominal orbit radius.
Nominal orbit radius
dc
–
dc=max(rt+rrobot+δ,dcmin) .
Angular expansion
Δθ
20∘
Extends lateral views beyond the observed occluder boundary.
Table 4: Geometric parameters for adaptive occluder-aware virtual-camera trajectory sampling.
Parameter
Symbol
Value
Description
Observation/future fusion balance
β
0.60
Balances mˉt and mtFA .
Semantic score weight
αsem
1.0
Weight for target relevance St(p) .
Exploration score weight
αexp
1.0
Weight for expected reveal Et(p) .
Risk score weight
αrisk
1.0
Weight for traversal risk Rt(p) .
Imagined semantic discount
λimg
0.50
Discount for imagined target cues.
Semantic-transfer depth tolerance
τd,i(m)
Adaptive
Radius- and depth-dependent; see Appendix C.
Table 5: Default coefficients for FAVM construction and fusion.
Figure 6: Trajectory-conditioned 3DGS generation and fusion. Each row shows left-bypass, right-bypass, over-the-top, and merged futures.
Figure 7: Future-Aware Value Map construction. The figure shows the current observation, left/right/up imagined futures, per-trajectory semantic, exploration, risk, and contribution maps, and the final fused affordance map. Warm colors indicate higher utility, except in risk maps where they indicate higher penalty; the in-figure label IFAVM denotes the FAVM decomposition.
Occlusion
ImagineNav++ SR ↑
Ours SR ↑
Δ SR ↑
Low
0.682
0.720
+0.038
Medium
0.629
0.674
+0.045
High
0.564
0.652
+0.088
Table 6: HM3D success rates stratified by occlusion severity. Δ SR denotes Ours minus ImagineNav++.
Figure 8: Successful occlusion-heavy simulation rollouts. Each row shows the current observation, imagined future views, top-down checkpoints, first-person rollout frames, and the final target reveal.
Figure 9: End-to-end failure modes. Rows show localization, detection, occlusion-handling, and termination errors with imagined hypotheses, execution checkpoints, final evidence, and diagnostic cues.
Figure 10: FlashWorld-related failure modes. Rows show hallucinated target evidence, scale/depth drift, missed occluded evidence, and trigger gaps.
Figure 11: Real-world deployment architecture for Go2 sensing, FastAPI communication, remote GPU inference, and velocity-command execution.