Organizations: Intelligent Space Robotics Laboratory, Skolkovo Institute of Science and Technology Moscow, Bolshoy Boulevard 30, bld. 1, 121205, Moscow, Russia.
We present HorizonNav, a video diffusion-based framework for anticipatory robot navigation that transforms a single RGB image and a text prompt into a geometrically aligned representation of plausible surrounding 3D space. Given the initial observation and a textual description of the reconstruction objective, a vision-language model, guided by designed instructions, interprets the scene and constructs structured prompts for a video diffusion model. The resulting video sequence provides plausible future observations of the environment, which are subsequently processed using monocular ORB-SLAM3 for camera pose estimation and Video Depth Anything for dense depth prediction. Our central contribution is a geometry-guided depth alignment stage that combines the estimated camera poses with triangulated visual landmarks to constrain dense depth predictions across multiple views. This alignment improves the geometric consistency of the reconstructed 3D environment, addressing inconsistencies that temporal coherence of the generated video alone cannot resolve. The experimental evaluation of navigation in imagined 3D environment shows that across a 19-scene horizon study, planning-feasible scenes increase from 13/19 at P20 to 15/19 at P81, while successful RRT trials increase from 62/90 to 75/90. In a separate corridor study, 6 of 25 evaluable generated-camera trajectories violate the reconstructed collision constraints; RRT planner recovers feasible alternatives in 5 of those 6 cases. A larger archive contains 58 usable reconstructions from 67 attempted cases, of which 36 support a downstream plan and 22 do not, exposing reconstruction and free-space failure modes rather than hiding them without lidar. Across 15 corridor reconstructions, the mean reconstruction error is approximately 0.30 m at matched structural landmarks.
Figures & tables
Fig. 1: Overview of the HorizonNav framework. Visual reasoning and exploration planning: a VLM analyzes the initial RGB image and language task to generate structured exploration commands. Exploratory video generation: Wan 2.2 synthesizes plausible future observations along multiple virtual camera trajectories. Geometry-guided 3D reconstruction: ORB-SLAM3 estimates camera poses and sparse landmarks, while Video Depth Anything predicts dense depth maps. Landmark-guided depth alignment, multi-view registration, and point-cloud fusion produce a geometrically consistent imagined 3D map for subsequent robot navigation and path planning.
Fig. 2: Qualitative generated-view geometry. Left: representative exploration branches showing generated RGB, dense-depth diagnostic, ORB-SLAM3 sparse structure, and corresponding real-view references. Right: reconstructed structure registered to the external LiDAR reference for visualization and evaluation; LiDAR is not available to HorizonNav during reconstruction or planning.
Prefix
Feasible scenes
Successful RRT trials
P20
13/19
62/90
P40
14/19
70/90
P60
14/19
70/90
P81
15/19
75/90
TABLE I: Planning utility across generated-view horizons.
State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou 310027, China · Huzhou Institute of Zhejiang University, Huzhou 313000, China · Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou 310027, China