Digital elevation modeling of planetary surfaces is essential for studying past and ongoing geological processes. Wide-angle imagery acquired during spacecraft descent promises to offer a low-cost option for high-resolution terrain reconstruction. However, accurate 3D reconstruction from such imagery is challenging due to strong radial distortion and limited parallax from vertically descending, predominantly nadir-facing cameras. Conventional multi-view stereo exhibits limited depth range and reduced fidelity under these conditions and also lacks domain-specific priors. We present the first study of modern neural reconstruction methods for planetary descent imaging. We also develop a novel approach that incorporates an explicit neural height field representation, which provides a strong prior since planetary surfaces are generally continuous, smooth, solid, and free from floating objects. This study demonstrates that neural approaches offer a strong and competitive alternative to traditional multi-view stereo (MVS) methods. Experiments on simulated descent sequences over high-fidelity lunar and Mars terrains demonstrate that the proposed approach achieves increased spatial coverage while maintaining satisfactory estimation accuracy.
Figures & tables
Figure 1: First: Height map and x-y camera center (green cross) for the lunar scene. Remaining: Example simulated descent views toward the center of the lunar scene, including the top-most and bottom-most views.
Figure 2: First: Height map and x-y camera center (green cross) for the Mars scene. Remaining: Example simulated descent views toward the top-right corner, including the top-most and bottom-most views.
Figure 3: Fisheye Ray Casting: maps each pixel through a wide-angle lens to a 3D ray using the lens’s radial distortion model.
Figure 4: Our method uses two scene representations that each define the density and colour of sampled points (x,y,z) along the ray. In the NeRF representation (purple) , an MLPρ maps (x,y,z) into density ρxyz;A and density embedding embxyz . MLPc maps embxyz into colour, which is then multiplied by ray direction dependent lighting ( lightd ) to get final colour: cxyz;A . In the heightmap representation (green) , given our world frame is defined so the surface is perpendicular to the z-axis, the (x,y) is passed onto HeightField to predict heights hxy and the density embedding of the (x,y) column: embxy . Then, ρxy=ρxyconst below the height, smoothly decaying to zero via a sigmoid above it. MLPc maps embxy into colour of the column along z ; which is then multiplied by lightd to get cxyz;B for each z along (x,y) column.
Figure 5: Qualitative comparison of DEM reconstructions from simulated lunar fisheye descent imagery against a high-resolution mesh ground truth. The height is calculated from the top-most camera. Our method achieves improved surface coverage while maintaining satisfactory elevation accuracy.
Figure 6: Qualitative comparison of reconstructed DEMs from simulated Mars fisheye descent imagery, using a high-resolution mesh as ground truth. Our method achieves superior surface coverage while maintaining satisfactory elevation accuracy. The height is calculated from the top-most camera. Notably, Nerfacto suffers from floating artifacts, whereas our approach effectively suppresses these floaters, resulting in a smoother and more coherent surface reconstruction.
Table 1: Quantitative comparison with Metashape and Nerfacto on lunar and Mars datasets. AED and RED were computed after filling NaNs with the spatially nearest available point, while [email protected] counts a point as correct only if the relative elevation difference is below 0.1, treating NaNs as incorrect. Ours has shown better [email protected] compared to baselines.
Figure 7: Qualitative comparisons of DEMs using simulated lunar fisheye descent images, with a high-resolution mesh as ground truth. Our full model demonstrates superior coverage while maintaining high-quality reconstruction compared to two ablations.
This study presents a vision system for planetary rovers, combining real-time perception with offline terrain reconstruction. The real-time module integrates CLAHE enhanced stereo imagery, YOLOv11n based object detection, and a neural network to estimate object distances. The offline module uses the Depth Anything V2 metric monocular depth estimation model to generate depth maps from captured images, which are fused into dense point clouds using Open3D. Real world distance estimates from the real time pipeline provide reliable metric context alongside the qualitative reconstructions. Evaluation on Chandrayaan 3 NavCam stereo imagery, benchmarked against a CAHV based utility, shows that the neural network achieves a median depth error of 2.26 cm within a 1 to 10 meter range. The object detection model maintains a balanced precision recall tradeoff on grayscale lunar scenes. This architecture offers a scalable, compute-efficient vision solution for autonomous planetary exploration.
Lomash Relia, Jai G Singla, Amitabh +1
Department of Computer Engineering Devang Patel Institute of Advanced Technology and Research Anand, India · Space Applications Centre Indian Space Research Organization Ahmedabad, India
Reconstructing printable 3D models from Mars rover imagery is challenging because Martian terrain is low-texture, irregular, and partially observed. We evaluate a pipeline that estimates stereo depth from NASA Curiosity images, completes geometry, and exports watertight OBJ meshes. On Middlebury, RAFT-Stereo outperforms semi-global block matching (SGBM), reducing disparity MAE from 3.22px to 0.73px and increasing valid prediction coverage from 76.3% to 100.0%. On Curiosity imagery, however, RAFT's denser disparities show weaker edge alignment and higher photometric reprojection error, suggesting that benchmark accuracy does not directly transfer to Martian terrain reconstruction. Geometry completion demonstrates a tradeoff between local fidelity and global connectivity. We find that alpha shapes preserve accurate but fragmented structure, Poisson reconstruction produces more coherent meshes but adds unsupported surfaces, and a deterministic diffusion-fill baseline is intermediate but sensitive to stereo quality. Overall, standard stereo and completion methods can produce printable approximations of Martian terrain, but reliable reconstruction requires stronger domain-specific validation.
On-orbit inspection imagery is crucial as it enables characterization of non-cooperative resident space objects, providing the geometry and structural condition essential for active debris removal and on-orbit servicing mission planning. However, most existing neural implicit surface reconstruction methods have been confined to synthetic or hardware-in-the-loop data with known camera poses and controlled illumination. In this work, we present a pipeline for neural implicit surface reconstruction of non-cooperative space objects from monocular inspection imagery. We demonstrate it on publicly released ISS inspection footage from the STS-119 mission and publicly released on-orbit inspection footage of an H-IIA rocket upper stage. We find that segmentation-based background removal is essential for successful camera pose estimation from real on-orbit footage, where background variation between frames caused direct processing to fail entirely. We further incorporate photometric correction of per-frame exposure variations and analyze its behavior across datasets, finding that performance in shadowed regions varies with the illumination characteristics of the input footage.
Bala Prenith Reddy Gopu, Patrick Quinn, George M. Nehma +4
Department of Aerospace, Physics and Space Sciences, Florida Institute of Technology, 150 W University Blvd, Melbourne, 32901, FL, USA · Creare LLC, Hanover, 03755, NH, USA