Neural 3D Reconstruction of Planetary Surfaces from Descent-Phase Wide-Angle Imagery

Melonie de Almeida; George Brydon; Divya M. Persaud; John H. Williamson; Paul Henderson

cs.CVApr 14, 2026

Neural 3D Reconstruction of Planetary Surfaces from Descent-Phase Wide-Angle Imagery

Authors: Melonie de Almeida, George Brydon, Divya M. Persaud, John H. Williamson, Paul Henderson

Organizations: University of Glasgow

Abstract

Digital elevation modeling of planetary surfaces is essential for studying past and ongoing geological processes. Wide-angle imagery acquired during spacecraft descent promises to offer a low-cost option for high-resolution terrain reconstruction. However, accurate 3D reconstruction from such imagery is challenging due to strong radial distortion and limited parallax from vertically descending, predominantly nadir-facing cameras. Conventional multi-view stereo exhibits limited depth range and reduced fidelity under these conditions and also lacks domain-specific priors. We present the first study of modern neural reconstruction methods for planetary descent imaging. We also develop a novel approach that incorporates an explicit neural height field representation, which provides a strong prior since planetary surfaces are generally continuous, smooth, solid, and free from floating objects. This study demonstrates that neural approaches offer a strong and competitive alternative to traditional multi-view stereo (MVS) methods. Experiments on simulated descent sequences over high-fidelity lunar and Mars terrains demonstrate that the proposed approach achieves increased spatial coverage while maintaining satisfactory estimation accuracy.

Figures & tables

Panel of Figure 1 — Figure 1: First: Height map and x-y camera center (green cross) for the lunar scene. Remaining: Example simulated descent views toward the center of the lunar scene, including the top-most and bottom-most views.

Panel of Figure 2 — Figure 2: First: Height map and x-y camera center (green cross) for the Mars scene. Remaining: Example simulated descent views toward the top-right corner, including the top-most and bottom-most views.

Figure 3: Fisheye Ray Casting: maps each pixel through a wide-angle lens to a 3D ray using the lens’s radial distortion model.

Figure 4: Our method uses two scene representations that each define the density and colour of sampled points $\mathbf{(x,y,z)}$ along the ray. In the NeRF representation (purple) , an $\mathrm{MLP_{\rho}}$ maps $\mathbf{(x,y,z)}$ into density $\mathbf{\rho_{xyz;A}}$ and density embedding $\mathbf{emb_{xyz}}$ . $\mathrm{MLP_{c}}$ maps $\mathbf{emb_{xyz}}$ into colour, which is then multiplied by ray direction dependent lighting ( $\mathbf{light_{d}}$ ) to get final colour: $\mathbf{c_{xyz;A}}$ . In the heightmap representation (green) , given our world frame is defined so the surface is perpendicular to the z-axis, the $\mathbf{(x,y)}$ is passed onto $\mathrm{HeightField}$ to predict heights $\mathbf{h_{xy}}$ and the density embedding of the $\mathbf{(x,y)}$ column: $\mathbf{emb_{xy}}$ . Then, $\mathbf{\rho_{xy}}=\mathbf{\rho_{xy}^{\text{const}}}$ below the height, smoothly decaying to zero via a sigmoid above it. $\mathrm{MLP_{c}}$ maps $\mathbf{emb_{xy}}$ into colour of the column along $z$ ; which is then multiplied by $\mathbf{light_{d}}$ to get $\mathbf{c_{xyz;B}}$ for each $z$ along $\mathbf{(x,y)}$ column.

Figure 5: Qualitative comparison of DEM reconstructions from simulated lunar fisheye descent imagery against a high-resolution mesh ground truth. The height is calculated from the top-most camera. Our method achieves improved surface coverage while maintaining satisfactory elevation accuracy.

Figure 6: Qualitative comparison of reconstructed DEMs from simulated Mars fisheye descent imagery, using a high-resolution mesh as ground truth. Our method achieves superior surface coverage while maintaining satisfactory elevation accuracy. The height is calculated from the top-most camera. Notably, Nerfacto suffers from floating artifacts, whereas our approach effectively suppresses these floaters, resulting in a smoother and more coherent surface reconstruction.

Dataset	Model	AED (m) $\downarrow$	RED (m) $\downarrow$	[email protected] (%) $\uparrow$
Moon	Metashape	196.49	23.05	92.30
	Nerfacto	198.16	60.30	99.16
	Ours (w/o MVS sup.)	188.98	46.63	99.52
	Ours (with MVS sup.)	117.84	27.46	100.00
Mars	Metashape	1388.09	72.64	70.32
	Nerfacto	9549.76	2022.66	49.88

Table 1: Quantitative comparison with Metashape and Nerfacto on lunar and Mars datasets. AED and RED were computed after filling NaNs with the spatially nearest available point, while [email protected] counts a point as correct only if the relative elevation difference is below 0.1, treating NaNs as incorrect. Ours has shown better [email protected] compared to baselines.

Figure 7: Qualitative comparisons of DEMs using simulated lunar fisheye descent images, with a high-resolution mesh as ground truth. Our full model demonstrates superior coverage while maintaining high-quality reconstruction compared to two ablations.

Explore similar work

Deep Learning Aided Vision System for Planetary Rovers
Mar 26, 2026Lomash Relia, Jai G Singla, Amitabh +1Robotic Perception Metric Depth Estimation
Department of Computer Engineering Devang Patel Institute of Advanced Technology and Research Anand, India · Space Applications Centre Indian Space Research Organization Ahmedabad, India
Benchmarking stereo reconstruction for 3D printable Martian terrain models
Jun 9, 2026Josephine Wang3D Reconstruction Stereo Matching
MIT Cambridge, MA, USA
From Images2Mesh: A 3D Surface Reconstruction Pipeline for Non-Cooperative Space Objects
Apr 30, 2026Bala Prenith Reddy Gopu, Patrick Quinn, George M. Nehma +4Implicit Neural Representations 3D Reconstruction
Department of Aerospace, Physics and Space Sciences, Florida Institute of Technology, 150 W University Blvd, Melbourne, 32901, FL, USA · Creare LLC, Hanover, 03755, NH, USA

Mar 26, 2026J/K move · Enter open · S save

Deep Learning Aided Vision System for Planetary Rovers

Lomash Relia, Jai G Singla, Amitabh, Nitant Dube

Department of Computer Engineering Devang Patel Institute of Advanced Technology and Research Anand, India · Space Applications Centre Indian Space Research Organization Ahmedabad, India

Robotic Perception Metric Depth Estimation Depth Estimation

Open paper AskPDF arXiv

This study presents a vision system for planetary rovers, combining real-time perception with offline terrain reconstruction. The real-time module integrates CLAHE enhanced stereo imagery, YOLOv11n based object detection, and a neural network to estimate object distances. The offline module uses the Depth Anything V2 metric monocular depth estimation model to generate depth maps from captured images, which are fused into dense point clouds using Open3D. Real world distance estimates from the real time pipeline provide reliable metric context alongside the qualitative reconstructions. Evaluation on Chandrayaan 3 NavCam stereo imagery, benchmarked against a CAHV based utility, shows that the neural network achieves a median depth error of 2.26 cm within a 1 to 10 meter range. The object detection model maintains a balanced precision recall tradeoff on grayscale lunar scenes. This architecture offers a scalable, compute-efficient vision solution for autonomous planetary exploration.

Abstract

Explore similar work

Deep Learning Aided Vision System for Planetary Rovers

Benchmarking stereo reconstruction for 3D printable Martian terrain models

From Images2Mesh: A 3D Surface Reconstruction Pipeline for Non-Cooperative Space Objects