OrTeN: Neural Inverse Graphics for Joint Terrain and Line-scan Camera Trajectory Reconstruction

Josef Biberstein; Sertac Karaman

cs.CVOct 16, 2026

OrTeN: Neural Inverse Graphics for Joint Terrain and Line-scan Camera Trajectory Reconstruction

Authors: Josef Biberstein, Sertac Karaman

Organizations: Massachusetts Institute of Technology

Abstract

Digital terrain maps (DTMs) are used for landing site selection, terrain-relative navigation, and geological analysis in planetary exploration. Existing state-of-practice reconstruction techniques for creating DTMs from satellite imagery use multi-view stereo, requiring complex pipelines for dealing with camera error, changing illumination, and images with limited disparity. This work introduces Orbital Terrain Networks (OrTeN), a DTM construction method that uses neural inverse rendering to jointly optimize a textured height map and orbital camera corrections from line-scan satellite images. OrTeN operates directly on calibrated line-scan images with pixel values expressed as radiance factor (I/F) and corrects both broad trajectory pose inconsistencies and high-frequency pointing jitter jointly with surface reconstruction during training. Surface-projected patch normalized cross-correlation (NCC) enforces consistency between observations of the same terrain, refining an initial rough reconstruction. A semi-empirical reflectance model and shadow rendering are used to fit surface appearance from multi-date imagery with differing solar illumination. OrTeN requires only calibrated line-scan images and initial camera poses and does not use any structural priors including ranging/depth measurements or initial rough DTMs. OrTeN is evaluated on four planetary imagery datasets. Aligned terrain RMSEs are 2.35 m at Jezero Crater, 2.82 m at Gale Crater, 16.18 m at Olympus Mons, and 0.43 m at McGetchin Crater.

Figures & tables

Figure 2: Key elements of the OrTeN pipeline. Two linescan acquisitions $i,j$ observe overlapping terrain swaths. The dashed trajectory indicates the original extrinsics, which are corrected during bundle adjustment. Bundle adjustment refines position and attitude, including angular jitter. Terrain networks predict height, albedo, material mixture, and a Sun-dependent ambient factor. Gray-to-red camera-ray samples illustrate interval opacity $\alpha_{j}$ , with transition width controlled by $\beta$ . Sun-directed opacity integration determines visibility $V$ . Source image patches are lifted onto the current height field and compared with neighbor observations through normalized cross-correlation $\rho$ , constraining terrain and cameras jointly.

Panel of Figure 3 — Figure 3: OrTeN’s architecture. Learned networks are blue, optimized parameters are red, and analytic operations are gray. Layer boxes give output dimensions, and arrow colors indicate activations. One spatial encoding [ 30 ] feeds the height, albedo, and Lunar–Lambert material outputs. The ambient network uses a fourth-order spherical harmonic (SH) encoding of the Sun direction. Height also supplies normals and solar visibility $V_{\eta}$ . Proposal sampling is omitted for clarity.

Dataset	GSD	Band	Reference DTM	Acc. / prec.
	(m/px)	(nm)	(post spacing)	(m)
Jezero a Crater	6.41	500–700	Mars 2020 (1 m) [ 16 ]	$\sim$ 10–20 / $\sim$ 0.1–0.4 [ 20 ]
Gale a Crater	6.23	500–700	Gale mosaic (1 m) [ 10 ]	$\sim$ 10–20 / $\sim$ 0.1–0.4 [ 20 ]
Olympus a Mons	5.56	500–700	Audited ASP (11.12 m)	— / 11.68 ‡
McGetchin b Crater	0.84	400–750	NAC NEW200CRAT3 (3 m) [ 5 ]	1.51 / 1.46 [ 5 ]

Table 1: Dataset instrument and DTM reference overview. Instruments are a CTX [ 25 ] and b LROC NAC [ 32 ] . Acc. / prec. denotes vertical accuracy / precision. GSD is estimated by intersecting adjacent pixel rays from initial cameras with the local ENU

xy

plane. ‡ For Olympus Mons, precision was estimated from the RMS elevation difference between two stereo DTMs generated from image pairs with no cameras in common during construction of the ASP reference used to evaluate OrTeN.

Figure 4: McGetchin Crater with and without BA. Renderings and crater errors use shared scales. Boxes use the baseline camera. RMSE a : 0.43 / 10.36 m. PSNR: 35.13 / 21.04 dB.

Jezero Crater
Scene / method	RMSE a / MAE a / slope RMSE a	PSNR / SSIM
	(m / m / ∘ )	(dB / —)
OrTeN	2.35 / 1.61 / 1.87	28.20 / 0.771
EO-NeRF + depth	62.08 / 47.61 / 39.15	25.26 / 0.598
ASP	4.90 / 3.56 / 4.81	— / —
Gale Crater

Table 2: DTM reconstruction comparison. Best values per scene are bold. Height errors are evaluated across each method’s valid portion of the fixed evaluation footprint. Slope uses local quadratic fits. PSNR and SSIM use held-out pixels and a fixed intensity range per scene. ASP produces no visual reconstruction.

	RMSE a / PSNR
Configuration	Jezero	Gale	Olympus	McGetchin
Baseline	2.35 / 28.20	2.82 / 31.37	16.18 / 29.71	0.43 / 35.13
No BA	4.26 / 26.40	9.25 / 28.39	30.01 / 24.31	10.36 / 21.04
No jitter	2.40 / 28.21	2.76 / 29.56	16.47 / 30.34	0.46 / 34.64
No NCC	16.80 / 29.85	42.22 / 29.09	67.40 / 22.06	0.52 / 34.63
No shadows	2.32 / 28.06	2.90 / 31.31	15.81 / 29.34	0.43 / 35.21

Table 3: Core OrTeN ablations. No BA also removes jitter. No shadows removes attached and cast visibility and the shadow prior.

Figure 5: Jezero Crater with and without NCC. Green boxes locate DTM error crops using each run’s learned camera extrinsics and rigid alignment to the reference terrain. RMSE a : 2.35 / 16.80 m. PSNR: 28.20 / 29.85 dB.

Figure 6: Olympus Mons shadow ablation. Cyan marks shadows detected by the first-valley prior. Images share an intensity scale.

3D Reconstruction

3D reconstruction recovers a scene’s shape, appearance, and camera motion from images, video, depth, or other sensor measurements.

See all 955 papers

Neural 3D Reconstruction of Planetary Surfaces from Descent-Phase Wide-Angle Imagery
Apr 14, 2026Melonie de Almeida, George Brydon, Divya M. Persaud +23D Reconstruction Multi-View 3D Reconstruction
University of Glasgow
SatGeo-NeRF: Geometrically Regularized NeRF for Satellite Imagery
Mar 23, 2026Valentin Wagner, Sebastian Bullinger, Michael Arens +13D Reconstruction Neural Radiance Fields
Fraunhofer IOSB, Ettlingen, Germany · Karlsruhe Institute of Technology, Karlsruhe, Germany
NeRF-based Spacecraft Reconstruction from Monocular Imagery Under Illumination Variability and Pose Uncertainty
May 18, 2026Antoine Legrand, Renaud Detry, Christophe De VleeschouwerDistribution Shift 3D Reconstruction
Aerospacelab, Mont-Saint-Guibert, 1435, Belgium · ICTEAM, UCLouvain, Louvain-la-Neuve, 1348, Belgium

Apr 14, 2026J/K move · Enter open · S save

Neural 3D Reconstruction of Planetary Surfaces from Descent-Phase Wide-Angle Imagery

Melonie de Almeida, George Brydon, Divya M. Persaud, John H. Williamson+1

University of Glasgow

3D Reconstruction Multi-View 3D Reconstruction

Open paper AskPDF arXiv

Digital elevation modeling of planetary surfaces is essential for studying past and ongoing geological processes. Wide-angle imagery acquired during spacecraft descent promises to offer a low-cost option for high-resolution terrain reconstruction. However, accurate 3D reconstruction from such imagery is challenging due to strong radial distortion and limited parallax from vertically descending, predominantly nadir-facing cameras. Conventional multi-view stereo exhibits limited depth range and reduced fidelity under these conditions and also lacks domain-specific priors. We present the first study of modern neural reconstruction methods for planetary descent imaging. We also develop a novel approach that incorporates an explicit neural height field representation, which provides a strong prior since planetary surfaces are generally continuous, smooth, solid, and free from floating objects. This study demonstrates that neural approaches offer a strong and competitive alternative to traditional multi-view stereo (MVS) methods. Experiments on simulated descent sequences over high-fidelity lunar and Mars terrains demonstrate that the proposed approach achieves increased spatial coverage while maintaining satisfactory estimation accuracy.

Abstract

3D Reconstruction

Explore similar work

Neural 3D Reconstruction of Planetary Surfaces from Descent-Phase Wide-Angle Imagery

SatGeo-NeRF: Geometrically Regularized NeRF for Satellite Imagery

NeRF-based Spacecraft Reconstruction from Monocular Imagery Under Illumination Variability and Pose Uncertainty