cs.CVOct 16, 2026

OrTeN: Neural Inverse Graphics for Joint Terrain and Line-scan Camera Trajectory Reconstruction

Authors: Josef Biberstein, Sertac Karaman

Organizations: Massachusetts Institute of Technology

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Digital terrain maps (DTMs) are used for landing site selection, terrain-relative navigation, and geological analysis in planetary exploration. Existing state-of-practice reconstruction techniques for creating DTMs from satellite imagery use multi-view stereo, requiring complex pipelines for dealing with camera error, changing illumination, and images with limited disparity. This work introduces Orbital Terrain Networks (OrTeN), a DTM construction method that uses neural inverse rendering to jointly optimize a textured height map and orbital camera corrections from line-scan satellite images. OrTeN operates directly on calibrated line-scan images with pixel values expressed as radiance factor (I/F) and corrects both broad trajectory pose inconsistencies and high-frequency pointing jitter jointly with surface reconstruction during training. Surface-projected patch normalized cross-correlation (NCC) enforces consistency between observations of the same terrain, refining an initial rough reconstruction. A semi-empirical reflectance model and shadow rendering are used to fit surface appearance from multi-date imagery with differing solar illumination. OrTeN requires only calibrated line-scan images and initial camera poses and does not use any structural priors including ranging/depth measurements or initial rough DTMs. OrTeN is evaluated on four planetary imagery datasets. Aligned terrain RMSEs are 2.35 m at Jezero Crater, 2.82 m at Gale Crater, 16.18 m at Olympus Mons, and 0.43 m at McGetchin Crater.

Figures & tables

Figure 2: Key elements of the OrTeN pipeline. Two linescan acquisitions i,ji,j observe overlapping terrain swaths. The dashed trajectory indicates the original extrinsics, which are corrected during bundle adjustment. Bundle adjustment refines position and attitude, including angular jitter. Terrain networks predict height, albedo, material mixture, and a Sun-dependent ambient factor. Gray-to-red camera-ray samples illustrate interval opacity αj\alpha_{j} , with transition width controlled by β\beta . Sun-directed opacity integration determines visibility VV . Source image patches are lifted onto the current height field and compared with neighbor observations through normalized cross-correlation ρ\rho , constraining terrain and cameras jointly.
Figure 3: OrTeN’s architecture. Learned networks are blue, optimized parameters are red, and analytic operations are gray. Layer boxes give output dimensions, and arrow colors indicate activations. One spatial encoding [ 30 ] feeds the height, albedo, and Lunar–Lambert material outputs. The ambient network uses a fourth-order spherical harmonic (SH) encoding of the Sun direction. Height also supplies normals and solar visibility VηV_{\eta} . Proposal sampling is omitted for clarity.
DatasetGSDBandReference DTMAcc. / prec.
 (m/px)(nm)(post spacing)(m)
Jezero a Crater6.41500–700Mars 2020 (1 m) [ 16 ]∼\sim 10–20 / ∼\sim 0.1–0.4 [ 20 ]
Gale a Crater6.23500–700Gale mosaic (1 m) [ 10 ]∼\sim 10–20 / ∼\sim 0.1–0.4 [ 20 ]
Olympus a Mons5.56500–700Audited ASP (11.12 m)— / 11.68 ‡
McGetchin b Crater0.84400–750NAC NEW200CRAT3 (3 m) [ 5 ]1.51 / 1.46 [ 5 ]
Table 1: Dataset instrument and DTM reference overview. Instruments are a CTX [ 25 ] and b LROC NAC [ 32 ] . Acc. / prec. denotes vertical accuracy / precision. GSD is estimated by intersecting adjacent pixel rays from initial cameras with the local ENU xyxy plane. ‡ For Olympus Mons, precision was estimated from the RMS elevation difference between two stereo DTMs generated from image pairs with no cameras in common during construction of the ASP reference used to evaluate OrTeN.
Figure 4: McGetchin Crater with and without BA. Renderings and crater errors use shared scales. Boxes use the baseline camera. RMSE a : 0.43 / 10.36 m. PSNR: 35.13 / 21.04 dB.
Scene / methodRMSE a / MAE a / slope RMSE aPSNR / SSIM
 (m / m / ∘ )(dB / —)
Jezero Crater
OrTeN2.35 / 1.61 / 1.8728.20 / 0.771
EO-NeRF + depth62.08 / 47.61 / 39.1525.26 / 0.598
ASP4.90 / 3.56 / 4.81— / —
Gale Crater
Table 2: DTM reconstruction comparison. Best values per scene are bold. Height errors are evaluated across each method’s valid portion of the fixed evaluation footprint. Slope uses local quadratic fits. PSNR and SSIM use held-out pixels and a fixed intensity range per scene. ASP produces no visual reconstruction.
 RMSE a / PSNR
ConfigurationJezeroGaleOlympusMcGetchin
Baseline2.35 / 28.202.82 / 31.3716.18 / 29.710.43 / 35.13
No BA4.26 / 26.409.25 / 28.3930.01 / 24.3110.36 / 21.04
No jitter2.40 / 28.212.76 / 29.5616.47 / 30.340.46 / 34.64
No NCC16.80 / 29.8542.22 / 29.0967.40 / 22.060.52 / 34.63
No shadows2.32 / 28.062.90 / 31.3115.81 / 29.340.43 / 35.21
Table 3: Core OrTeN ablations. No BA also removes jitter. No shadows removes attached and cast visibility and the shadow prior.
Figure 5: Jezero Crater with and without NCC. Green boxes locate DTM error crops using each run’s learned camera extrinsics and rigid alignment to the reference terrain. RMSE a : 2.35 / 16.80 m. PSNR: 28.20 / 29.85 dB.
Figure 6: Olympus Mons shadow ablation. Cyan marks shadows detected by the first-valley prior. Images share an intensity scale.
More in this topic

3D Reconstruction

3D reconstruction recovers a scene’s shape, appearance, and camera motion from images, video, depth, or other sensor measurements.

See all 955 papers

Explore similar work

CardsList
Apr 14, 2026

Neural 3D Reconstruction of Planetary Surfaces from Descent-Phase Wide-Angle Imagery

Digital elevation modeling of planetary surfaces is essential for studying past and ongoing geological processes. Wide-angle imagery acquired during spacecraft descent promises to offer a low-cost option for high-resolution terrain reconstruction. However, accurate 3D reconstruction from such imagery is challenging due to strong radial distortion and limited parallax from vertically descending, predominantly nadir-facing cameras. Conventional multi-view stereo exhibits limited depth range and reduced fidelity under these conditions and also lacks domain-specific priors. We present the first study of modern neural reconstruction methods for planetary descent imaging. We also develop a novel approach that incorporates an explicit neural height field representation, which provides a strong prior since planetary surfaces are generally continuous, smooth, solid, and free from floating objects. This study demonstrates that neural approaches offer a strong and competitive alternative to traditional multi-view stereo (MVS) methods. Experiments on simulated descent sequences over high-fidelity lunar and Mars terrains demonstrate that the proposed approach achieves increased spatial coverage while maintaining satisfactory estimation accuracy.

Melonie de Almeida, George Brydon, Divya M. Persaud +2

University of Glasgow

3D ReconstructionMulti-View 3D Reconstruction
AskPDF
Mar 23, 2026

SatGeo-NeRF: Geometrically Regularized NeRF for Satellite Imagery

We present SatGeo-NeRF, a geometrically regularized NeRF for satellite imagery that mitigates overfitting-induced geometric artifacts observed in current state-of-the-art models using three model-agnostic regularizers. Gravity-Aligned Planarity Regularization aligns depth-inferred, approximated surface normals with the gravity axis to promote local planarity, coupling adjacent rays via a corresponding surface approximation to facilitate cross-ray gradient flow. Granularity Regularization enforces a coarse-to-fine geometry-learning scheme, and Depth-Supervised Regularization stabilizes early training for improved geometric accuracy. On the DFC2019 satellite reconstruction benchmark, SatGeo-NeRF improves the Mean Altitude Error by 13.9% and 11.7% relative to state-of-the-art baselines such as EO-NeRF and EO-GS.

Valentin Wagner, Sebastian Bullinger, Michael Arens +1

Fraunhofer IOSB, Ettlingen, Germany · Karlsruhe Institute of Technology, Karlsruhe, Germany

3D ReconstructionNeural Radiance FieldsRemote Sensing
AskPDF
May 18, 2026

NeRF-based Spacecraft Reconstruction from Monocular Imagery Under Illumination Variability and Pose Uncertainty

Autonomous rendezvous and proximity operations around uncooperative, unknown spacecraft are critical for active debris removal and on-orbit servicing missions. A key component of such operations is the offline reconstruction of a 3D model of the target from a set of 2D images. This task is challenging due to two main factors. First, in-orbit illumination conditions exhibit considerable variability, and change rapidly over time. Second, the inaccuracy of pose information in the images, results in 3D reconstruction uncertainty. To overcome these challenges, we propose to extend Neural Radiance Fields with per-image degrees of freedom: a learnable appearance embedding that captures the illumination conditions specific to each image, and an image-specific pose correction term that refines its noisy pose label to increase 3D consistency across images. These parameters add minimal complexity, as they are learned jointly with the NeRF, yet they substantially improve robustness to illumination variability and pose inaccuracies. We validate our approach on three image sets representative of in-orbit operations, demonstrating its effectiveness for offline reconstruction and highlighting its suitability for online reconstruction, an open problem in the field.

Antoine Legrand, Renaud Detry, Christophe De Vleeschouwer

Aerospacelab, Mont-Saint-Guibert, 1435, Belgium · ICTEAM, UCLouvain, Louvain-la-Neuve, 1348, Belgium

Distribution Shift3D ReconstructionMonocular 3D ReconstructionCamera Pose EstimationNeural Radiance Fields
AskPDF