HiPredict: Predicting the missing HiRISE RED4 spectral channel with machine learning

Ardan Suphi; Ben Moseley; Gareth S Collins; Valentin T Bickel; Vidhya G Rangarajan

cs.LGOct 16, 2026

HiPredict: Predicting the missing HiRISE RED4 spectral channel with machine learning

Authors: Ardan Suphi, Ben Moseley, Gareth S Collins, Valentin T Bickel, Vidhya G Rangarajan

Organizations: Department of Earth Science and Engineering, Imperial College London, London, UK · Center for Space and Habitability, University of Bern, Bern, Switzerland · Institute of Space Research, German Aerospace Center (DLR), Berlin, Germany

Abstract

The High Resolution Imaging Science Experiment (HiRISE) onboard the Mars Reconnaissance Orbiter (MRO) acquires imagery of Mars at 25--30 cm/pixel, supporting studies of martian geology, climate history, and landing-site characterization. Its 14-detector focal plane assembly includes several overlapping detectors that enable the acquisition of three-channel colour-infrared images, which are important for identifying compositional variations and studying active surface processes. In 2023, a detector in the center of the focal plane assembly (RED4) began failing intermittently, creating a gap in acquired imagery and reducing the instrument's colour-imaging capability by half. The HiRISE processing pipeline provides a synthetic red (SYN4) channel obtained by linearly scaling the co-located infrared (IR10) channel. However, this approach cannot capture observed non-linear relationships between the colour channels, limiting its accuracy and applicability. Here, we present HiPredict, a machine-learning framework that reconstructs RED4 from co-located IR10 and blue-green (BG12) imagery, neighbouring red-channel observations, and acquisition metadata. The model employs a U-Net architecture to learn non-linear spatiotemporal-spectral relationships between these inputs and is trained on 81,717 training samples, representing 13,625 observations. Evaluated on 23,348 test samples, representing 3,892 pre-failure observations, HiPredict reconstructs RED4 with a Normalised Mean Absolute Error of 1.1%, substantially outperforming SYN4 (3.1%). The reconstructed products improve colour image recovery and more closely represent features of interest such as icy craters, stratigraphy exposed along ridge crests and chloride deposits. We release HiPredict as an open software tool. This work demonstrates the potential of machine learning to mitigate data loss as planetary missions age and naturally degrade.

Figures & tables

Panel of Figure 2 — Figure 2: A comparison showing the impact of the two CCD failures. As the BG12 and IR10 detectors overlap with RED4, this band can be predicted but as there is nothing which images in the RED9 spatial area, this data cannot be recovered. Observation ESP_023753_1660 ( $-13.8236^{\circ},309.643^{\circ}\text{E}$ ).

Panel of Figure 3 — Figure 3: Overview of HiPredict - (A) Overview of the HiPredict architecture showing the inputs and two model heads. The inputs, top to bottom, are the two band images from the target location, the overlap images from neighbouring CCDs, and metadata. (B) The spatial head, a U-Net based architecture. (C) The brightness head, a lightweight convolutional operator.

	TDI (%)			Binning (%)
	128	64	Other	1	2	Other
Train	75.2	24.8	0.0	34.0	66.0	0.0
True	70.2	24.4	5.3	30.6	65.8	3.6

Table 1: TDI and binning distributions of the training and full HiRISE datasets.

	NMAE (%) $\downarrow$		SSIM $\uparrow$		PWT 1% $\uparrow$
Name	mean	median	mean	median	mean	median
Linear	4.012	3.508	0.542	0.596	4.926	2.632
Linear with meta	4.234	3.594	0.544	0.588	4.851	2.634
Basic UNet	2.064	1.339	0.687	0.747	13.841	12.750
Customised UNet	6.453	2.990	0.632	0.690	6.729	2.924
SYN4	3.118	1.545	0.663	0.722	11.069	9.148

Table 2: Performance metrics for unseen data. Arrows indicate if higher or lower values are better.

Panel of Figure 4 — Figure 4: Top row: True HiRISE images using the original RED4. Bottom row: HiPredict predictions for the same areas. Images are plotted in ‘IRB’ format, meaning the infrared/red/blue-green bands are mapped to the red/green/blue channels to create the colour image, which helps to highlight subtle surface composition differences. Left column: ESP_055232_1595 ( $-20.1668^{\circ},226.817^{\circ}\text{E}$ ). Middle column: ESP_056705_1440 ( $-35.4419^{\circ},342.328^{\circ}\text{E}$ ). Right column: ESP_012940_1655 ( $-14.0974^{\circ},318.282^{\circ}\text{E}$ ).

Figure 5: Using high SYN4 error as a proxy for colourful and complex images, we can evaluate HiPredict performance on this subset.

Figure 6: Log-transformed SHAP analysis of six most consequential metadata values provided to the model, excluding the top 10 outliers. Points to the left of center have a positive impact on model performance, points to the right a negative impact. Red points indicate a larger value for that metadata item and blue a lower value. Verticality indicates density of data points. The complete SHAP plot can be seen in Supplemental Information S6

Panel of Figure 7 — Figure 7: A deeper comparison of the models using icy impacts as examples.

Image Reconstruction

Image reconstruction recovers images, volumes, or scenes from incomplete, noisy, indirect, or compressed measurements using physical models and learned priors.

See all 286 papers

MarsRecon: Self-Supervised and Multimodal Surface Representations for Mars
Sep 17, 2026Akshay Naik, Marius F. R. Juston, Jay MahajanGeospatial Representation Learning Self-Supervised Visual Representation Learning
Department of Electrical and Computer Engineering The Grainger College of Engineering University of Illinois Urbana-Champaign Urbana, IL 61801-3080, USA · Department of Computer Science University of Illinois Urbana-Champaign Urbana, IL 61801-3080, USA
MARTIAN: A Rendering Framework for Aerial Mars Imagery from HiRISE Orbital Data
May 28, 2026Dario Pisanti, Georgios GeorgakisVisual Navigation Aerial Robotics
Space Robotics Research Group, SnT, University of Luxembourg. · Jet Propulsion Laboratory, California Institute of Technology.
MarsFM: Shading-Regularized Flow Matching for Martian Relief Estimation
Sep 17, 2026Marius F. R. JustonRemote Sensing Image Understanding Flow Matching
Department of Industrial and Enterprise Systems Engineering The Grainger College of Engineering University of Illinois Urbana-Champaign, Urbana IL 61801-3080, USA

Sep 17, 2026J/K move · Enter open · S save

MarsRecon: Self-Supervised and Multimodal Surface Representations for Mars

Akshay Naik, Marius F. R. Juston, Jay Mahajan

Department of Electrical and Computer Engineering The Grainger College of Engineering University of Illinois Urbana-Champaign Urbana, IL 61801-3080, USA · Department of Computer Science University of Illinois Urbana-Champaign Urbana, IL 61801-3080, USA

Geospatial Representation Learning Self-Supervised Visual Representation Learning Cross-Modal Representation Learning Remote Sensing

Open paper AskPDF arXiv

High-resolution orbital imagery offers a rich record of the Martian surface, but sparse geological labels limit supervised representation learning. We present MarsRecon, a geospatially aware pipeline for learning visual and multimodal representations from HiRISE observations of Olympus Mons. The pipeline calibrates NASA Planetary Data System products, extracts valid georeferenced patches, and trains a masked autoencoder on unlabeled imagery. Increasing input resolution and filtering invalid tokens reduced held-out reconstruction loss from 0.1751 to 0.1342 in the principal Stage A model series. We then freeze the visual encoder and align its features with observation text, coordinates, and local--global image context. The strongest current local-primary model achieves image-to-text recall@10 of 0.3787, text-to-image recall@10 of 0.9161, and local-to-global recall@10 of 0.4350 on the held-out test split. These results establish a working Mars-specific pretraining and retrieval pipeline; further crop-overlap controls and downstream geological evaluations are needed to assess the broader utility of its embeddings.

Abstract

Image Reconstruction

Explore similar work

MarsRecon: Self-Supervised and Multimodal Surface Representations for Mars

MARTIAN: A Rendering Framework for Aerial Mars Imagery from HiRISE Orbital Data

MarsFM: Shading-Regularized Flow Matching for Martian Relief Estimation