MANTLE: A Framework for Adaptive In-Situ Planetary Perception Using a Modular Uplink Principle

Pranav Durai; Gary Doran

cs.CVAug 28, 2026

MANTLE: A Framework for Adaptive In-Situ Planetary Perception Using a Modular Uplink Principle

Authors: Pranav Durai, Gary Doran

Organizations: Stanford Center for Innovation in In Vivo Imaging Stanford University School of Medicine 318 Campus Drive, Room S040, Stanford, CA 94305, USA · Jet Propulsion Laboratory California Institute of Technology 4800 Oak Grove Drive, Pasadena, CA 91109, USA

Abstract

Planetary surface exploration missions rely increasingly on autonomous robotic platforms capable of interpreting complex terrain to ensure safe navigation, enable targeted science, and improve operational efficiency, as demonstrated across past Mars missions from Viking through Perseverance. Among the key perception capabilities, landform classification provides contextual information for landing site selection and scientific analysis, while boulder segmentation supports hazard assessment and path planning. This paper presents MANTLE, a multi-task adaptive network for terrain and landform extraction. The model uses a shared DINOv2 backbone for high-level feature extraction with task-specific heads: a classification head for large-scale landform classification, and a segmentation head for pixel-wise boulder localization, each trained on curated datasets built respectively from HiRISE orbital imagery and MSL surface-level imagery. The classification head achieved a test accuracy of 92.56% across seven Martian terrain classes, while the segmentation head achieved a validation IoU of 0.753 and showed strong cross-sol generalization on a held-out test set from previously unseen rover traverses. A key advantage of MANTLE is its modular, extensible design, formalized here as the Modular Uplink Principle: only a shared, frozen backbone needs to remain onboard, while subsequent perception capabilities are trained on Earth as lightweight task-specific heads and uplinked without retraining the full model. This work demonstrates two such high-impact capabilities, terrain classification and boulder segmentation, as an initial realization of a framework built to support many more over a mission's lifetime. With this foundation, future explorers need not arrive on Mars fully formed, but can continue to learn, adapt, and grow more capable with every uplink.

Figures & tables

Fig. 1 : Example annotation from the AI4MARS dataset: (Left) raw MSL Navcam image, and (Right) corresponding polygon-based terrain segmentation labels [ 12 ] .

Panel of Figure 2 — Fig. 2 : Representative HiRISE image patches for each of the seven Martian terrain classes in the High-Resolution HiRISE Landform Dataset.

Fig. 3 : (Left) High-resolution HiRISE image captured by NASA MRO’s HiRISE payload: ESP_011283_2265_RED. (Right) Workflow for extracting ROI tiles from HiRISE images, showing the removal of black-pixel-dominated areas and the creation of high-resolution $1024\times 1024$ pixel patches.

Panel of Figure 4 — Fig. 4 : Representative image–mask pairs from the MSL Boulder Dataset across different sols, illustrating the diversity of boulder sizes, densities, and terrain contexts captured across the dataset.

Fig. 5 : Proposed MANTLE architecture: a shared frozen DINOv2 ViT-S/14 backbone (snowflake) feeds two independently trained task-specific heads (flames denote trainable modules). Orbital HiRISE tiles are resized to $224\times 224$ ( $16\times 16$ patch tokens) for the terrain classification head, while ground-level MSL images are resized to $784\times 784$ ( $56\times 56$ patch tokens) for the boulder segmentation head. Each head is trained separately against the frozen backbone, so new heads can be added without retraining or modifying the backbone.

Panel of Figure 6 — Fig. 6 : Spatial feature representations extracted by the shared, frozen DINOv2 backbone across both input modalities. (Top) Raw input imagery; (Bottom) corresponding patch-token activation magnitude heatmaps.

Panel of Figure 7 — Fig. 7 : Qualitative segmentation results across the IoU distribution on the held-out cross-sol test set (Sol $\geq$ 2305). Each panel shows the source MSL Mastcam image (left) and the predicted boulder probability heatmap (right), with warmer colors indicating higher predicted boulder likelihood. (a), (b), and (c) illustrate strong segmentation of large boulders in both densely and sparsely distributed arrangements; (d) illustrates the dominant failure mode such as missed small rocks and pebbles, which has negligible impact on rover navigation.

Fig. 8 : The Modular Uplink Principle. New perception capabilities are added to a deployed rover in three steps: (1) a task-specific head is trained and validated on Earth against a copy of the frozen, flight-qualified backbone, yielding a lightweight (KB–MB scale) module; (2) the trained head is uplinked via the Deep Space Network (DSN) and relayed to the surface by a Mars relay orbiter; and (3) the head executes onboard as a plug-in to the existing frozen backbone alongside previously deployed heads, enabling a new capability without retraining or modifying the backbone or onboard flight software.

Component	Task	Parameters	FP32 Size	INT8 Size (est.)
Shared backbone (DINOv2 ViT-S/14)	Feature extraction	$2.2\times 10^{7}$	${\sim}85$ MB	${\sim}22$ MB
Classification head (MANTLE)	Terrain classification	$100{,}359$	${\sim}0.4$ MB	${\sim}0.1$ MB
Segmentation head (MANTLE)	Boulder segmentation	$759{,}013$	${\sim}2.9$ MB	${\sim}0.76$ MB

TABLE I : Measured parameter counts and serialized sizes: shared backbone vs. the two MANTLE task-specific heads. INT8 sizes are estimated at 1 byte per parameter.

Space Robotics

Space robotics develops robots and autonomy for exploring, assembling, and maintaining spacecraft and planetary environments.

See all 62 papers

Global-Local Contextual Progressive Expansion Network for Martian Landslide Segmentation in Multimodal Remote Sensing Imagery
Sep 11, 2026Leo Thomas Ramos, Sidike Paheding, Abel A. Reyes-Angulo +4Multi-Angular Remote Sensing Remote Sensing Image Segmentation
Computer Vision Center, Universitat Aut`onoma de Barcelona, Barcelona, 08193, Spain · Department of Computer Science and Engineering, Fairfield University, Fairfield, CT 06824, USA · Michigan Technological University, Houghton, MI 49931, USA +3
DualSwinFusionSeg: Multimodal Martian Landslide Segmentation via Dual Swin Transformer with Multi-Scale Fusion and UNet++
Mar 14, 2026Shahriar Kabir, Abdullah Muhammed Amimul Ehsan, Istiak Ahmmed Rifti +1Multimodal Semantic Segmentation Image Segmentation
Bangladesh University of Engineering and Technology, Bangladesh · University of California, Riverside, USA
nnMNet: Baseline for Martian Terrain Semantic Segmentation
Aug 30, 2026Ming-Han Lee, Chi-Yeh ChenSemantic Segmentation Remote Sensing Image Segmentation
Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan 701, Taiwan

Sep 11, 2026J/K move · Enter open · S save

Global-Local Contextual Progressive Expansion Network for Martian Landslide Segmentation in Multimodal Remote Sensing Imagery

Leo Thomas Ramos, Sidike Paheding, Abel A. Reyes-Angulo, Rajaneesh A.+3

Computer Vision Center, Universitat Aut`onoma de Barcelona, Barcelona, 08193, Spain · Department of Computer Science and Engineering, Fairfield University, Fairfield, CT 06824, USA · Michigan Technological University, Houghton, MI 49931, USA+3

Multi-Angular Remote Sensing Remote Sensing Image Segmentation Semantic Segmentation Remote Sensing

Open paper AskPDF arXiv

Automated landslide segmentation on Mars is one of the important tasks for understanding its surface processes, and all will aid in future space exploration. However, it remains a relatively underexplored open challenge because landslide morphology is highly variable, foreground regions are often sparse or irregular, and orbital observations combine heterogeneous spectral and topographic cues. In this context, this work investigates the capability of deep learning to address Martian landslide segmentation through an extensive assessment of modern neural segmentation models. To the best of our knowledge, this is the first study to provide such a comprehensive exploration in this domain. We further propose TransCPLES, a U-shaped network that couples Contextual Progressive Layer Expansion feature extraction with Transformer-based contextual reasoning, enabling the model to capture local geomorphic patterns and broader spatial dependencies for more reliable landslide delineation. Experiments on MMLSv2, a seven-band multimodal Martian landslide dataset, show that TransCPLES achieves the best overall performance when evaluated on geographically distinct samples, with consistent delineation across different landslide extents, stable foreground discrimination, and a favorable balance between accuracy and computational cost compared with several state-of-the-art convolutional, attention-based, and Transformer-based segmentation models. With this work, we hope to provide a useful reference and encourage further research and development in deep learning for planetary remote sensing. Code will be available after publication.

Abstract

Space Robotics

Explore similar work

Global-Local Contextual Progressive Expansion Network for Martian Landslide Segmentation in Multimodal Remote Sensing Imagery

DualSwinFusionSeg: Multimodal Martian Landslide Segmentation via Dual Swin Transformer with Multi-Scale Fusion and UNet++

nnMNet: Baseline for Martian Terrain Semantic Segmentation