Space missions generate massive volumes of high-resolution orbital and surface imagery that far exceed the capacity for manual inspection. Detecting rare phenomena is scientifically critical, yet traditional supervised learning struggles due to scarce labeled examples and closed-world assumptions that prevent discovery of genuinely novel observations. In this work, we investigate Visual Anomaly Detection (VAD) as a framework for automated discovery in planetary exploration. We present the first empirical evaluation of state-of-the-art feature-based VAD methods on real planetary imagery, encompassing both orbital lunar data and Mars rover surface imagery. To support this evaluation, we introduce two benchmarks: (i) a lunar dataset derived from Lunar Reconnaissance Orbiter Camera Narrow Angle imagery, comprising of fresh and degraded craters as anomalies alongside normal terrain; and (ii) a Mars surface dataset designed to reflect the characteristics of rover-acquired imagery. We evaluate multiple VAD approaches with a focus on computationally efficient, edge-oriented solutions suitable for onboard deployment, applicable to both orbital platforms surveying the lunar surface and surface rovers operating on Mars. Our results demonstrate that feature-based VAD methods can effectively identify rare planetary surface phenomena while remaining feasible for resource-constrained environments. By grounding anomaly detection in planetary science, this work establishes practical benchmarks and highlights the potential of open-world perception systems to support a range of mission-critical applications, including tactical planning, landing site selection, hazard detection, bandwidth-aware data prioritization, and the discovery of unanticipated geological processes.
Figures & tables
Fig. 1: Images from the Lunar Dataset. (a) Representative normal terrain images. (b) Anomalous images. (c) Corresponding anomaly maps produced by PaDiM overlaid on the anomalous images, where red regions highlight the areas identified as anomalous. Despite being trained exclusively on normal terrain, PaDiM successfully localizes crater structures without any supervision.
Fig. 2: Images from the Mars Dataset. (a) Representative normal terrain images, (b) Anomalous images (c) Corresponding anomaly maps produced by PaDiM overlaid on the anomalous images, where red regions highlight the areas identified as anomalous.
Full
5% Positives
5% Positive + Contamination
Model
img-ROC
img-F1
AUC-PR
img-ACC
img-ROC
img-F1
AUC-PR
img-ROC
img-F1
AUC-PR
Padim
0.68
0.53
0.52
0.73
0.71
0.28
0.16
0.63
0.21
0.14
CFA
0.66
0.52
0.45
0.71
0.69
0.21
0.11
0.63
0.18
0.08
RD4AD
0.52
0.49
0.32
0.69
0.58
0.14
0.07
0.47
0.48
0.29
SimpleNet
0.51
0.48
0.33
0.69
0.51
0.14
0.06
0.46
0.11
0.05
PatchCore
0.62
0.47
0.40
0.59
0.65
0.21
0.11
0.64
0.18
0.07
TABLE I: Anomaly detection results on the Lunar dataset under three evaluation protocols (Fully, 5% Positives, and 5% Positives+Contamination) discussed in Section III-C , showing all metrics reported in Section IV-A : image-level AUROC (img-ROC), F1 score (img-F1), and area under the Precision-Recall curve (AUC-PR). Bold values indicate the best-performing model for each metric and setting.
Full
5% Positives
5% Positives + Contamination
Model
img-ROC
img-F1
AUC-PR
img-ROC
img-F1
AUC-PR
img-ROC
img-F1
AUC-PR
Padim
0.70
0.72
0.68
0.73
0.22
0.12
0.71
0.22
0.10
CFA
0.68
0.68
0.61
0.68
0.22
0.09
0.62
0.15
0.07
RD4AD
0.66
0.73
0.65
0.66
0.22
0.12
0.65
0.20
0.10
SimpleNet
0.60
0.69
0.59
0.61
0.16
0.10
0.62
0.19
0.12
PatchCore
0.81
0.79
0.75
0.80
0.25
0.16
0.40
0.11
0.04
TABLE II: Anomaly detection results on the Mars dataset under three evaluation protocols (Fully, 5% Positives, and 5% Positives+Contamination) discussed in Section III-C , showing all metrics reported in Section IV-A , image-level AUROC (img-ROC), F1 score (img-F1), and area under the Precision-Recall curve (AUC-PR). Bold values indicate the best-performing model for each metric and setting.
Model
Memory Footprint (MB)
Peak CPU Memory (MB)
Latency (ms)
PaDiM
10.7
1
40
CFA
5.7
1
18
RD4AD
116
3
41
SimpleNet
11.5
3
30
PatchCore
17.4
1
28
STFPM
10.4
3
24
TABLE III: Computational profile of evaluated models.
The Lunar Reconnaissance Orbiter (LRO) has been collecting high-resolution images (at around 0.5-2 meters per pixel linearly with its Narrow Angle Camera) of the Moon since 2009, amassing a large dataset of images and offering researchers the opportunity to study the surface of the Moon at unprecedented scale. Here, we aim to test the abilities of the Beta-Variational Autoencoder (VAE) created by Lesnikowski et al. (2024), an unsupervised learning model which identifies anomalous features across the Moon's surface, locating not only scientifically useful geologic formations such as rockfall deposits, fresh impact craters, irregular mare patches, or volcanic pits/collapsed lava tubes, but also artificial objects such as landed spacecraft. This investigation further gauged the model's ability to locate anomalous surface features, successfully recovering two places of interest (Plaskett Crater and Paracelsus C Crater) and numerous landed technological assets at a statistically significant rate.
Cameron Kelahan, Daniel Angerhausen, Adam Lesnikowski +1
University of Warwick, Coventry, United Kingdom CV4 7AL · Universit`a degli Studi di Padova, Via 8 Febbraio, 2 - 35122 Padova, Italy · James Madison University, 800 South Main Street, Harrisonburg, VA 22807 +3
Recent advances in multimodal vision-language models (VLMs) have enabled joint reasoning over visual and textual information, yet their application to planetary science remains largely unexplored. A key hindrance is the absence of large-scale datasets that pair real planetary imagery with detailed scientific descriptions. In this work, we introduce LLaVA-LE (Large Language-and-Vision Assistant for Lunar Exploration), a vision-language model specialized for lunar surface and subsurface characterization. To enable this capability, we curate a new large-scale multimodal lunar dataset, LUCID (LUnar Caption Image Dataset) consisting of 96k high-resolution panchromatic images paired with detailed captions describing lunar terrain characteristics, and 81k question-answer (QA) pairs derived from approximately 20k images in the LUCID dataset. Leveraging this dataset, we fine-tune LLaVA using a two-stage training curriculum: (1) concept alignment for domain-specific terrain description, and (2) instruction-tuned visual question answering. We further design evaluation benchmarks spanning multiple levels of reasoning complexity relevant to lunar terrain analysis. Evaluated against GPT and Gemini judges, LLaVA-LE achieves a 3.3x overall performance gain over Base LLaVA and 2.1x over our Stage 1 model, with a reasoning score of 1.070, exceeding the judge's own reference score, highlighting the effectiveness of domain-specific multimodal data and instruction tuning to advance VLMs in planetary exploration. Code is available at https://github.com/OSUPCVLab/LLaVA-LE.
Visual Place Recognition (VPR) is an important means for loop closure detection in visual Simultaneous Localization and Mapping (SLAM). While state-of-the-art VPR and SLAM approaches excel in urban and feature-rich natural environments, the vast and feature-poor environments of extraterrestrial surfaces like on the Moon or Mars pose severe challenges for existing approaches. This paper presents a novel benchmark, VPR-Space, to evaluate VPR capabilities on extraterrestrial surfaces. There are three contributions: (1) We collected sixteen VPR database-query datasets based on existing data from Space Analog Domains (SAD), terrestrial environments that resemble properties of the Moon or Mars. The collection includes a range of challenges regarding environment, viewpoint change, and illumination conditions. (2) To address the ill-posed nature of VPR, we propose a novel approach to generate the ground-truth information in VPR benchmarks. Instead of defining that a query and datbase image show the same place if and only if their camera poses are within fixed position and viewing direction thresholds, we propose to make the decision dependent on whether the match would be valuable in the downstream SLAM task. To avoid the overhead and restrictions of evaluating VPR in-the-loop within SLAM systems, we provide VPR ground-truth (GT) matrices based on the performance of an ensemble of state-of-the-art Relative Pose Estimators (RPE). Under this GT we require a VPR method to identify exactly those image pairs for which a subsequently applied RPE is expected to provide a valuable pose estimate in a SLAM context. (3) We evaluated nine state-of-the-art VPR approaches. The results show considerable room for improvement and indicate directions for future research. We provide a pre-release version of the dataset on github: https://github.com/rfechner/VPR-Space
Richard Fechner, Peer Neubert
Institute for Computational Visualistics, University of Koblenz