Organizations: Department of Space Systems Engineering, Faculty of Aerospace Engineering, Kluyverweg 1, HS Delft, 2629, Netherlands · Airbus Defence and Space GmbH, Claude-Dornier Straße, Immenstaad am Bodensee, 88090, Germany
This study details an artificial intelligence (AI)-based methodology for the semantic segmentation of space camera faults. Specifically, we address the segmentation of straylight effects induced by solar presence around the camera's Field of View (FoV). Anomalous images are sourced from our published dataset. Our approach emphasizes generalization across diverse flare textures, leveraging pre-training on a public dataset (Flare7k++) including flares in various non-space contexts to mitigate the scarcity of realistic space-specific data. A DeepLabV3 model with MobileNetV3 backbone performs the segmentation task. The model design targets deployment in spacecraft resource-constrained hardware. Finally, based on a proposed interface between our model and the onboard navigation pipeline, we develop custom metrics to assess the model's performance in the system-level context.
Figures & tables
Property
Value
Number of Images
1000
Training/Validation Split
80/20
Input Size
1024×1024×3
Color Bit Depth
8 bits per channel
Number of Classes
2 ( Nominal , Straylight )
Table 1: Custom dataset properties
Figure 1: Binary segmentation of all dataset fault classes with DeepLabV3. Training data is described in Table 1 . Training run for 500 epochs with Binary Cross-entropy loss and Adam optimizer with 0.0001 initial learning rate.
Figure 2: Sample images synthetically generated and relative segmentation mask from Flare7k++ [ 11 ]
Property
Value
Number of Images
7962
Input Size
512×512×3
Color Bit Depth
8 bits per channel
Number of Classes
2 ( Nominal , Straylight )
Table 2: Flare7k++ dataset properties [ 11 ]
Figure 3: Binary segmentation of straylight. Pre-trained model inferenced on custom test dataset
Figure 4: Binary segmentation of straylight. Fine-tuned model inferenced on custom dataset. Training takes 500 epochs, employs Binary Cross-entropy loss and Adam optimizer with 0.0001 initial learning rate.
Figure 5: Onboard camera sensor data processing scheme
Precision
Recall
mIoU
Pre-trained Model
0.259
0.403
0.188
Fine-tuned Model
0.908
0.958
0.873
Table 3: Segmentation results for the Straylight model.
Figure 6: More segmentation results for the pre-trained (left) and fine-tuned (right) models
Table 4: Custom metrics for image faults segmentation
PaP
PaR
PamIoU
Standard Ground Truth
0.849
0.308
0.855
Smoothed Ground Truth
0.832
0.991
0.783
Table 5: Comparison between the proposed custom metrics, before and after Gaussian smoothing, evaluated on the fine-tuned model.
Figure 7: Connected regions identified by the regionprops algorithm are marked by red crosses. Standard ground truth (left) and Gaussian smoothed (right)
PaP
Precision
PaR
Recall
PamIoU
mIoU
Pre-trained Model
0.456
0.259
0.769
0.403
0.313
0.188
Fine-tuned Model
0.832
0.908
0.991
0.958
0.783
0.873
Table 6: Results for the Straylight model, as evaluated on custom metrics and conventional segmentation metrics