Cross-Dataset Transfer and Unknown-Class Detection in Imbalanced SAR Ship Classification

Ch Muhammad Awais; Marco Reggiannini; Davide Moroni; Giulio Del Corso

cs.CVSep 15, 2026

Cross-Dataset Transfer and Unknown-Class Detection in Imbalanced SAR Ship Classification

Authors: Ch Muhammad Awais, Marco Reggiannini, Davide Moroni, Giulio Del Corso

Organizations: Institute of Science and Technology, National Research Council, Pisa, Italy

Abstract

Ship classification from Synthetic Aperture Radar (SAR) imagery is a critical computer vision task, yet the robustness of models under deployment shifts remains unclear. While models are often trained on one dataset and deployed on another, we lack a comprehensive understanding of their cross-dataset generalization. To address this, we evaluate six pretrained models on two SAR ship datasets in three settings: in-domain classification, cross-dataset transfer, and unknown-class detection. For unknown detection, we hold out all classes one at a time. In-domain, SARDet100K gives the best balanced accuracy on both datasets (73.4% on FUSARShip and 53.2% on OpenSARShip). In cross-dataset transfer, we observe strong failures: some models show moderate accuracy but near-chance balanced accuracy (for example, 64.5% accuracy but 33.3% balanced accuracy for OpenSARShip to FUSARShip). In unknown detection, performance depends on the held-out class and dataset, while MC-dropout variance is often close to random. These findings show that cross-dataset generalization in SAR remains limited and that task-specific uncertainty scores are often more informative than MC-dropout variance for held-out-class detection, although their relative ranking depends on the dataset and held-out class.

Figures & tables

Figure 1: Overview of the evaluation pipeline used in this study. SAR images are encoded with an Earth Observation (EO) foundation model to extract representations, followed by training of a shared classifier head. The trained model is then evaluated in three settings: in-domain, cross-dataset, and unknown-class settings. In-domain and cross-dataset branches report classification performance, while the unknown-class branch is used for uncertainty-based analysis.

Dataset	Res.	Cargo	Tanker	Fishing
FShip	$\sim$ 1.5 m	1693	785	148
OShip	$\sim$ 20 m	5303	139	1825

Table 1: Physical properties and instance counts; FUSARShip (FShip); OpenSARShip (OShip). ’Res.’ = spatial resolution.

	FUSARShip			OpenSARShip
Model	Acc	BAcc	Macro-F1	Acc	BAcc	Macro-F1
SD	75.68 $\pm$ 1.97	73.42 $\pm$ 3.98	75.40 $\pm$ 3.16	76.05 $\pm$ 0.58	53.22 $\pm$ 3.38	56.66 $\pm$ 3.37
DF	69.58 $\pm$ 1.63	56.57 $\pm$ 2.68	59.32 $\pm$ 2.06	74.62 $\pm$ 0.54	44.38 $\pm$ 2.62	47.09 $\pm$ 3.41
SJ	69.20 $\pm$ 2.28	58.34 $\pm$ 3.49	61.26 $\pm$ 3.30	74.36 $\pm$ 0.43	45.60 $\pm$ 2.66	48.31 $\pm$ 2.96
S4	68.94 $\pm$ 1.65	57.51 $\pm$ 2.33	60.38 $\pm$ 2.41	73.71 $\pm$ 0.49	38.17 $\pm$ 1.51	37.62 $\pm$ 2.62
SM	66.49 $\pm$ 1.66	50.91 $\pm$ 4.75	53.53 $\pm$ 5.12	73.16 $\pm$ 0.21	38.69 $\pm$ 2.09	36.99 $\pm$ 3.17

Table 2: In-domain results. Best value per metric is bold . Abbreviations: SD=SARDet100K, DF=DOFA, SJ=SARJEPA, S4=SSL4EO, SM=ScaleMAE, PV=Prithvi, Acc=Accuracy, BAcc=Balanced Accuracy.

	FUSARShip $\rightarrow$ OpenSARShip			OpenSARShip $\rightarrow$ FUSARShip
Model	Acc	BAcc	Macro-F1	Acc	BAcc	Macro-F1
SJ	58.15 $\pm$ 6.50	29.25 $\pm$ 2.04	27.88 $\pm$ 1.15	57.11 $\pm$ 2.95	30.46 $\pm$ 1.25	25.18 $\pm$ 0.77
DF	56.48 $\pm$ 4.64	45.63 $\pm$ 4.53	29.55 $\pm$ 1.90	60.79 $\pm$ 1.08	31.58 $\pm$ 0.63	25.77 $\pm$ 0.74
SM	54.31 $\pm$ 2.32	30.71 $\pm$ 0.67	30.42 $\pm$ 0.74	64.11 $\pm$ 0.77	33.10 $\pm$ 0.40	26.06 $\pm$ 0.19
PV	51.13 $\pm$ 6.35	43.40 $\pm$ 5.09	27.94 $\pm$ 1.53	64.49 $\pm$ 0.32	33.34 $\pm$ 0.16	26.30 $\pm$ 0.36
SD	48.01 $\pm$ 1.77	36.81 $\pm$ 0.74	31.77 $\pm$ 0.99	53.63 $\pm$ 3.98	31.98 $\pm$ 4.07	25.97 $\pm$ 2.24

Table 3: Cross-dataset results. Best value per metric is bold . Rows are ordered by F

\rightarrow

O rank; O

\rightarrow

F values are reordered to match.

Unknown	Acc (SD)	BAcc (SD)	Mac-F1 (SD)	Softmax	MC-Var	$s_{\text{class}}$	$s_{\text{domain}}$
FUSARShip
Cargo	97.17 $\pm$ 0.92	92.60 $\pm$ 1.63	94.53 $\pm$ 1.64	62.87 (PV)	51.72 (PV)	59.58 (SD)	55.35 (SD)
Fishing	97.68 $\pm$ 0.33	88.65 $\pm$ 3.03	91.59 $\pm$ 1.40	42.06 (SD)	49.02 (SD)	66.29 (SJ)	58.05 (SM)
Tanker	75.87 $\pm$ 1.72	70.49 $\pm$ 2.73	71.03 $\pm$ 2.54	42.51 (DF)	50.17 (DF)	87.94 (PV)	86.19 (PV)
OpenSARShip
Cargo	95.21 $\pm$ 0.66	74.27 $\pm$ 4.56	78.38 $\pm$ 3.96	45.83 (SJ)	49.76 (SJ)	57.91 (DF)	55.55 (DF)

Table 4: Unknown-class AUROC. For each dataset and held-out class, each column reports the best model value for that score.

Scenario	Report	Uncertainty
In-domain	BA, macro-F1	Secondary
Cross-dataset	BA + class recall; accuracy secondary	Required for screening
Unknown-class	AUROC; multiple held-out classes	Required for reject/flag
Pre-deployment policy	Bidirectional CD + multi-class UC	Use multiple scores, not only softmax/MC-dropout

Table 5: Practical guidance for evaluation. Recommended metrics, uncertainty use, and decision rules across operating regimes.

Data	Unk.	Smax	MC-V	$s_{\text{class}}$	$s_{\text{domain}}$
FS	Cargo	74.17 $\pm$ 2.59 (PV)	66.08 $\pm$ 2.88 (PV)	73.30 $\pm$ 2.21 (SD)	69.87 $\pm$ 2.04 (SD)
FS	Fishing	24.99 $\pm$ 1.32 (SD)	30.73 $\pm$ 2.69 (S4)	46.13 $\pm$ 3.86 (SJ)	40.94 $\pm$ 3.47 (SJ)
FS	Tanker	5.00 $\pm$ 0.34 (DF)	9.61 $\pm$ 4.17 (PV)	40.42 $\pm$ 4.16 (SM)	37.94 $\pm$ 4.84 (SM)
OS	Cargo	69.83 $\pm$ 1.73 (S4)	73.06 $\pm$ 0.80 (S4)	78.39 $\pm$ 0.56 (S4)	76.62 $\pm$ 0.51 (S4)
OS	Fishing	6.55 $\pm$ 3.77 (PV)	2.28 $\pm$ 0.36 (SJ)	23.17 $\pm$ 7.40 (SM)	23.46 $\pm$ 7.35 (SM)
OS	Tanker	29.69 $\pm$ 1.99 (SD)	26.42 $\pm$ 1.28 (SD)	24.77 $\pm$ 0.72 (SD)	24.96 $\pm$ 1.05 (SD)

Table 6: Unknown-class AUPR (complement to main Table 4 ). Best model per held-out class.

Model	$s_{\text{class}}$	$s_{\text{domain}}$	MC-Var	Bal. Acc
FUSARShip
SD	42.49 $\pm$ 2.85	41.49 $\pm$ 2.13	51.11 $\pm$ 4.05	73.42 $\pm$ 3.98
SJ	56.94 $\pm$ 2.63	55.41 $\pm$ 3.15	50.95 $\pm$ 3.49	58.34 $\pm$ 3.49
S4	52.16 $\pm$ 3.46	51.16 $\pm$ 3.28	50.78 $\pm$ 3.22	57.51 $\pm$ 2.33
DF	47.61 $\pm$ 2.70	50.11 $\pm$ 2.24	50.54 $\pm$ 3.40	56.57 $\pm$ 2.68
SM	56.10 $\pm$ 4.05	56.52 $\pm$ 2.32	51.20 $\pm$ 4.44	50.91 $\pm$ 4.75

Table 7: Error detection AUROC: in-domain. No single uncertainty score is consistently strongest across all models and datasets; MC-dropout remains near random, while prototype-based scores vary substantially by model and dataset.

Model	FUSARShip			OpenSARShip
	C	T	F	C	T	F
SD	85.54 $\pm$ 3.37	81.60 $\pm$ 7.84	53.13 $\pm$ 8.48	93.12 $\pm$ 1.82	29.40 $\pm$ 5.44	37.14 $\pm$ 9.90
SJ	85.49 $\pm$ 4.68	52.53 $\pm$ 11.05	37.01 $\pm$ 9.66	96.53 $\pm$ 1.28	13.56 $\pm$ 3.27	26.71 $\pm$ 8.03
S4	89.12 $\pm$ 3.47	55.73 $\pm$ 6.50	27.67 $\pm$ 6.92	98.50 $\pm$ 0.56	6.56 $\pm$ 1.92	9.43 $\pm$ 4.03
DF	87.82 $\pm$ 2.62	47.73 $\pm$ 8.15	34.17 $\pm$ 6.43	96.79 $\pm$ 0.83	14.20 $\pm$ 2.88	22.14 $\pm$ 6.93
SM	87.91 $\pm$ 4.25	39.73 $\pm$ 12.99	25.10 $\pm$ 11.00	99.72 $\pm$ 0.21	0.35 $\pm$ 0.46	16.00 $\pm$ 6.24

Table 8: In-domain per-class accuracy. Columns show Cargo, Tanker, Fishing recognition rates. Imbalance in class performance signals potential brittleness under shift.

Direction	Model	$s_{\text{class}}$	$s_{\text{domain}}$	MC-Var	Bal. Acc
FS $\rightarrow$ OS
	SJ	50.08 $\pm$ 3.08	55.99 $\pm$ 4.74	49.55 $\pm$ 2.04	29.25 $\pm$ 2.04
	DF	45.46 $\pm$ 2.66	46.99 $\pm$ 1.98	49.79 $\pm$ 1.55	45.63 $\pm$ 4.53
	SM	35.08 $\pm$ 2.63	67.09 $\pm$ 2.12	50.47 $\pm$ 1.07	30.71 $\pm$ 0.67
	PV	75.60 $\pm$ 6.16	74.14 $\pm$ 4.23	49.80 $\pm$ 2.02	43.40 $\pm$ 5.09
	SD	65.72 $\pm$ 1.79	68.60 $\pm$ 1.70	46.65 $\pm$ 1.27	36.81 $\pm$ 0.74

Table 9: Error detection AUROC: cross-dataset. Prototype scores (35–79) capture distribution shift; MC-dropout ineffective. Example: Prithvi achieves 75.60

s_{\text{class}}

despite 43.40% balanced accuracy, flagging errors where accuracy alone fails.

Model	Per-class Acc			Acc	BAcc
	Cargo	Tanker	Fishing
FUSARShip to OpenSARShip
SJ	77.59 $\pm$ 9.41	5.73 $\pm$ 1.74	4.43 $\pm$ 4.08	58.15 $\pm$ 6.50	29.25 $\pm$ 2.04
DF	75.04 $\pm$ 6.55	2.28 $\pm$ 0.88	59.57 $\pm$ 16.59	56.48 $\pm$ 4.64	45.63 $\pm$ 4.53
SM	65.27 $\pm$ 4.78	26.58 $\pm$ 5.08	0.29 $\pm$ 0.97	54.31 $\pm$ 2.32	30.71 $\pm$ 0.67
PV	67.11 $\pm$ 9.09	4.10 $\pm$ 0.85	59.00 $\pm$ 21.95	51.13 $\pm$ 6.35	43.40 $\pm$ 5.09

Table 10: Cross-dataset per-class accuracy. Direction-dependent class collapse evident: Tanker near 0% in OpenSARShip to FUSARShip; Fishing

\approx 0\%

in both directions. These per-class failures drive aggregate balanced accuracy collapse and motivate uncertainty-based error detection.

Classification

Classification maps observations to discrete labels, including familiar, unknown, hierarchical, or multiple categories.

See all 407 papers

Sep 15, 2026

Enhancing Foundation Models for Imbalanced SAR Ship Classification via Targeted Oversampling

Remote-sensing foundation models offer strong representations for SAR imagery, but their behavior under severe long-tail class imbalance is still not well characterized. We benchmark DOFA and SAR-JEPA on the imbalanced OpenSARShip dataset and compare them with ImageNet-pretrained baselines under a fixed, training-efficient protocol that keeps the backbone frozen. To mitigate imbalance without fine-tuning, we apply four oversampling methods in embedding space exclusively to minority classes and train a lightweight classifier head on the augmented embeddings. Across both foundation models, oversampling improves Macro-F1 and test accuracy relative to their respective baselines, with the largest Macro-F1 gains observed for DOFA using ADASYN (34.39 to 38.56) and for SAR-JEPA using SVM-SMOTE (25.89 to 32.30). We also report class-wise behavior, showing that aggregate improvements can coexist with persistent failures on specific rare classes. Code for embedding extraction and reproducible multi-seed evaluation is provided to support rapid experimentation on free-tier hardware.

Ch Muhammad Awais, Marco Reggiannini, Davide Moroni

Institute of Science and Technology, National Research Council, Pisa, Italy

Class-Imbalanced Learning Remote Sensing Synthetic Aperture Radar

Ask PDF

Apr 20, 2026

Prompting Foundation Models for Zero-Shot Ship Instance Segmentation in SAR Imagery

Synthetic Aperture Radar (SAR) plays a critical role in maritime surveillance, yet deep learning for SAR analysis is limited by the lack of pixel-level annotations. This paper explores how general-purpose vision foundation models can enable zero-shot ship instance segmentation in SAR imagery, eliminating the need for pixel-level supervision. A YOLOv11-based detector trained on open SAR datasets localizes ships via bounding boxes, which then prompt the Segment Anything Model 2 (SAM2) to produce instance masks without any mask annotations. Unlike prior SAM-based SAR approaches that rely on fine tuning or adapters, our method demonstrates that spatial constraints from a SAR-trained detector alone can effectively regularize foundation model predictions. This design partially mitigates the optical-SAR domain gap and enables downstream applications such as vessel classification, size estimation, and wake analysis. Experiments on the SSDD benchmark achieve a mean IoU of 0.637 (89% of a fully supervised baseline) with an overall ship detection rate of 89.2%, confirming a scalable, annotation-efficient pathway toward foundation-model-driven SAR image understanding.

Islam Mansour, Francescopaolo Sica, Michael Schmitt

Department of Aerospace Engineering, University of the Bundeswehr Munich, Neubiberg, Germany

Instance Segmentation Promptable Image Segmentation Remote Sensing Synthetic Aperture Radar

Ask PDF

Aug 3, 2026

OSSDD - a New Open Dataset for Sentinel-1 Ship Detection

Ship detection in Synthetic Aperture Radar (SAR) images plays an important role for maritime situational awareness, especially with respect to different illegal activities at sea such as illegal fishing, smuggling or border violations. Modern ship detection methods using neural networks usually require large training datasets, which are considerably scarcer in the SAR domain than in the electro-optical domain. While several free datasets exist for this task, their availability and usability vary. In this paper, OpenSARShip-Ship Detection Dataset (OSSDD), a new dataset based on the well-known OpenSARShip 1.0 dataset is proposed for training neural networks for SAR ship detection. OSSDD is freely available and contains 15,197 Sentinel-1 amplitude patches in VV and VH polarization, binary ship masks, axis-aligned bounding box and rotated bounding box annotations for a total of 55,759 ships. The construction of the dataset, the contents and structure of the downloadable data and experiments with three common detector models (Faster R-CNN, FCOS, DETR) are shown and discussed. The results serve as benchmarks for future experiments. The dataset is available on Hugging Face at https://huggingface.co/datasets/sylviaHoch/OpenSARShip-Ship-Detection-Dataset.

Horst Hammer, Sylvia Hochstuhl, Antje Thiele +4

Fraunhofer Institute of Optronics, System Technologies and Image Exploitation - IOSB, Gutleuthausstraße 1, Ettlingen, 76275, Germany · Karlsruhe Institute of Technology - KIT, Kaiserstraße 12, Karlsruhe, 76131, Germany · HENSOLDT Sensors GmbH, Wörthstraße 85, Ulm, 89077, Germany

Object Detection Synthetic Aperture Radar Remote Sensing Benchmarks Ship Detection

Ask PDF

Abstract

Classification

Explore similar work

Enhancing Foundation Models for Imbalanced SAR Ship Classification via Targeted Oversampling

Prompting Foundation Models for Zero-Shot Ship Instance Segmentation in SAR Imagery

OSSDD - a New Open Dataset for Sentinel-1 Ship Detection