Cross-Dataset Transfer and Unknown-Class Detection in Imbalanced SAR Ship Classification

Ch Muhammad Awais; Marco Reggiannini; Davide Moroni; Giulio Del Corso

cs.CVSep 15, 2026

Cross-Dataset Transfer and Unknown-Class Detection in Imbalanced SAR Ship Classification

Authors: Ch Muhammad Awais, Marco Reggiannini, Davide Moroni, Giulio Del Corso

Organizations: Institute of Science and Technology, National Research Council, Pisa, Italy

Abstract

Ship classification from Synthetic Aperture Radar (SAR) imagery is a critical computer vision task, yet the robustness of models under deployment shifts remains unclear. While models are often trained on one dataset and deployed on another, we lack a comprehensive understanding of their cross-dataset generalization. To address this, we evaluate six pretrained models on two SAR ship datasets in three settings: in-domain classification, cross-dataset transfer, and unknown-class detection. For unknown detection, we hold out all classes one at a time. In-domain, SARDet100K gives the best balanced accuracy on both datasets (73.4% on FUSARShip and 53.2% on OpenSARShip). In cross-dataset transfer, we observe strong failures: some models show moderate accuracy but near-chance balanced accuracy (for example, 64.5% accuracy but 33.3% balanced accuracy for OpenSARShip to FUSARShip). In unknown detection, performance depends on the held-out class and dataset, while MC-dropout variance is often close to random. These findings show that cross-dataset generalization in SAR remains limited and that task-specific uncertainty scores are often more informative than MC-dropout variance for held-out-class detection, although their relative ranking depends on the dataset and held-out class.

Figures & tables

Figure 1: Overview of the evaluation pipeline used in this study. SAR images are encoded with an Earth Observation (EO) foundation model to extract representations, followed by training of a shared classifier head. The trained model is then evaluated in three settings: in-domain, cross-dataset, and unknown-class settings. In-domain and cross-dataset branches report classification performance, while the unknown-class branch is used for uncertainty-based analysis.

Dataset	Res.	Cargo	Tanker	Fishing
FShip	$\sim$ 1.5 m	1693	785	148
OShip	$\sim$ 20 m	5303	139	1825

Table 1: Physical properties and instance counts; FUSARShip (FShip); OpenSARShip (OShip). ’Res.’ = spatial resolution.

	FUSARShip			OpenSARShip
Model	Acc	BAcc	Macro-F1	Acc	BAcc	Macro-F1
SD	75.68 $\pm$ 1.97	73.42 $\pm$ 3.98	75.40 $\pm$ 3.16	76.05 $\pm$ 0.58	53.22 $\pm$ 3.38	56.66 $\pm$ 3.37
DF	69.58 $\pm$ 1.63	56.57 $\pm$ 2.68	59.32 $\pm$ 2.06	74.62 $\pm$ 0.54	44.38 $\pm$ 2.62	47.09 $\pm$ 3.41
SJ	69.20 $\pm$ 2.28	58.34 $\pm$ 3.49	61.26 $\pm$ 3.30	74.36 $\pm$ 0.43	45.60 $\pm$ 2.66	48.31 $\pm$ 2.96
S4	68.94 $\pm$ 1.65	57.51 $\pm$ 2.33	60.38 $\pm$ 2.41	73.71 $\pm$ 0.49	38.17 $\pm$ 1.51	37.62 $\pm$ 2.62
SM	66.49 $\pm$ 1.66	50.91 $\pm$ 4.75	53.53 $\pm$ 5.12	73.16 $\pm$ 0.21	38.69 $\pm$ 2.09	36.99 $\pm$ 3.17

Table 2: In-domain results. Best value per metric is bold . Abbreviations: SD=SARDet100K, DF=DOFA, SJ=SARJEPA, S4=SSL4EO, SM=ScaleMAE, PV=Prithvi, Acc=Accuracy, BAcc=Balanced Accuracy.

	FUSARShip $\rightarrow$ OpenSARShip			OpenSARShip $\rightarrow$ FUSARShip
Model	Acc	BAcc	Macro-F1	Acc	BAcc	Macro-F1
SJ	58.15 $\pm$ 6.50	29.25 $\pm$ 2.04	27.88 $\pm$ 1.15	57.11 $\pm$ 2.95	30.46 $\pm$ 1.25	25.18 $\pm$ 0.77
DF	56.48 $\pm$ 4.64	45.63 $\pm$ 4.53	29.55 $\pm$ 1.90	60.79 $\pm$ 1.08	31.58 $\pm$ 0.63	25.77 $\pm$ 0.74
SM	54.31 $\pm$ 2.32	30.71 $\pm$ 0.67	30.42 $\pm$ 0.74	64.11 $\pm$ 0.77	33.10 $\pm$ 0.40	26.06 $\pm$ 0.19
PV	51.13 $\pm$ 6.35	43.40 $\pm$ 5.09	27.94 $\pm$ 1.53	64.49 $\pm$ 0.32	33.34 $\pm$ 0.16	26.30 $\pm$ 0.36
SD	48.01 $\pm$ 1.77	36.81 $\pm$ 0.74	31.77 $\pm$ 0.99	53.63 $\pm$ 3.98	31.98 $\pm$ 4.07	25.97 $\pm$ 2.24

Table 3: Cross-dataset results. Best value per metric is bold . Rows are ordered by F

\rightarrow

O rank; O

\rightarrow

F values are reordered to match.

Unknown	Acc (SD)	BAcc (SD)	Mac-F1 (SD)	Softmax	MC-Var	$s_{\text{class}}$	$s_{\text{domain}}$
FUSARShip
Cargo	97.17 $\pm$ 0.92	92.60 $\pm$ 1.63	94.53 $\pm$ 1.64	62.87 (PV)	51.72 (PV)	59.58 (SD)	55.35 (SD)
Fishing	97.68 $\pm$ 0.33	88.65 $\pm$ 3.03	91.59 $\pm$ 1.40	42.06 (SD)	49.02 (SD)	66.29 (SJ)	58.05 (SM)
Tanker	75.87 $\pm$ 1.72	70.49 $\pm$ 2.73	71.03 $\pm$ 2.54	42.51 (DF)	50.17 (DF)	87.94 (PV)	86.19 (PV)
OpenSARShip
Cargo	95.21 $\pm$ 0.66	74.27 $\pm$ 4.56	78.38 $\pm$ 3.96	45.83 (SJ)	49.76 (SJ)	57.91 (DF)	55.55 (DF)

Table 4: Unknown-class AUROC. For each dataset and held-out class, each column reports the best model value for that score.

Scenario	Report	Uncertainty
In-domain	BA, macro-F1	Secondary
Cross-dataset	BA + class recall; accuracy secondary	Required for screening
Unknown-class	AUROC; multiple held-out classes	Required for reject/flag
Pre-deployment policy	Bidirectional CD + multi-class UC	Use multiple scores, not only softmax/MC-dropout

Table 5: Practical guidance for evaluation. Recommended metrics, uncertainty use, and decision rules across operating regimes.

Data	Unk.	Smax	MC-V	$s_{\text{class}}$	$s_{\text{domain}}$
FS	Cargo	74.17 $\pm$ 2.59 (PV)	66.08 $\pm$ 2.88 (PV)	73.30 $\pm$ 2.21 (SD)	69.87 $\pm$ 2.04 (SD)
FS	Fishing	24.99 $\pm$ 1.32 (SD)	30.73 $\pm$ 2.69 (S4)	46.13 $\pm$ 3.86 (SJ)	40.94 $\pm$ 3.47 (SJ)
FS	Tanker	5.00 $\pm$ 0.34 (DF)	9.61 $\pm$ 4.17 (PV)	40.42 $\pm$ 4.16 (SM)	37.94 $\pm$ 4.84 (SM)
OS	Cargo	69.83 $\pm$ 1.73 (S4)	73.06 $\pm$ 0.80 (S4)	78.39 $\pm$ 0.56 (S4)	76.62 $\pm$ 0.51 (S4)
OS	Fishing	6.55 $\pm$ 3.77 (PV)	2.28 $\pm$ 0.36 (SJ)	23.17 $\pm$ 7.40 (SM)	23.46 $\pm$ 7.35 (SM)
OS	Tanker	29.69 $\pm$ 1.99 (SD)	26.42 $\pm$ 1.28 (SD)	24.77 $\pm$ 0.72 (SD)	24.96 $\pm$ 1.05 (SD)

Table 6: Unknown-class AUPR (complement to main Table 4 ). Best model per held-out class.

Model	$s_{\text{class}}$	$s_{\text{domain}}$	MC-Var	Bal. Acc
FUSARShip
SD	42.49 $\pm$ 2.85	41.49 $\pm$ 2.13	51.11 $\pm$ 4.05	73.42 $\pm$ 3.98
SJ	56.94 $\pm$ 2.63	55.41 $\pm$ 3.15	50.95 $\pm$ 3.49	58.34 $\pm$ 3.49
S4	52.16 $\pm$ 3.46	51.16 $\pm$ 3.28	50.78 $\pm$ 3.22	57.51 $\pm$ 2.33
DF	47.61 $\pm$ 2.70	50.11 $\pm$ 2.24	50.54 $\pm$ 3.40	56.57 $\pm$ 2.68
SM	56.10 $\pm$ 4.05	56.52 $\pm$ 2.32	51.20 $\pm$ 4.44	50.91 $\pm$ 4.75

Table 7: Error detection AUROC: in-domain. No single uncertainty score is consistently strongest across all models and datasets; MC-dropout remains near random, while prototype-based scores vary substantially by model and dataset.

Model	FUSARShip			OpenSARShip
	C	T	F	C	T	F
SD	85.54 $\pm$ 3.37	81.60 $\pm$ 7.84	53.13 $\pm$ 8.48	93.12 $\pm$ 1.82	29.40 $\pm$ 5.44	37.14 $\pm$ 9.90
SJ	85.49 $\pm$ 4.68	52.53 $\pm$ 11.05	37.01 $\pm$ 9.66	96.53 $\pm$ 1.28	13.56 $\pm$ 3.27	26.71 $\pm$ 8.03
S4	89.12 $\pm$ 3.47	55.73 $\pm$ 6.50	27.67 $\pm$ 6.92	98.50 $\pm$ 0.56	6.56 $\pm$ 1.92	9.43 $\pm$ 4.03
DF	87.82 $\pm$ 2.62	47.73 $\pm$ 8.15	34.17 $\pm$ 6.43	96.79 $\pm$ 0.83	14.20 $\pm$ 2.88	22.14 $\pm$ 6.93
SM	87.91 $\pm$ 4.25	39.73 $\pm$ 12.99	25.10 $\pm$ 11.00	99.72 $\pm$ 0.21	0.35 $\pm$ 0.46	16.00 $\pm$ 6.24

Table 8: In-domain per-class accuracy. Columns show Cargo, Tanker, Fishing recognition rates. Imbalance in class performance signals potential brittleness under shift.

Direction	Model	$s_{\text{class}}$	$s_{\text{domain}}$	MC-Var	Bal. Acc
FS $\rightarrow$ OS
	SJ	50.08 $\pm$ 3.08	55.99 $\pm$ 4.74	49.55 $\pm$ 2.04	29.25 $\pm$ 2.04
	DF	45.46 $\pm$ 2.66	46.99 $\pm$ 1.98	49.79 $\pm$ 1.55	45.63 $\pm$ 4.53
	SM	35.08 $\pm$ 2.63	67.09 $\pm$ 2.12	50.47 $\pm$ 1.07	30.71 $\pm$ 0.67
	PV	75.60 $\pm$ 6.16	74.14 $\pm$ 4.23	49.80 $\pm$ 2.02	43.40 $\pm$ 5.09
	SD	65.72 $\pm$ 1.79	68.60 $\pm$ 1.70	46.65 $\pm$ 1.27	36.81 $\pm$ 0.74

Table 9: Error detection AUROC: cross-dataset. Prototype scores (35–79) capture distribution shift; MC-dropout ineffective. Example: Prithvi achieves 75.60

s_{\text{class}}

despite 43.40% balanced accuracy, flagging errors where accuracy alone fails.

Model	Per-class Acc			Acc	BAcc
	Cargo	Tanker	Fishing
FUSARShip to OpenSARShip
SJ	77.59 $\pm$ 9.41	5.73 $\pm$ 1.74	4.43 $\pm$ 4.08	58.15 $\pm$ 6.50	29.25 $\pm$ 2.04
DF	75.04 $\pm$ 6.55	2.28 $\pm$ 0.88	59.57 $\pm$ 16.59	56.48 $\pm$ 4.64	45.63 $\pm$ 4.53
SM	65.27 $\pm$ 4.78	26.58 $\pm$ 5.08	0.29 $\pm$ 0.97	54.31 $\pm$ 2.32	30.71 $\pm$ 0.67
PV	67.11 $\pm$ 9.09	4.10 $\pm$ 0.85	59.00 $\pm$ 21.95	51.13 $\pm$ 6.35	43.40 $\pm$ 5.09

Table 10: Cross-dataset per-class accuracy. Direction-dependent class collapse evident: Tanker near 0% in OpenSARShip to FUSARShip; Fishing

\approx 0\%

in both directions. These per-class failures drive aggregate balanced accuracy collapse and motivate uncertainty-based error detection.

Classification

Classification maps observations to discrete labels, including familiar, unknown, hierarchical, or multiple categories.

See all 407 papers

Enhancing Foundation Models for Imbalanced SAR Ship Classification via Targeted Oversampling
Sep 15, 2026Ch Muhammad Awais, Marco Reggiannini, Davide MoroniClass-Imbalanced Learning Remote Sensing
Institute of Science and Technology, National Research Council, Pisa, Italy
Prompting Foundation Models for Zero-Shot Ship Instance Segmentation in SAR Imagery
Apr 20, 2026Islam Mansour, Francescopaolo Sica, Michael SchmittInstance Segmentation Promptable Image Segmentation
Department of Aerospace Engineering, University of the Bundeswehr Munich, Neubiberg, Germany
OSSDD - a New Open Dataset for Sentinel-1 Ship Detection
Aug 3, 2026Horst Hammer, Sylvia Hochstuhl, Antje Thiele +4Object Detection Synthetic Aperture Radar
Fraunhofer Institute of Optronics, System Technologies and Image Exploitation - IOSB, Gutleuthausstraße 1, Ettlingen, 76275, Germany · Karlsruhe Institute of Technology - KIT, Kaiserstraße 12, Karlsruhe, 76131, Germany · HENSOLDT Sensors GmbH, Wörthstraße 85, Ulm, 89077, Germany

Sep 15, 2026J/K move · Enter open · S save

Enhancing Foundation Models for Imbalanced SAR Ship Classification via Targeted Oversampling

Ch Muhammad Awais, Marco Reggiannini, Davide Moroni

Institute of Science and Technology, National Research Council, Pisa, Italy

Class-Imbalanced Learning Remote Sensing Synthetic Aperture Radar

Open paper AskPDF arXiv

Remote-sensing foundation models offer strong representations for SAR imagery, but their behavior under severe long-tail class imbalance is still not well characterized. We benchmark DOFA and SAR-JEPA on the imbalanced OpenSARShip dataset and compare them with ImageNet-pretrained baselines under a fixed, training-efficient protocol that keeps the backbone frozen. To mitigate imbalance without fine-tuning, we apply four oversampling methods in embedding space exclusively to minority classes and train a lightweight classifier head on the augmented embeddings. Across both foundation models, oversampling improves Macro-F1 and test accuracy relative to their respective baselines, with the largest Macro-F1 gains observed for DOFA using ADASYN (34.39 to 38.56) and for SAR-JEPA using SVM-SMOTE (25.89 to 32.30). We also report class-wise behavior, showing that aggregate improvements can coexist with persistent failures on specific rare classes. Code for embedding extraction and reproducible multi-seed evaluation is provided to support rapid experimentation on free-tier hardware.

Abstract

Classification

Explore similar work

Enhancing Foundation Models for Imbalanced SAR Ship Classification via Targeted Oversampling

Prompting Foundation Models for Zero-Shot Ship Instance Segmentation in SAR Imagery

OSSDD - a New Open Dataset for Sentinel-1 Ship Detection