IFC-SSL-Bench: Benchmarking Self-Supervised Learning for Annotation-Efficient Multispectral Imaging Flow Cytometry

Angelo Jovin Yamachui Sitcheu; Roxana-Teodora Fetescu; Thomas Lautenschlager; Nils Friederich; Linus Schilpp; Stephan Allgeier; Luca Deininger; Daniel Hübschmann

cs.CVOct 22, 2026

IFC-SSL-Bench: Benchmarking Self-Supervised Learning for Annotation-Efficient Multispectral Imaging Flow Cytometry

Authors: Angelo Jovin Yamachui Sitcheu, Roxana-Teodora Fetescu, Thomas Lautenschlager, Nils Friederich, Linus Schilpp, Stephan Allgeier, Luca Deininger, Daniel Hübschmann,

Organizations: Institute for Automation and Applied Informatics, Karlsruhe Institute of Technology, Eggenstein-Leopoldshafen, Germany · Institute of Biological and Chemical Systems, Karlsruhe Institute of Technology, Eggenstein-Leopoldshafen, Germany · Department of Biosystems Science and Engineering, ETH Zürich, Basel, Switzerland · Heidelberg Institute for Stem Cell Technology and Experimental Medicine, HI-STEM gGmbH, Heidelberg, Germany · Innovation and Service Unit for Bioinformatics and Precision Medicine, German Cancer Research Center, Heidelberg, Germany · German Cancer Consortium, DKTK, Heidelberg, Germany · Division of Translational Precision Medicine, Institute of Human Genetics, Heidelberg University, Heidelberg, Germany · Computational Oncology Group, Molecular Precision Oncology Program, National Center for Tumor Diseases Heidelberg and German Cancer Research Center, Heidelberg, Germany

Abstract

Imaging Flow Cytometry (IFC) generates large multispectral datasets, yet the need for expert annotations bottlenecks the development of supervised deep learning models. While Self-Supervised Learning (SSL) offers a solution, its application to IFC remains fragmented and lacks systematic benchmarking. To address this, we present IFC-SSL-Bench, a modular framework for evaluating SSL in IFC. We benchmark six SSL approaches across four backbones on two heterogeneous datasets. By assessing downstream classification performance across annotation regimes of the training sets (1%-100%), we quantify annotation efficiency and identify optimal configurations for high-dimensional biological imaging. SSL matches the performance of fully supervised baselines trained on 100% of annotated data while using as little as 5% of annotations for morphological classification, with advantages in low-data regimes below 5%. Across both datasets, the contrastive approach MoCo-v2 paired with ConvNeXt backbones emerged as the top-performing configuration, with ConvNeXtV1-T best for immunological synapse classification and ConvNeXtV2-T for morphological classification. We identify a clear pattern in transfer dynamics: ImageNet initialization benefits morphological classification, consistent with its visual similarity to natural images, whereas random initialization better captures the more domain-specific immunological synapse classification. SSL elevates class-specific F1-scores by over 35 percentage points for structurally complex classes at 1% of annotated data. We synthesize these insights into a unified set of guidelines for deploying self-supervised workflows in high-throughput IFC, and release the benchmark as a resource for future work.

Figures & tables

Figure 1: Overview of the modular SSL benchmarking framework. The pipeline is organized into three core modules: 1, data preparation. This module covers the dataset split generation, channel selection, and domain-specific transformations. 2, the learning module, which consists of the self-supervised pretraining and supervised baseline. Six self-supervised objectives spanning three families of invariance-based SSL are pretrained on the unannotated set: contrastive (SimCLR, MoCo-v2, MoCo-v3), asymmetric ( BYOL , DINO), and redundancy reduction ( BT ). In parallel, the annotated set is used to train end-to-end supervised baselines for each backbone, weight initialization, and annotated data fraction ( $1\%$ , $5\%$ , $10\%$ , $25\%$ , $50\%$ , and $100\%$ ). 3, downstream analysis evaluates the pretrained encoders on image classification at the same annotated data fractions as the supervised baselines through four evaluation protocols ( kNN , linear probing, XGBoost, and full finetuning) quantified by macro F1 scores. This quantitative analysis is complemented by a qualitative assessment of the learned representations. Scale bars in the multichannel sample images represent $5\mu m$ .

Figure 2: Overview of IFC datasets and distribution of annotated and unannotated subsets. (A) Representative multichannel imagery and class distribution of the annotated RBC dataset [ 20 ] . For each class, three imaging channels are shown alongside the sample counts for training, validation, and testing splits. (B) Distribution of the RBC unannotated subset across acquisition batches (bags (CE) and timepoints (Day)). Stacked bars represent the sample volume per CE, partitioned by day. (C) Representative multichannel imagery and class distribution of the annotated ISF dataset [ 25 , 57 ] . Training, validation, and testing counts are indicated per class. (D) Distribution of the ISF unannotated subset across four independent experiments. Stacked bars illustrate donor diversity within each experiment, showing the contribution of individual donors to the total unannotated pool. Scale bars in (A) and (C) represent $5\mu m$ .

Backbone	RBC	ISF – 5CH
ResNet50 – $\theta_{\mathrm{Rnd}}$	$0.875\pm 0.030$	$0.870\pm 0.007$
ResNet50 – $\theta_{\mathrm{IN}}$	$0.920\pm 0.006$	$0.752\pm 0.021$
ConvNeXtV1 – T – $\theta_{\mathrm{Rnd}}$	$0.839\pm 0.006$	$0.802\pm 0.018$
ConvNeXtV1 – T – $\theta_{\mathrm{IN}}$	$0.921\pm 0.005$	$0.889\pm 0.007$
ConvNeXtV2 – T – $\theta_{\mathrm{Rnd}}$	$0.816\pm 0.019$	$0.848\pm 0.002$
ConvNeXtV2 – T – $\theta_{\mathrm{IN}}$	$\mathbf{0.922\pm 0.011}$	$0.877\pm 0.011$

Table 1: Test set macro F1-scores of the supervised end-to-end baselines (mean

\pm

std) on

100\%

annotated data fraction.

\theta_{\mathrm{IN}}

and

\theta_{\mathrm{Rnd}}

denote initialization from ImageNet-supervised weights and random initialization, respectively. The best-performing supervised baseline for each dataset is bolded .

Figure 3: Test set performance of supervised baselines for both datasets ( RBC and ISF – 5CH) across backbone variants, annotated data fractions, and weights initialization strategies

Figure 4: Benchmarking self-supervised representations for the ISF – 5CH dataset. (A) Ranking of combinations of SSL methods and backbones across downstream evaluation protocols. Heatmaps display the rank of $48$ model combinations (six SSL methods times four backbones times two initialization strategies) across kNN , linear probing, and XGBoost protocols for each annotated fraction of $\mathcal{D}_{\mathrm{ann}}$ . Each cell is split to compare initialization $\theta_{\mathrm{Rnd}}$ (left/green, $24$ ranks) vs. $\theta_{\mathrm{IN}}$ (right/blue, $24$ ranks). Darker hues indicate superior ranking (rank 1). Markers identify the corresponding backbone architecture for the specific configuration. (B) SSL method performance ranking for each backbone from each annotated fraction based on linear probing evaluation. Markers with bolded edges represent $\theta_{\mathrm{IN}}$ and normal edges $\theta_{\mathrm{Rnd}}$ . (C) Architectural robustness and initialization bias: Comparison of average backbone rank after linear probing between initializations $\theta_{\mathrm{Rnd}}$ and $\theta_{\mathrm{IN}}$ . Error bars indicate the inter SSL method standard deviation. (D) Finetuning trajectories of top-ranked configurations. For each initialization mode ( $\theta_{\mathrm{Rnd}}$ or $\theta_{\mathrm{IN}}$ ) and evaluation protocol, the best-performing SSL -backbone pairing as indicated in Table 2 was selected. (E) Finetuning performance gain between the best supervised baseline at $100\%$ of $\mathcal{D}_{\mathrm{ann}}$ ( ViT – S/16 – $\theta_{\mathrm{IN}}$ ) and the best SSL combination (ConvNeXtV1 – T – $\theta_{\mathrm{Rnd}}$ with MoCo-v2). The table displays the absolute improvements ( pp delta) in overall macro-F1-score along with the single class showing the most substantial improvement for each annotated fraction of $\mathcal{D}_{\mathrm{ann}}$ . Fractions on the x-axis in panels (B) , (C) , and (D) are displayed on a logarithmic scale for clarity in low-data regimes. Ranks (1 = top performer) per annotated fraction ( $1\%\mbox{--}100\%$ ) in (B) and (C) were calculated based on test set macro F1-scores using a linear probing protocol.

Backbone	Method	kNN		Linear probing		XGBoost
		$\mathbf{\theta_{Rnd}}$	$\mathbf{\theta_{IN}}$	$\mathbf{\theta_{Rnd}}$	$\mathbf{\theta_{IN}}$	$\mathbf{\theta_{Rnd}}$	$\mathbf{\theta_{IN}}$
ResNet50	SimCLR	$0.633\pm 0.011$	$0.639\pm 0.007$	$0.845\pm 0.020$	$0.799\pm 0.006$	$0.832\pm 0.005$	$0.824\pm 0.005$
	MoCo-v2	$0.710\pm 0.012$	$0.740\pm 0.013$	$0.844\pm 0.015$	$0.831\pm 0.005$	$0.849\pm 0.006$	$0.859\pm 0.012$
	MoCo-v3	$\underline{0.722\pm 0.006}$	$\underline{\mathbf{0.760\pm 0.018}}$	$0.852\pm 0.009$	$\underline{0.856\pm 0.014}$	$\underline{\mathbf{0.850\pm 0.010}}$	$\underline{\mathbf{0.867\pm 0.012}}$
	DINO	$0.495\pm 0.018$	$0.299\pm 0.017$	$0.717\pm 0.005$	$0.433\pm 0.023$	$0.688\pm 0.010$	$0.449\pm 0.006$
	BT	$0.662\pm 0.013$	$0.599\pm 0.003$	$\underline{0.857\pm 0.010}$	$0.807\pm 0.013$	$0.840\pm 0.010$	$0.820\pm 0.001$

Table 2: Evaluation of the SSL methods on the fully annotated ISF – 5CH dataset: Test set macro F1-scores (mean

\pm

std) for kNN evaluation, linear probing, and XGBoost.

\theta_{\mathrm{IN}}

and

\theta_{\mathrm{Rnd}}

denote initialization from ImageNet-supervised weights and random initialization, respectively. For each evaluation protocol and weight initialization, the best score per backbone is underlined , and the best score across all backbones is highlighted in bold .

Figure 5: Benchmarking self-supervised representations for the RBC dataset. (A) Ranking of combinations of SSL methods and backbones across downstream evaluation protocols. Heatmaps display the rank of $48$ model combinations (six SSL methods times four backbones times two initialization strategies) across kNN , linear probing, and XGBoost protocols for each annotated fraction. Each cell is split to compare initialization $\theta_{\mathrm{Rnd}}$ (left/green, $24$ ranks) vs. $\theta_{\mathrm{IN}}$ (right/blue, $24$ ranks). Darker hues indicate superior ranking (rank 1). Markers identify the corresponding backbone architecture for the specific configuration. (B) SSL method performance ranking for each backbone from each annotated fraction based on linear probing evaluation. Markers with bolded edges represent $\theta_{\mathrm{IN}}$ and normal edges $\theta_{\mathrm{Rnd}}$ . (C) Architectural robustness and initialization bias: Comparison of average backbone rank after linear probing between initializations $\theta_{\mathrm{Rnd}}$ and $\theta_{\mathrm{IN}}$ . Error bars indicate the inter SSL method standard deviation. (D) Finetuning trajectories of top-ranked configurations. For each initialization mode ( $\theta_{\mathrm{Rnd}}$ or $\theta_{\mathrm{IN}}$ ) and evaluation protocol, the best-performing SSL -backbone pairing as indicated in Table 3 was selected. (E) Finetuning performance gain between the best supervised baseline at $100\%$ of $\mathcal{D}_{\mathrm{ann}}$ (ConvNeXtV2 – T – $\theta_{\mathrm{IN}}$ ) and the best SSL combination for each annotated fraction from (E) . The table displays the absolute improvements (percentage point delta) in overall macro-F1-score along with the single class showing the most substantial improvement for each annotated fraction of $\mathcal{D}_{\mathrm{ann}}$ . Fractions on the x-axis in panels (B) , (C) , and (D) are displayed on a logarithmic scale for clarity in low-data regimes. Ranks (1 = top performer) per annotated fraction ( $0.1\%\mbox{--}100\%$ ) in (B) and (C) were calculated based on test set macro F1-scores using a linear probing protocol.

Backbone	Method	kNN		Linear probing		XGBoost
		$\mathbf{\theta_{Rnd}}$	$\mathbf{\theta_{IN}}$	$\mathbf{\theta_{Rnd}}$	$\mathbf{\theta_{IN}}$	$\mathbf{\theta_{Rnd}}$	$\mathbf{\theta_{IN}}$
ResNet50	SimCLR	$0.579\pm 0.037$	$0.796\pm 0.002$	$0.752\pm 0.018$	$0.873\pm 0.008$	$0.698\pm 0.013$	$0.891\pm 0.008$
	MoCo-v2	$0.461\pm 0.026$	$0.829\pm 0.020$	$0.651\pm 0.001$	$0.865\pm 0.010$	$0.641\pm 0.026$	$0.901\pm 0.014$
	MoCo-v3	$0.481\pm 0.032$	$\underline{\mathbf{0.843\pm 0.025}}$	$0.585\pm 0.175$	$0.857\pm 0.013$	$0.663\pm 0.055$	$0.902\pm 0.004$
	DINO	$0.558\pm 0.006$	$0.701\pm 0.027$	$0.754\pm 0.030$	$0.829\pm 0.003$	$\underline{0.755\pm 0.011}$	$0.826\pm 0.013$
	BT	$\underline{0.593\pm 0.006}$	$0.814\pm 0.009$	$\underline{0.774\pm 0.014}$	$\underline{0.891\pm 0.012}$	$0.722\pm 0.007$	$\underline{\mathbf{0.904\pm 0.004}}$

Table 3: Evaluation of the SSL methods on the fully annotated RBC dataset: Test set macro F1-scores (mean

\pm

std) for kNN evaluation, linear probing, and XGBoost.

\theta_{\mathrm{IN}}

and

\theta_{\mathrm{Rnd}}

Figure 6: Latent space visualization of the pretrained backbones on full $\mathcal{D}_{\mathrm{una}}$ for the (A) ISF – 5CH and (B) RBC datasets. We selected for each initialization strategy ( $\theta_{\mathrm{IN}}$ or $\theta_{\mathrm{Rnd}}$ ) the best SSL combination after linear probing evaluation and color-coded test samples with annotations. ARI stands for Adjusted Rand Index and NMI for Normalized Mutual Information . Scale bars in illustratives images of (B) represent $5\mu m$ . Experiments in (A) represent the individual experiments as described in [ 57 , 25 ] .

Class within experiment
			ConvNeXtV1 – T – $\theta_{\mathrm{Rnd}}$ + MoCo-v2		ViT – S/16 – $\theta_{\mathrm{IN}}$ + DINO
Grouping	$n$	$k$	ARI	NMI	ARI	NMI
Annotated class	$1,169$	$9$	$0.078$	$0.182$	$0.111$	$0.259$
Experiment	$1,169$	$4$	$0.811$	$0.792$	$0.212$	$0.349$
Experiment 1	$252$	$9$	$0.188$	$0.353$	$0.201$	$0.413$
Experiment 2	$139$	$9$	$0.160$	$0.398$	$0.306$	$0.533$

Table 4: Assessing the cluster structure of the learned representations on the ISF – 5CH dataset with ARI and NMI . Unsupervised

k

-means clustering of embeddings with

k

set to the number of groups and

n

representing the number of images. Class within experiment is computed separately for each experiment; the weighted mean averages over experiments proportionally to the number of images. The random control repeats the within-experiment analysis on size-matched random groupings (

10

repetitions).

Backbone	# Parameters	Blocks	Kernel/Patch size
ResNet50	$\sim 23.5$ M	$16$	$3\times 3$
ConvNeXtV1 – T	$\sim 27.9$ M	$18$	$7\times 7$
ConvNeXtV2 – T	$\sim 27.9$ M	$18$	$7\times 7$
ViT – S/16	$\sim 21.7$ M	$12$	$16\times 16$

Table 5: Overview of the benchmarked backbone architectures. Parameter counts expressed in millions.

Backbone	BloodMNIST
MedMNIST ResNet50 [ 6 ]	$0.950$
MedMNIST ResNet18 [ 6 ]	$0.963$
ResNet50 – $\theta_{\mathrm{Rnd}}$	$0.987\pm 0.000$
ResNet50 – $\theta_{\mathrm{IN}}$	$0.990\pm 0.000$
ConvNeXtV1 – T – $\theta_{\mathrm{Rnd}}$	$0.947\pm 0.000$
ConvNeXtV1 – T – $\theta_{\mathrm{IN}}$	$\mathbf{0.990\pm 0.000}$

Table 6: Accuracies of the supervised end-to-end baselines (mean

\pm

std) on the BloodMNIST dataset.

\theta_{\mathrm{IN}}

and

\theta_{\mathrm{Rnd}}

denote initialization from ImageNet-supervised weights and random initialization, respectively. We report accuracies here for consistency with the MedMNIST convention [ 6 ] . The best-performing supervised baseline for each dataset is bolded .

Hyperparameter	Baseline	Finetuning
Image size	$224\times 224$	$224\times 224$
Batch size	$128$	$128$
Epochs	$200$	$200$
Learning Rate (LR) end-to-end	$1e-3$	–
LR backbone	–	$1e-4$
LR head	–	$1e-3$

Table 7: Hyperparameters used for supervised trainings. The batch size was accordingly adjusted to eight when the training set had less then

128

samples. " – " indicates unavailability.

Figure S1: Supervised baseline evaluation under varying initialization, data constraints, and domain-specific transformation. (A) Transfer learning effect ( $\Delta$ Macro F1 score = Macro F1 score $\theta_{\mathrm{Rnd}}-$ Macro F1 score $\theta_{\mathrm{IN}}$ ) measured as the difference in macro F1-score between random initialization ( $\theta_{\mathrm{Rnd}}$ ) and ImageNet-supervised ( $\theta_{\mathrm{IN}}$ ) across different annotated fractions for the ISF – 5CH and RBC datasets. (B) Macro F1-score under extreme data scarcity ( $0.1\%$ annotated fraction) on the RBC dataset, comparing all backbone architectures across both initialization strategies. (C) Impact of the domain specific Logicle transformation on the ISF – 5CH dataset. The baseline is ConvNeXtV1 – T $\theta_{\mathrm{Rnd}}$ trained end-to-end on $100\%$ of $\mathcal{D}_{\mathrm{ann}}$ .

Augmentation technique	Two-view	DINO
Global image size	$224\times 224$	$224\times 224$
Local image size	–	$96\times 96$
Number of views (global, local)	$2,0$	$2,6$
Horizontal flip probability	$0.5$	$0.5$
Vertical flip probability	$0.5$	$0.5$
Global crop scale	$0.08-1$	$0.4-1$

Table 8: List of data augmentation parameters for self-supervised pretraining. The methods using a two-view strategy are SimCLR, MoCo-v2 & v3, BYOL, and BT . DINO uses a multi-crop strategy and the probability of applying Gaussian blur follows the original publication (probability of

1{.}0

for the first global crop,

0{.}1

for the second global crop and

0{.}5

for all local crops). " – " indicates unavailability.

Common
Parameters	SimCLR	MoCo-v2	MoCo-v3	BYOL	BT	DINO
Batch size	$128$
Embed. dim	$256$
Epochs	$200$
Optimization Batch Normalization (BN)
Optimizer	LARS	SGD	SGD	LARS	LARS	SGD

Table 9: Hyperparameters used during self-supervised pretrainings. Common values to all methods are centralized (under MoCo-v3). " – " indicates unavailability. Pretraining on the ISF – 5CH dataset occurred for only

10

epochs.

Figure S2: Comparative analysis of SSL strategies across backbones and annotated fractions for the kNN evaluation protocol on the ISF – 5CH dataset. Each cell displays the test set mean macro F1-score $\pm$ standard deviation across three runs for a specific backbone, SSL method, and weights initialization strategy. The left (green) and right (blue) halves of each cell represent random initialization ( $\theta_{\mathrm{Rnd}}$ ) and ImageNet pretraining ( $\theta_{\mathrm{IN}}$ ) respectively, with color intensity mapped to absolute performance $[0,1]$ . To highlight relative performance, we underline the leading SSL method within each specific backbone. Bold entries denote the globally optimal SSL configuration across all evaluated backbones for a given data fraction and initialization mode. Bottom rows indicate the median SSL performance per fraction and initialization. Values have been rounded to two decimal places to enhance legibility relative to Table 2 . Furthermore, font colors within cells dynamically switch between black and white based on background intensity to ensure maximum readability of the results.

Figure S3: Comparative analysis of SSL strategies across backbones and annotated fractions for the linear probing evaluation protocol on the ISF – 5CH dataset. Each cell displays the test set mean macro F1-score $\pm$ standard deviation across three runs for a specific backbone, SSL method, and weights initialization strategy. The left (green) and right (blue) halves of each cell represent random initialization ( $\theta_{\mathrm{Rnd}}$ ) and ImageNet pretraining ( $\theta_{\mathrm{IN}}$ ) respectively, with color intensity mapped to absolute performance $[0,1]$ . To highlight relative performance, we underline the leading SSL method within each specific backbone. Bold entries denote the globally optimal SSL configuration across all evaluated backbones for a given data fraction and initialization mode. Bottom rows indicate the median SSL performance per fraction and initialization. Values have been rounded to two decimal places to enhance legibility relative to Table 2 . Furthermore, font colors within cells dynamically switch between black and white based on background intensity to ensure maximum readability of the results.

Figure S4: Comparative analysis of SSL strategies across backbones and annotated fractions for the XGBoost evaluation protocol on the ISF – 5CH dataset. Each cell displays the test set mean macro F1-score $\pm$ standard deviation across three runs for a specific backbone, SSL method, and weights initialization strategy. The left (green) and right (blue) halves of each cell represent random initialization ( $\theta_{\mathrm{Rnd}}$ ) and ImageNet pretraining ( $\theta_{\mathrm{IN}}$ ) respectively, with color intensity mapped to absolute performance $[0,1]$ . To highlight relative performance, we underline the leading SSL method within each specific backbone. Bold entries denote the globally optimal SSL configuration across all evaluated backbones for a given data fraction and initialization mode. Bottom rows indicate the median SSL performance per fraction and initialization. Values have been rounded to two decimal places to enhance legibility relative to Table 2 . Furthermore, font colors within cells dynamically switch between black and white based on background intensity to ensure maximum readability of the results.

Figure S5: Comparative analysis of SSL strategies across backbones and annotated fractions for the kNN evaluation protocol on the RBC dataset. Each cell displays the test set mean macro F1-score $\pm$ standard deviation across three runs for a specific backbone, SSL method, and weights initialization strategy. The left (green) and right (blue) halves of each cell represent random initialization ( $\theta_{\mathrm{Rnd}}$ ) and ImageNet pretraining ( $\theta_{\mathrm{IN}}$ ) respectively, with color intensity mapped to absolute performance $[0,1]$ . To highlight relative performance, we underline the leading SSL method within each specific backbone. Bold entries denote the globally optimal SSL configuration across all evaluated backbones for a given data fraction and initialization mode. Bottom rows indicate the median SSL performance per fraction and initialization. Values have been rounded to two decimal places to enhance legibility relative to Table 3 . Furthermore, font colors within cells dynamically switch between black and white based on background intensity to ensure maximum readability of the results.

Figure S6: Comparative analysis of SSL strategies across backbones and annotated fractions for the linear probing evaluation protocol on the RBC dataset. Each cell displays the test set mean macro F1-score $\pm$ standard deviation across three runs for a specific backbone, SSL method, and weights initialization strategy. The left (green) and right (blue) halves of each cell represent random initialization ( $\theta_{\mathrm{Rnd}}$ ) and ImageNet pretraining ( $\theta_{\mathrm{IN}}$ ) respectively, with color intensity mapped to absolute performance $[0,1]$ . To highlight relative performance, we underline the leading SSL method within each specific backbone. Bold entries denote the globally optimal SSL configuration across all evaluated backbones for a given data fraction and initialization mode. Bottom rows indicate the median SSL performance per fraction and initialization. Values have been rounded to two decimal places to enhance legibility relative to Table 3 . Furthermore, font colors within cells dynamically switch between black and white based on background intensity to ensure maximum readability of the results.

Figure S7: Comparative analysis of SSL strategies across backbones and annotated fractions for the XGBoost evaluation protocol on the RBC dataset. Each cell displays the test set mean macro F1-score $\pm$ standard deviation across three runs for a specific backbone, SSL method, and weights initialization strategy. The left (green) and right (blue) halves of each cell represent random initialization ( $\theta_{\mathrm{Rnd}}$ ) and ImageNet pretraining ( $\theta_{\mathrm{IN}}$ ) respectively, with color intensity mapped to absolute performance $[0,1]$ . To highlight relative performance, we underline the leading SSL method within each specific backbone. Bold entries denote the globally optimal SSL configuration across all evaluated backbones for a given data fraction and initialization mode. Bottom rows indicate the median SSL performance per fraction and initialization. Values have been rounded to two decimal places to enhance legibility relative to Table 3 . Furthermore, font colors within cells dynamically switch between black and white based on background intensity to ensure maximum readability of the results.

Self-Supervised Learning

Self-supervised learning uses structure within unlabeled data to train representations and models for downstream tasks.

See all 256 papers

May 18, 2026

Task-Aligned Self-Supervised Learning for Medical Image Analysis: A Task-Oriented Review with Practical Design Guidelines

Self-supervised learning (SSL) is increasingly used in medical image analysis to reduce dependence on costly expert annotations by learning transferable representations from unlabeled data. However, SSL performance depends not only on model architecture, but also on whether the pretext task preserves information required by the downstream clinical objective. This review presents a task-oriented synthesis of SSL methods for medical imaging, focusing on how pretext-task design interacts with imaging modality, label availability, and downstream performance. We analyze 75 studies published from 2017 to 2025 and organize them into four paradigms: contrastive learning, non-contrastive and predictive learning, generative and reconstruction-based learning, and hybrid learning. Rather than cataloging methods chronologically, we examine how these paradigms support classification, segmentation, detection, reconstruction, and regression. The evidence suggests that no SSL strategy is universally optimal. Contrastive objectives generally encourage global discriminative representations and are well aligned with classification, but may underrepresent subtle or localized pathology. Spatial prediction, masked modeling, and reconstruction-based objectives better preserve anatomical structure and are often more suitable for segmentation and dense prediction. Hybrid methods can provide balanced representations, although they increase training complexity. Across modalities, SSL is most beneficial in low-label and few-shot regimes, but its effectiveness depends on modality-aware augmentation, pathology-preserving corruption, and clinically meaningful evaluation. We conclude with practical design guidelines and identify open challenges, including pathology-aware pretext tasks, resource-efficient training for high-dimensional data, and standardized evaluation protocols.

Chathura Wimalasiri, Kishor Nandakishor, Marimuthu Palaniswami

Department of Electrical and Electronic Engineering, University of Melbourne, Parkville, VIC 3052, Australia

Contrastive Learning Self-Supervised Learning Medical Image Analysis Medical Image Representation Learning

Ask PDF

Sep 24, 2026

Learning a Flow to Self-Supervised Representations

Explicit geometric references offer a direct way to structure self-supervised representations. Existing adversarial distribution-matching formulations, however, require costly encoder-critic optimization. We introduce Flow-Based Distribution Matching (FBDM), a non-adversarial framework that learns this reference-directed geometry through spherical conditional velocity regression. An ETF-inspired reference allows its number of components K' to exceed the auxiliary flow dimension d* while retaining structured geometric separation. We assign both augmented views of each image to the same target, while limiting how many images each reference center can receive. An explicit alignment loss further pulls the two views' representations closer together. Experiments across benchmarks ranging from CIFAR to ImageNet show that FBDM achieves performance nearly on par with DM and remains competitive with existing SSL methods. Matched training-cost comparisons show a 1.48- to 1.83-fold speedup over DM with a negligible increase in GPU memory usage. We also provide a theoretical explanation for the usefulness of the learned representations: under stated conditions, we bound the downstream misclassification rate in terms of the FBDM pretraining loss.

Yuling Jiao, Wensen Ma, Houduo Qi +1

School of Artificial Intelligence, National Center for Applied Mathematics in Hubei, Hubei Key Laboratory of Computational Science, Wuhan University, Wuhan, China. · Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong SAR, China. · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong SAR, China.

Flow Matching Representation Learning Geometric Representation Learning Self-Supervised Visual Representation Learning

Ask PDF

Aug 27, 2026

Domain-Specific Self-Supervised Representation Learning for Retinal Fundus Classification

Despite the growing number of public datasets, annotated medical images remain scarce. Supervised learning methods achieve strong performance on many benchmarks, however require large amounts of labeled data, which are costly and time-consuming to obtain in the medical domain. To address this limitation, contrastive self-supervised learning (SSL) has emerged as a promising alternative for learning useful representations from unlabeled data. In this work, we investigate two SSL frameworks, SimSiam and SimCLR, for retinal disease classification from fundus images. We focus on understanding how augmentation strategies and training parameters influence representation learning under resource-constrained settings. Given limited data and computational capacity, we explore the feasibility of training SSL models with small batch sizes incorporated with retinal-specific augmentation techniques. Through a series of experiments, we assess the quality of learned representations via linear evaluation and fine-tuning across downstream tasks, including multi-disease classification and diabetic retinopathy grading. Our results show that tailoring augmentation strategies to the characteristics of retinal images plays a critical role in improving performance. Even under constrained settings, lightweight SSL frameworks can learn transferable representations that reduce dependence on large annotated datasets and achieve competitive results.

Bekzat Nurlanbekova, Fung Fung Ting

School of Information Technology Monash University Malaysia, Malaysia

Retinal Disease Classification Medical Image Representation Learning Self-Supervised Learning Self-Supervised Visual Representation Learning Medical Image Classification

Ask PDF

Abstract

Self-Supervised Learning

Explore similar work

Task-Aligned Self-Supervised Learning for Medical Image Analysis: A Task-Oriented Review with Practical Design Guidelines

Learning a Flow to Self-Supervised Representations

Domain-Specific Self-Supervised Representation Learning for Retinal Fundus Classification