Reliability-Aware Benchmarking of Self-Supervised Learning in Histopathology Under 1-10% Label Regimes
Organizations: Westcliff University, Irvine, CA, United States. · George Mason University, Fairfax, VA, United States. · University of Miami, Coral Gables, FL, United States.
Abstract
Expert annotation in digital pathology is expensive, and low-label regimes remain common in realistic histopathology workflows. Self-supervised learning (SSL) offers a practical way to learn transferable representations from unlabeled pathology images, but performance-only evaluation can hide unreliable confidence behavior under domain shift. We present a reliability-aware benchmark of SimCLR, BYOL, and an MAE-inspired masked-reconstruction baseline on PatchCamelyon (PCam; 96x96) and PANDA (prostate cancer) under 1%, 5%, and 10% labeled regimes with frozen-encoder linear probing. To address common single-run concerns, ResNet-18 results are repeated over 3 splits x 3 seeds (n=9) and ViT-B/16 over 1 split x 3 seeds (n=3) under practical compute constraints, with mean+/-std reporting and per-run logs. Beyond AUROC, accuracy, and F1-score, we report Expected Calibration Error and Brier score, OOD confidence diagnostics for cross-dataset PCam<->PANDA scoring, federated-vs-centralized SimCLR comparisons on PCam, and cross-dataset transfer to colorectal histology tiles. SSL improves label efficiency on the more controlled PCam task, while PANDA's noisier patch-level proxy can remain near coin-flip with higher variance. The study highlights that histopathology SSL should be assessed not only by ranking performance but also by calibration, robustness, and transparent reproducibility diagnostics before clinical interpretation.
Figures & tables
| Split | Train 0/1 | Val 0/1 | Overlap |
|---|---|---|---|
| 0 | 844 / 854 | 211 / 214 | 0 |
| 1 | 844 / 854 | 211 / 214 | 0 |
| 2 | 844 / 854 | 211 / 214 | 0 |
| Dataset | Model / method | Labels | AUROC | ECE |
|---|---|---|---|---|
| PCam | ResNet-18 SimCLR | 1% | ||
| PCam | ResNet-18 SimCLR | 10% | ||
| PCam | ResNet-18 BYOL | 10% | ||
| PANDA | ResNet-18 BYOL | 1% | ||
| PANDA | ResNet-18 BYOL | 10% | ||
| PANDA | ViT-B/16 BYOL | 10% |
| Study | Setting | AUROC | ECE / note |
|---|---|---|---|
| Federated SSL | PCam 1%, centralized SimCLR | ECE | |
| Federated SSL | PCam 1%, federated SimCLR | ECE | |
| Federated SSL | PCam 10%, centralized SimCLR | ECE | |
| Federated SSL | PCam 10%, federated SimCLR | ECE | |
| MAE baseline | PCam 1%, MAE-inspired | ECE | |
| OOD diagnostic | PANDA PCam 5% | raw | flip |
Self-Supervised Learning
Self-supervised learning uses structure within unlabeled data to train representations and models for downstream tasks.
See all 279 papers