Organizations: Westcliff University, Irvine, CA, United States. · George Mason University, Fairfax, VA, United States. · University of Miami, Coral Gables, FL, United States.
Expert annotation in digital pathology is expensive, and low-label regimes remain common in realistic histopathology workflows. Self-supervised learning (SSL) offers a practical way to learn transferable representations from unlabeled pathology images, but performance-only evaluation can hide unreliable confidence behavior under domain shift. We present a reliability-aware benchmark of SimCLR, BYOL, and an MAE-inspired masked-reconstruction baseline on PatchCamelyon (PCam; 96x96) and PANDA (prostate cancer) under 1%, 5%, and 10% labeled regimes with frozen-encoder linear probing. To address common single-run concerns, ResNet-18 results are repeated over 3 splits x 3 seeds (n=9) and ViT-B/16 over 1 split x 3 seeds (n=3) under practical compute constraints, with mean+/-std reporting and per-run logs. Beyond AUROC, accuracy, and F1-score, we report Expected Calibration Error and Brier score, OOD confidence diagnostics for cross-dataset PCam<->PANDA scoring, federated-vs-centralized SimCLR comparisons on PCam, and cross-dataset transfer to colorectal histology tiles. SSL improves label efficiency on the more controlled PCam task, while PANDA's noisier patch-level proxy can remain near coin-flip with higher variance. The study highlights that histopathology SSL should be assessed not only by ranking performance but also by calibration, robustness, and transparent reproducibility diagnostics before clinical interpretation.
Figures & tables
Split
Train 0/1
Val 0/1
Overlap
0
844 / 854
211 / 214
0
1
844 / 854
211 / 214
0
2
844 / 854
211 / 214
0
Table 1: PANDA binary proxy split checks. The experimental subset contains 1055 class-0 and 1068 class-1 samples, with stratified train/validation partitions and zero index overlap.
Figure 1: Label efficiency under 1–10% labelled regimes. AUROC is reported as mean ± std. ResNet-18 uses n=9 ; ViT-B/16 uses n=3 with one fixed split.
Dataset
Model / method
Labels
AUROC
ECE
PCam
ResNet-18 SimCLR
1%
0.6760±0.1060
0.1174±0.0157
PCam
ResNet-18 SimCLR
10%
0.8685±0.0049
0.0828±0.0079
PCam
ResNet-18 BYOL
10%
0.8599±0.0053
0.0900±0.0075
PANDA
ResNet-18 BYOL
1%
0.5854±0.1011
0.0871±0.0486
PANDA
ResNet-18 BYOL
10%
0.5239±0.0737
0.0699±0.0389
PANDA
ViT-B/16 BYOL
10%
0.7176±0.0178
0.0806±0.0208
Table 2: Selected low-label and 10% label results. A compact view of the most informative quantitative findings; full metric tables are omitted to respect the CIBB page limit.
Figure 2: PCam calibration and probability-quality diagnostics. Lower values are better for both ECE and Brier score. ResNet-18 SimCLR/BYOL improve markedly as the label fraction increases.
Study
Setting
AUROC
ECE / note
Federated SSL
PCam 1%, centralized SimCLR
0.687±0.082
ECE 0.101±0.006
Federated SSL
PCam 1%, federated SimCLR
0.860±0.003
ECE 0.004±0.001
Federated SSL
PCam 10%, centralized SimCLR
0.869±0.004
ECE 0.086±0.008
Federated SSL
PCam 10%, federated SimCLR
0.874±0.001
ECE 0.006±0.002
MAE baseline
PCam 1%, MAE-inspired
0.708±0.006
ECE 0.089±0.018
OOD diagnostic
PANDA → PCam 5%
raw 0.031±0.025
flip 0.969±0.025
Table 3: Diagnostic and extension results. Compact summary of the main non-benchmark evidence.
Figure 3: OOD diagnostics between PCam and PANDA. AUROC flip is shown only to diagnose score inversion. Lower OOD uncertainty than ID uncertainty indicates overconfident OOD predictions.
Figure 4: PCA projections of SSL embeddings for PCam and PANDA. Left: SimCLR; right: BYOL. PANDA samples form a compact, distinct region, consistent with domain-shift behavior.
Pathology foundation models are typically benchmarked on classification performance at full label availability and on in-distribution test sets, while calibration, behavior at low label budgets, and robustness under cross-institutional shift are seldom evaluated jointly. We introduce PRISM, a tile-level benchmark that measures the reliability of pathology foundation models along four axes: model choice, label fraction, calibration, and out-of-distribution transfer. PRISM evaluates eight foundation models (CLIP, PLIP, CONCH, UNI, Virchow2, GigaPath, H-Optimus-0, Midnight) on six tile-level datasets at six label fractions (1% to 100%) with three seeds (864 linear probe runs), extended with a five-hospital Camelyon17 study of 20 directed transfer pairs. Four phenomena emerge. (i) Under full supervision, AUROC rank predicts post-temperature calibration rank consistently across datasets (Spearman rho = 0.52, I^2 = 0%), whereas at 1% labels the relationship becomes dataset-specific (rho = -0.24, I^2 = 67%). (ii) On harder datasets the linear probe collapses to a single predicted class below a dataset-specific label fraction. (iii) Whether additional source-domain labels help or harm target calibration tracks how well the probe transfers: below a transfer AUROC of roughly 0.85, more labels worsen target ECE in 86-88% of combinations. (iv) Temperature scaling fails to reduce ECE in 49 of 288 cells, miscalibration is predominantly under-confidence, and a source-fitted temperature is harmful under failing transfer. PRISM is released as a pip-installable package with reference results for all 288 cells. Code and data: https://github.com/alicemkyn/prism-benchmark
Semi-supervised semantic segmentation in computational pathology remains challenging due to scarce pixel-level annotations and unreliable pseudo-label supervision. We propose UniSemAlign, a dual-modal semantic alignment framework that enhances visual segmentation by injecting explicit class-level structure into pixel-wise learning. Built upon a pathology-pretrained Transformer encoder, UniSemAlign introduces complementary prototype-level and text-level alignment branches in a shared embedding space, providing structured guidance that reduces class ambiguity and stabilizes pseudo-label refinement. The aligned representations are fused with visual predictions to generate more reliable supervision for unlabeled histopathology images. The framework is trained end-to-end with supervised segmentation, cross-view consistency, and cross-modal alignment objectives. Extensive experiments on the GlaS and CRAG datasets demonstrate that UniSemAlign substantially outperforms recent semi-supervised baselines under limited supervision, achieving Dice improvements of up to 2.6% on GlaS and 8.6% on CRAG with only 10% labeled data, and strong improvements at 20% supervision. Code is available at: https://github.com/thailevann/UniSemAlign
Le-Van Thai, Tien Dat Nguyen, Hoai Nhan Pham +3
AI VIETNAM Lab, Vietnam · Hanoi University of Science and Technology, Vietnam
Foundation models for computational pathology are usually evaluated using AUC and accuracy, while calibration is often left untested. This matters because a model can be accurate on average but still assign overly confident probabilities to cases that are difficult even for pathologists. We study calibration across eight pathology foundation models. Using pathologist agreement as a measure of diagnostic difficulty, we find that calibration error is consistently higher on low-agreement cases than on high-agreement cases. This pattern is not apparent from aggregate expected calibration error (ECE) alone. We then propose synthetic agreement calibration, a method for improving calibration without collecting multi-annotator labels. Given a trained linear probe, we select high-confidence embeddings as class anchors and interpolate between anchors from opposite classes. The interpolation weights encode a continuous notion of diagnostic ambiguity, which we use as a synthetic agreement signal to retrain the probe with agreement-aware label smoothing. On MHIST, which includes annotations from seven pathologists, synthetic agreement calibration recovers most of the calibration improvement obtained by label smoothing based on real pathologist agreement, while substantially reducing low-agreement ECE relative to the uncalibrated baseline. Discrimination metrics are preserved. On PatchCamelyon and BreakHis, public histopathology datasets without multi-annotator labels, the method improves calibration across the evaluated foundation models, whereas annotator-dependent approaches cannot be used without additional expert annotation.
Wenjun Liu, Saeed Hassanpour
Department of Computer Science Dartmouth College Hanover, NH, USA · Departments of Biomedical Data Science, Computer Science, and Epidemiology Dartmouth College, Hanover, NH, USA