Can You Trust Frozen Hematology Foundation Models under Acquisition Shift?

Jai Kumar Sharma; Peeyush Tapadiya

cs.CVAug 25, 2026

Can You Trust Frozen Hematology Foundation Models under Acquisition Shift?

Authors: Jai Kumar Sharma, Peeyush Tapadiya

Organizations: Virginia Tech · Accenture

Abstract

Frozen hematology foundation-model (FM) embeddings reach near-saturated in-domain white-blood-cell (WBC) accuracy, but clinical deployment demands reliability across scanners, sites, stains and preparation pipelines. We audit 15 frozen encoders (hematology, pathology, and general vision) across four public single-cell acquisition domains along two axes: accuracy robustness and calibration. In-domain linear-probe macro-F1 is saturated (0.98-0.997), yet cross-dataset macro-F1 drops 34-72% and rankings re-order: DinoBloom-L, the in-domain best, falls to 10th of 15 on the most-shifted target (MLL23) at the benchmark's shared 224-px input, behind RedDino and several general and pathology encoders. Rank transfer is probe-dependent: 1-NN retrieval is more stable on average than a source-fitted linear head (median $ρ$ 0.65 vs 0.45), but neither probe universally predicts target robustness. Calibration also collapses: source-trained probes are nearly calibrated in-domain (expected calibration error, ECE, 0.004) but confidently wrong off-domain (ECE 0.35), and source-fitted temperature scaling transfers poorly. We further audit pretraining exposure and identify MLL23 as DinoBloom's internal cohort; because DinoBloom's only held-out dataset is also our source domain, this benchmark cannot isolate exposure from scanner-associated shift. Label-free adaptation and marginal-entropy-based model selection appear safe under balanced evaluation but fail under realistic WBC class-prior shift. Class-Balanced Re-standardization (CBR), a training-free pseudo-label-balanced feature normalization, improves all evaluated target-prior scenario means and partially improves calibration, although encoder-level exceptions and residual miscalibration remain. Hematology FM benchmarks must therefore jointly audit accuracy, calibration, exposure, and class-prior robustness.

Figures & tables

encoder	Acevedo ‡	Matek	MLL23	MLL23 #	Raabin
RedDino (hema)	0.994	0.544	0.704	1	0.450
DinoBloom-S (hema)	0.995	0.613	0.671	2	0.341
DINOv2-B	0.991	0.595	0.650	3	0.444
DINOv2-S	0.988	0.505	0.634	4	0.248
Lunit-DINO (path)	0.995	0.409	0.609	5	0.288
DINOv2-L	0.991	0.555	0.588	6	0.315

Table 1: Cross-dataset leaderboard (linear-probe macro-F1, 5 seeds; source = Acevedo), sorted by MLL23. The “MLL23 #” column makes the re-ranking legible: the in-domain #1 (DinoBloom-L) falls to 10th of 15 on the most-shifted target (

0.15

macro-F1 behind the best encoder, RedDino). Bold = strict column max. The supervised ResNet-18 (scratch, below the rule) is a non-frozen baseline, not part of the 15-encoder ranking. Paired-bootstrap 95% CIs for the key MLL23 comparisons (Suppl. A.9) all exclude zero. Exposure status audited in Sec. 4.

Figure 1: Clean-to-target rank transfer is probe-dependent . Spearman $\rho$ between in-domain and shifted-target encoder rankings ( $n{=}15$ ), linear (left) and 1-NN (right). 1-NN is more stable on average (median $\rho$ 0.65 vs 0.45) but not uniformly predictive (Acevedo $\to$ Raabin 0.34). MLL23 (red, largest shift) is hardest for both and is not leakage-free for DinoBloom (Sec. 4; ${*}$ = in DinoBloom pretraining corpus).

Figure 2: Axis B (calibration). Left: reliability diagram (15 encoders pooled; Acevedo $\to$ MLL23): target probes sit below the diagonal (confidently wrong). Right: target ECE by arm (mean over 3 targets $\times$ 15 encoders): source temperature scaling barely helps; CBR + source-T partially helps; only oracle target-T (needs target labels, not deployable) substantially improves it, still above in-domain.

Figure 3: Label-free re-standardization under target class imbalance, from one canonical experiment (15 encoders $\times$ 5 source seeds $\times$ 25 draws $\times$ 3 targets $\times$ 6 priors). Global target standardization (BN-adaptation) hurts on realistic neutrophil-dominated priors, while CBR (class-balanced, ours) is positive in all 18 evaluated target-prior scenario means (bars = mean gain over no adaptation, dots = per target; per-encoder breakdown: Suppl. A.12.1).

Figure S1: (A.10) Marginal prediction entropy appears oracle-matching under balanced target sampling but fails under realistic skewed WBC priors. We report this as a failure case, not a recommended selection rule.

White Blood Cell Classification

See all 11 papers

Oct 6, 2026

One-Slide Calibration of Pathology Foundation Models

Scanner variation changes how pathology foundation models represent the same tissue. We introduce SlideRuler, which uses other regions within a slide as internal controls to predict a query region's scanner-induced shift. A transfer map learned from paired rescans enables calibration from a single scan at inference while keeping the foundation model fixed. Across two encoders and five SCORPION scanners, learned transfer reduces mean target-to-source embedding distance by 16.3-38.5% relative to raw embeddings. Replacing controls with unrelated same-scanner regions reveals a positive same-slide contribution in all four evaluation settings, including scanner holdout. A source-anchored variant reduces source-feature displacement by 47.7-83.6% relative to learned transfer while retaining most of its alignment gain. By drawing calibration information from the slide itself, SlideRuler offers a path toward more consistent use of frozen pathology models across imaging systems.

Ming Ren Hou, Tianyi Huang

Ryquo

Computational Pathology Pathology Foundation Models

Ask PDF

Oct 27, 2026

PRISM: Pathology Reliability In Scarce-label Medicine. A Tile-level Benchmark of Pathology Foundation Models

Pathology foundation models are typically benchmarked on classification performance at full label availability and on in-distribution test sets, while calibration, behavior at low label budgets, and robustness under cross-institutional shift are seldom evaluated jointly. We introduce PRISM, a tile-level benchmark that measures the reliability of pathology foundation models along four axes: model choice, label fraction, calibration, and out-of-distribution transfer. PRISM evaluates eight foundation models (CLIP, PLIP, CONCH, UNI, Virchow2, GigaPath, H-Optimus-0, Midnight) on six tile-level datasets at six label fractions (1% to 100%) with three seeds (864 linear probe runs), extended with a five-hospital Camelyon17 study of 20 directed transfer pairs. Four phenomena emerge. (i) Under full supervision, AUROC rank predicts post-temperature calibration rank consistently across datasets (Spearman rho = 0.52, I^2 = 0%), whereas at 1% labels the relationship becomes dataset-specific (rho = -0.24, I^2 = 67%). (ii) On harder datasets the linear probe collapses to a single predicted class below a dataset-specific label fraction. (iii) Whether additional source-domain labels help or harm target calibration tracks how well the probe transfers: below a transfer AUROC of roughly 0.85, more labels worsen target ECE in 86-88% of combinations. (iv) Temperature scaling fails to reduce ECE in 49 of 288 cells, miscalibration is predominantly under-confidence, and a source-fitted temperature is harmful under failing transfer. PRISM is released as a pip-installable package with reference results for all 288 cells. Code and data: https://github.com/alicemkyn/prism-benchmark

Alicem Koyun

Westcliff University

Computational Pathology Pathology Foundation Models Medical Image Benchmarks Histopathology Image Classification Medical Image Classification OOD Generalization

Ask PDF

Jan 16, 2026

Auditing Frozen-Encoder Anomaly Detection Across Mechanical Systems: Representation Provenance, Calibration, and Protocol Effects

This version reports a reproducibility audit of the frozen-encoder experiments presented in version 1. The numerical discrimination results are reproducible from the preserved artifacts, but their original attribution to interferometric pretraining is not supported. The released checkpoint contains a nested model state that loads without missing parameters, whereas loading the outer checkpoint dictionary leaves almost the entire EfficientNet-B0 feature stack uninitialized. Preserved embeddings labelled as interferometric have norms of order

10^{-12}

, matching freshly initialized EfficientNet-B0 networks and differing by more than twelve orders of magnitude from the preserved ImageNet embeddings. A second, separately preserved near-zero embedding set produces almost the same IMS 4th-test anomaly scores (

r=0.987

) and record-level discrimination (AUC

0.9812

versus

0.9818

). We therefore withdraw the causal claim that IMS performance demonstrates a morphological prior transferred from gravitational-wave instrumentation. We reanalyse the controlled IMS splits at matched observed false-positive rates and add multivariate classical signal baselines. The near-zero representations retain strong tail separation, particularly in the 2nd and 4th IMS runs, but this is now interpreted as an exploratory architecture-and-initialization effect coupled to Mahalanobis scoring. A separate PRONOSTIA audit shows that the original large warning times were induced by a lifetime-fraction baseline; under fixed-time evaluation, a ten-feature classical baseline outperforms the preserved encoder scores. These results illustrate how checkpoint provenance, finite-sample calibration, architecture, and target-domain baselines can create an appearance of cross-domain transfer. They also define the controls required before assigning physical meaning to frozen-representation anomaly scores.

Jose Sánchez Andreu

Independent Researcher, Murcia, Spain

Mahalanobis Distance Industrial Anomaly Detection Predictive Maintenance Unsupervised Anomaly Detection Anomaly Detection

Ask PDF

Abstract

White Blood Cell Classification

Explore similar work

One-Slide Calibration of Pathology Foundation Models

PRISM: Pathology Reliability In Scarce-label Medicine. A Tile-level Benchmark of Pathology Foundation Models

Auditing Frozen-Encoder Anomaly Detection Across Mechanical Systems: Representation Provenance, Calibration, and Protocol Effects