When Foundation Models Disagree: Representational Heterogeneity for Calibrated Uncertainty in Whole-Slide Image Classification
Organizations: University of Science and Technology of China · Stanford University · Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE) Lab · Jiangsu Key Laboratory of Multimodal Digital Twin Technology ·
Abstract
Pathology foundation models supply strong representations for whole-slide image classification, yet the confidence attached to their predictions remains hard to trust. Prevailing uncertainty estimators act within a single feature space and thereby overlook the uncertainty that the choice of foundation-model representation itself induces. This work introduces CALIFUSE, a calibration-oriented framework that fuses several pathology foundation models for the specific purpose of uncertainty estimation. Departing from accuracy-driven fusion, CALIFUSE reads representational heterogeneity across encoders as an explicit confidence cue. Slide-level decision consensus is coupled with chunk-level evidence agreement quantified through Jensen-Shannon divergence, so that both prediction stability and fine-grained evidence consistency inform the estimate. Foundation-model encoders stay frozen. A task-specific nnMIL classifier is still trained for each FM; CALIFUSE then estimates confidence at inference without training an extra fusion module. Evaluated on seven WSI classification tasks under several foundation-model combinations, the framework sharpens calibration, misclassification detection, and selective prediction relative to both the strongest single-encoder baseline and naive ensemble averaging, all while preserving competitive balanced accuracy.
Figures & tables
| Task | Prediction target | Cls | WSI / Pat. | Eval |
| Biomarker-derived classification | ||||
| BRCA-TNBC | TNBC vs. non-TNBC | 2 | 721 / 641 | 5 folds |
| GBM/LGG | IDH / 1p19q glioma subtype | 4 | 1572 / 814 | 5 folds |
| Tumor subtyping | ||||
| BLCA-Pap. | Papillary vs. non-pap. | 2 | 452 / 380 | 5 folds |
| Kidney | Renal cancer subtype | 3 | 920 / 890 | 5 folds |
| ECE | Brier | AUROC | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Task | Oracle | Ensemble | CaliFuse | Oracle | Ensemble | CaliFuse | Oracle | Ensemble | CaliFuse |
| BRCA-TNBC | 0.098 0.028 | 0.089 0.036 | 0.059 0.009 | 0.230 0.039 | 0.225 0.042 | 0.212 0.045 | 0.641 0.103 | 0.537 0.076 | 0.734 0.091 |
| BLCA-Papillary | 0.210 0.043 | 0.166 0.026 | 0.084 0.014 | 0.397 0.028 | 0.389 0.034 | 0.370 0.041 | 0.559 0.060 | 0.483 0.111 | 0.584 0.040 |
| Kidney | 0.050 0.013 | 0.034 0.011 | 0.042 0.016 | 0.111 0.027 | 0.115 0.022 | 0.093 0.015 | 0.677 0.071 | 0.641 0.090 | 0.851 0.048 |
| Sarcoma | 0.344 0.070 | 0.335 0.077 | 0.175 0.019 | 0.559 0.068 | 0.562 0.021 | 0.532 0.047 | 0.586 0.081 | 0.525 0.081 | 0.665 0.111 |
| GBM/LGG | 0.155 0.063 | 0.129 0.057 | 0.072 0.025 | 0.308 0.070 | 0.295 0.074 | 0.285 0.080 | 0.707 0.011 | 0.706 0.021 | 0.823 0.031 |
| ECE | Brier | AUROC | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Task | Oracle | Ensemble | CaliFuse | Oracle | Ensemble | CaliFuse | Oracle | Ensemble | CaliFuse |
| BRCA-TNBC | 0.098 0.028 | 0.086 0.029 | 0.068 0.013 | 0.230 0.039 | 0.219 0.041 | 0.211 0.045 | 0.641 0.103 | 0.604 0.050 | 0.759 0.086 |
| BLCA-Papillary | 0.210 0.043 | 0.170 0.036 | 0.097 0.027 | 0.397 0.028 | 0.392 0.041 | 0.371 0.065 | 0.559 0.060 | 0.486 0.118 | 0.602 0.110 |
| Kidney | 0.050 0.013 | 0.040 0.010 | 0.034 0.020 | 0.111 0.027 | 0.111 0.016 | 0.090 0.016 | 0.677 0.071 | 0.648 0.089 | 0.845 0.055 |
| Sarcoma | 0.344 0.070 | 0.346 0.091 | 0.169 0.037 | 0.559 0.068 | 0.558 0.032 | 0.534 0.056 | 0.586 0.081 | 0.538 0.064 | 0.716 0.104 |
| GBM/LGG | 0.155 0.063 | 0.129 0.056 | 0.072 0.028 | 0.308 0.070 | 0.295 0.079 | 0.281 0.076 | 0.707 0.011 | 0.694 0.021 | 0.827 0.035 |
| ECE | Brier | AUROC | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Task | Oracle | Ensemble | CaliFuse | Oracle | Ensemble | CaliFuse | Oracle | Ensemble | CaliFuse |
| BRCA-TNBC | 0.098 0.028 | 0.091 0.025 | 0.049 0.016 | 0.230 0.039 | 0.223 0.033 | 0.212 0.045 | 0.641 0.103 | 0.609 0.024 | 0.737 0.073 |
| BLCA-Papillary | 0.210 0.043 | 0.161 0.058 | 0.106 0.022 | 0.401 0.046 | 0.396 0.047 | 0.378 0.065 | 0.559 0.060 | 0.525 0.087 | 0.572 0.044 |
| Kidney | 0.050 0.013 | 0.040 0.012 | 0.035 0.013 | 0.111 0.027 | 0.113 0.022 | 0.093 0.021 | 0.677 0.071 | 0.637 0.067 | 0.852 0.063 |
| Sarcoma | 0.375 0.102 | 0.349 0.088 | 0.169 0.039 | 0.559 0.068 | 0.564 0.038 | 0.540 0.055 | 0.586 0.081 | 0.537 0.052 | 0.743 0.075 |
| GBM/LGG | 0.159 0.056 | 0.131 0.060 | 0.064 0.022 | 0.308 0.070 | 0.297 0.076 | 0.284 0.071 | 0.707 0.011 | 0.680 0.049 | 0.822 0.043 |
Uncertainty Calibration
Uncertainty calibration aligns model confidence and predictive intervals with observed correctness and outcome frequencies.
See all 296 papers