cs.CVNov 19, 2026

When Foundation Models Disagree: Representational Heterogeneity for Calibrated Uncertainty in Whole-Slide Image Classification

Authors: Yekai Shen, Yuqian Wang, Zishun Liao, Rongsheng Wang, Wenting Chen, S. Kevin Zhou, Qingsong Yao

Organizations: University of Science and Technology of China · Stanford University · Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE) Lab · Jiangsu Key Laboratory of Multimodal Digital Twin Technology · 

Abstract

Pathology foundation models supply strong representations for whole-slide image classification, yet the confidence attached to their predictions remains hard to trust. Prevailing uncertainty estimators act within a single feature space and thereby overlook the uncertainty that the choice of foundation-model representation itself induces. This work introduces CALIFUSE, a calibration-oriented framework that fuses several pathology foundation models for the specific purpose of uncertainty estimation. Departing from accuracy-driven fusion, CALIFUSE reads representational heterogeneity across encoders as an explicit confidence cue. Slide-level decision consensus is coupled with chunk-level evidence agreement quantified through Jensen-Shannon divergence, so that both prediction stability and fine-grained evidence consistency inform the estimate. Foundation-model encoders stay frozen. A task-specific nnMIL classifier is still trained for each FM; CALIFUSE then estimates confidence at inference without training an extra fusion module. Evaluated on seven WSI classification tasks under several foundation-model combinations, the framework sharpens calibration, misclassification detection, and selective prediction relative to both the strongest single-encoder baseline and naive ensemble averaging, all while preserving competitive balanced accuracy.

Figures & tables

Explore similar work

CardsList
  1. Uncertainty Estimation in Pathology Foundation Models via Deep Mutual Learning

    Jun 29, 2026Gbègninougbo Aurel Davy Tchokponhoue, Sevda Öğüt, Ali Idri +2Pathology Foundation ModelsDeep Ensembles

  2. Do Center Biases Propagate? Robustness of Pathology Foundation Models in Whole-Slide Image Classification

    Sep 23, 2026Ilán Carretero, Pablo Meseguer, Rocío del Amor +1Batch Effect CorrectionPathology Foundation Models

  3. Probing, Fusion, and Trustworthiness: A Systematic Evaluation of Foundation Model Representations for Multimodal Cancer Analysis

    Jun 15, 2026Jingyu Hu, Giuseppe Tripodi, Reed Naidoo +2Computational PathologyMultimodal Robustness