Zero-shot whole-slide image (WSI) prediction aims to generalize to new pathology tasks without parameter updates at deployment. MI-Zero makes these predictions by pooling patch-text similarities. Conditional weight generation, combined with supervision from diverse pathology tasks, offers an alternative: learning to construct predictive rules from task descriptions. We introduce MI-Hyper, a hypernetwork pretrained across mixed tasks to generate the aggregation and classification weights of multiple instance learning (MIL) models. We first train the generator and a shared MIL basis jointly using source labels. We then freeze the basis and refine the generator. The generated task weights are reused across cases. Across 308 tasks from 52 pathology datasets, MI-Hyper improves mean AUC, balanced accuracy and macro-F1 over MI-Zero by 1.85, 1.17 and 0.57 percentage points, respectively. On some molecular prediction tasks, both AUC and balanced accuracy improve by more than 20 points. These results support task-conditioned weight generation as a promising approach to weight-space transfer for long-sequence modeling.
Figures & tables
Figure 1: Constructing predictors from tiled WSIs. (a) MI-Zero pools patch–text similarity scores. (b) MI-Hyper constructs an MIL predictor using generated weights.
Figure 2: Mixed-task learning and zero-shot deployment of MI-Hyper. (a) The task registry covers 52 datasets, 308 tasks and 112 semantic groups. The global training partition contains 20,420 patients and 111,151 task–case pairs. Each fold optimizes on eligible source tasks and holds out target task IDs; cases may recur across source and target tasks. (b) Joint source-label learning is followed by fixed-basis refinement with cross-entropy and auxiliary losses (Eq. ( 8 )). (c) Task and class descriptions generate the aggregation query and classifier weights, which are reused across cases without target-task parameter updates.
Algorithm 1: MI-Hyper learning and inference
# Inputs: source FIT S, DEV D, descriptions,
# and query bags; encoders remain frozen
# Sample at most 2048 patches/training bag
# 1. Joint CE: task context + morphology text
W0, phi = initialize()
anchor = fit_ce(S, W0, phi, text=morphology,
Table 3
AUC ↑
BAcc ↑
Endpoint type
Task Num
MI-Zero
MI-Hyper
Δ (pp)
MI-Zero
MI-Hyper
Δ (pp)
Mutation
122
49.24
54.35
+5.11
49.57
53.11
+3.54
Stage
69
54.89
56.85
+1.96
38.45
40.71
+2.26
Molecular subtype
38
55.48
54.92
-0.56
30.53
32.31
+1.78
Grade
29
67.65
68.02
+0.37
44.35
44.94
+0.59
Histological subtype
19
81.84
72.33
-9.51
58.94
46.37
-12.58
Table 1: Results by endpoint type. AUC and BAcc (%) for all 16 endpoint types. Task Num denotes the number of tasks; tasks within each group are equally weighted. Δ is MI-Hyper minus MI-Zero in percentage points, computed before rounding. Gains are green and decreases gray; higher scores in non-summary rows are bold. All three metrics are reported in the supplement.
AUC ↑
BAcc ↑
Anatomy
Task Num
MI-Zero
MI-Hyper
Δ (pp)
MI-Zero
MI-Hyper
Δ (pp)
Colorectal
37
55.40
59.62
+4.22
46.98
50.28
+3.30
Lung
34
55.62
55.87
+0.25
44.83
45.50
+0.66
Uterus
29
51.82
60.55
+8.74
48.36
54.60
+6.24
Breast
27
59.28
55.66
-3.62
43.81
39.93
-3.88
Kidney
24
54.91
57.73
+2.82
44.03
46.15
+2.12
Table 2: Results by anatomical group. AUC and BAcc (%) for all 25 display groups. Task Num denotes the number of tasks; tasks within each group are equally weighted. Formatting follows Table 1 . Lip (reg.) retains the registry label; Retro./perit. denotes retroperitoneum/peritoneum. Full group names and all three metrics are given in the supplement.
AUC ↑
BAcc ↑
Axis
Group
Task Num
MI-Zero
MI-Hyper
Δ (pp)
MI-Zero
MI-Hyper
Δ (pp)
Bag length
≤1,000
33
67.04
66.90
-0.15
49.00
45.55
-3.44
1,000 – 2,500
27
59.46
57.69
-1.77
47.93
47.29
-0.63
2,500 – 5,000
195
55.47
56.62
+1.15
42.49
43.51
+1.02
>5,000
53
51.52
59.06
+7.54
47.27
52.78
+5.51
Class count
Binary
174
53.51
57.37
+3.86
51.83
54.57
+2.73
Table 3: Results by bag length and class count. Task Num denotes the number of tasks. Length groups use the task-average patch count Nˉ : Nˉ≤1,000 , 1,000<Nˉ≤2,500 , 2,500<Nˉ≤5,000 and Nˉ>5,000 . Class-count groups distinguish binary and multiclass tasks. Each partition covers all 308 tasks. AUC and BAcc are percentages. Formatting follows Table 1 . These groups do not isolate length robustness.
AUC ↑
BAcc ↑
Source
Task Num
MI-Zero
MI-Hyper
Δ (pp)
MI-Zero
MI-Hyper
Δ (pp)
TCGA
223
53.70
56.68
+2.98
43.05
45.92
+2.87
Additional
52
69.18
68.76
-0.41
48.67
44.56
-4.11
CPTAC
33
54.33
52.13
-2.20
47.59
45.56
-2.02
All tasks
308
56.38
58.24
+1.85
44.49
45.66
+1.17
Table 4: Results by dataset source. AUC and BAcc (%) with Task Num denoting the number of tasks. All 308 tasks are retained and equally weighted within each source. Source grouping does not alter the five-fold partition of 112 semantic groups. Formatting follows Table 1 .
AUC ↑
BAcc ↑
Task
n
MI-Zero
MI-Hyper
Δ (pp)
MI-Zero
MI-Hyper
Δ (pp)
Lung benign/ malignant
888
69.42
94.43
+25.01
65.58
86.67
+21.09
TCGA-ESCA subtype (CIN vs. ESCC)
141
76.96
94.63
+17.67
62.15
84.25
+22.09
COAD ATRX mutation
434
42.35
65.81
+23.46
45.96
62.86
+16.91
UCEC SETD2 mutation
505
46.47
71.46
+25.00
47.92
64.52
+16.60
SKCM mRNA subtype
291
51.36
65.59
+14.23
29.69
45.04
+15.34
Table 5: Six tasks with substantial gains. AUC and BAcc are percentages; n is the number of bags. Higher scores are bold. Δ is MI-Hyper minus MI-Zero in percentage points, computed before rounding, with gains in green. The lung task is from REG_dataset; ESCA uses the source-defined labels CIN and ESCC. Each example has at least 100 bags and 20 bags per class; the selection rule is given in the text. All 308 tasks contribute to the overall results.
Figure 3: Query similarities and attention maps. (a) Within- and between-task query cosine similarities for 63 tasks and the condition-level similarity matrix for the COAD diagnosis and ATRX mutation tasks. (b) Query distance and Jensen–Shannon divergence of attention distributions on 18 WSIs. Bars show the range across three cases; open markers denote same-label variants. (c) Two COAD WSIs, each shown with H&E and attention maps for histological diagnosis and ATRX mutation prediction. Each map is the mean of seven separately normalized attention distributions. Colors show Nαˉ (0.01–100): blue indicates lower and red higher relative attention; 1 denotes uniform attention. Scale bars: 1 mm.
Figure 4: Query–classifier recombination. Rows denote query donors and columns classifier (W,b) donors. B-A, B-T, U-A and U-T denote BLCA ARID1A, BLCA TP53, UCEC ARID1A and UCEC TP53. Cells give AUC or BAcc (%) averaged over seven conditions on complete target case pools. Purple outlines mark original pairs. Across genes, class rows align wildtype and mutant states.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
MI-Zero
MI-Hyper
Task
N
C
Metric
K= 1
K= 5
K= 10
K= 50
K= 100
Mean K
7-condition mean
REG_dataset
REG_dataset/bladder_grade
348
2
AUC BAcc F1
88.95 64.26 54.12
90.70 63.84 53.41
91.02 63.73 53.26
90.31 62.14 50.72
89.55 60.37 47.81
90.11 62.87 51.87
90.39 50.69 39.43
REG_dataset/bladder_invasion_depth
867
3
AUC BAcc F1
75.40 43.41 39.63
79.35 44.83 40.95
79.83 44.18 40.32
77.22 41.08 36.66
75.26 38.40 33.22
77.41 42.38 38.15
74.77 51.68 47.33
REG_dataset/bladder_invasiveness
808
3
AUC BAcc F1
87.76 64.98 59.20
91.47 66.20 61.56
92.62 67.15 62.21
94.12 68.84 63.83
93.88 71.05 64.38
91.97 67.65 62.24
83.45 42.65 37.06
REG_dataset/bladder_tumor_presence
867
2
AUC BAcc F1
53.66 51.45 30.65
60.84 55.36 30.20
63.98 57.07 30.41
69.56 61.13 31.32
70.69 61.36 31.23
63.75 57.27 30.76
28.24 31.10 20.34
Appendix
Table S1 : All 308 tasks under the same 112-group partition. Each cell lists AUC, BAcc and macro-F1 (%) from top to bottom. Each MI-Zero K column averages metrics over 50 prompts; Mean- K averages all 250 conditions. MI-Hyper uses the fixed five-fold checkpoints and averages metrics over seven predefined paired text conditions. All columns use metric averaging, without selecting the best condition for each task. N is the number of task–bag records and C the number of classes. Blue task IDs mark the six main-text examples; MI-Zero scores are gray.
Vision language models (VLM) pre-trained on datasets of histopathological image-caption pairs enabled zero-shot slide-level classification. The ability of VLM image encoders to extract discriminative features also opens the door for supervised fine-tuning for whole-slide image (WSI) classification, ideally using few labeled samples. Slide-level prediction frameworks require the incorporation of multiple instance learning (MIL) due to the gigapixel size of the WSI. Following patch-level feature extraction and aggregation, MIL frameworks rely on linear classifiers trained on top of the slide-level aggregated features. Classifier weight initialization has a large influence on Linear Probing performance in efficient transfer learning (ETL) approaches based on few-shot learning. In this work, we propose Zero-Shot Multiple-Instance Learning (ZS-MIL) to address the limitations of random classifier initialization that underperform zero-shot prediction in MIL problems. ZS-MIL uses the class-level embeddings of the VLM text encoder as the classification layer's starting point to compute each sample's bag-level probabilities. Through multiple experiments, we demonstrate the robustness of ZS-MIL compared to well-known weight initialization techniques both in terms of performance and variability in an ETL few-shot scenario for subtyping prediction.
Pablo Meseguer, Rocío del Amor, Valery Naranjo
Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano (HUMAN-Tech), Universitat Politècnica de Valencia, Valencia, Spain · Valencian Graduate School and Research Network for Artificial Intelligence (valgrAI), Valencia, Spain
Whole slide image (WSI) analysis is central to computational pathology, with multiple instance learning (MIL) emerging as the standard pipeline for slide-level diagnosis. However, conventional approaches formulate WSI diagnosis as a flat classification task over discrete labels, contradicting the inherently hierarchical, coarse-to-fine nature of clinical reasoning. Although recent hierarchical classifiers and vision-language models (VLMs) have sought to address this structural gap, they either fail to capture semantic continuity between related diagnoses or suffer from unconstrained text generation that produces taxonomic hallucinations and parent-child label violations. To address these limitations, we propose TaxoMIL, a taxonomy-constrained framework that reformulates WSI diagnosis as a multi-granularity text generation task. TaxoMIL utilizes a dual-head Transformer decoder to generate coarse- and fine-level diagnostic text, and introduces taxonomy-guided objectives that explicitly structure the label embedding space and strictly ground slide-level visual representations within the clinical taxonomy. Extensive experiments across three diverse WSI datasets demonstrate that TaxoMIL consistently outperforms state-of-the-art MIL classifiers and VLM-based generative methods, yielding accurate and hierarchy-aware diagnostic predictions. The code is released at https://github.com/QuIIL/TaxoMIL
Chaeyeon Lee, Khang Nguyen Quoc, Jinsol Song +3
School of Electrical Engineering, Korea University, Seoul, Republic of Korea · Department of Hospital Pathology, The Catholic University of Korea College of Medicine, Seoul, Republic of Korea
Multiple instance learning (MIL) has become the main paradigm for whole-slide image (WSI) analysis in computational pathology. However, existing MIL aggregators are still typically trained from scratch for each downstream task, relying on limited slide-level labels to learn both aggregation mechanisms and downstream discriminative representations simultaneously. As a result, they often suffer from unstable optimization, overfitting, and limited transferability. Similar to pretrained ResNet and Vision Transformer models in natural image learning, MIL also requires reusable pretrained initialization. However, high-quality slide-level pretraining data remain scarce, and MIL models are usually lightweight and weakly supervised, making large-scale pretraining difficult in practice. To address this challenge, we propose a distillation-based pretraining framework for MIL, which leverages two slide-level foundation models, TITAN and CARE, as teachers to transfer their representational knowledge into a diverse set of MIL architectures. To effectively balance supervision from different teachers, we further introduce an angular dispersion normalized distillation loss. The distilled weights are then used as initialization for downstream adaptation. We conduct systematic evaluations on 15 benchmark datasets under both linear probing and full-parameter fine-tuning, and further validate its advantages in few-shot scenarios. Experimental results show that pretraining generally improves MIL aggregators over from scratch training, especially in linear-probing and few-shot settings, while maintaining the computational efficiency of lightweight MIL models. Code is available at https://github.com/fu0201/MIL_Pretrained.
Mingxi Fu, Jiawen Li, Renao Yan +4
Tsinghua Shenzhen International Graduate School, Tsinghua University · University of Washington · City University of Hong Kong (Dongguan) +2