Dictionary-based Pathology Mining with Hard-instance-assisted Classifier Debiasing for Genetic Biomarker Prediction from WSIs
Authors: Ling Zhang, Boxiang Yun, Ting Jin, Qingli Li, Xinxing Li, Yan Wang
Organizations: Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Shanghai, China · Department of General Surgery, Tongji Hospital, Tongji University School of Medicine, Shanghai 200065, China
Prediction of genetic biomarkers, e.g., microsatellite instability in colorectal cancer is crucial for clinical decision making. But, two primary challenges hamper accurate prediction: (1) It is difficult to construct a pathology-aware representation involving the complex interconnections among pathological components. (2) WSIs contain a large proportion of areas unrelated to genetic biomarkers, which make the model easily overfit simple but irrelative instances. We hereby propose a Dictionary-based hierarchical pathology mining with hard-instance-assisted classifier Debiasing framework to address these challenges, dubbed as D2Bio. Our first module, dictionary-based hierarchical pathology mining, is able to mine diverse and very fine-grained pathological contextual interaction without the limit to the distances between patches. The second module, hard-instance-assisted classfier debiasing, learns a debiased classifier via focusing on hard but task-related features, without any additional annotations. Experimental results on five cohorts show the superiority of our method, with over 4% improvement in AUROC compared with the second best on the TCGA-CRC-MSI cohort. Our analysis further shows the clinical interpretability of D2Bio in genetic biomarker diagnosis and potential clinical utility in survival analysis. Code will be available at https://github.com/DeepMed-Lab-ECNU/D2Bio.
Figures & tables
Fig. 1: Motivation of our dictionary-based strategy. (a) Attention-based MIL methods: simply model the relationships between instances. (b) Graph-based methods: model instance relationships without construction of real pathological components. (c) Proposed dictionary-based method: utilizes a learnable dictionary to group instances into pathological components and hierarchically mine pathological contextual interaction.
Fig. 2: Motivation of our hard-instance-assisted classifier debiasing strategy. Sufferring from redundant task-irrelevant instances, during inference, biased classifier misjudges the bag-level labels by overfitting simple instances. Learning hard instances which include task-specific instances can assist the classifier to reduce the bias.
Fig. 3: Illustration of D 2 Bio. The overall framework consists of two parts: 1) dictionary-based hierarchical pathology mining, 2) hard-instance assisted-classifier debiasing. Given a WSI X , our D 2 Bio first initialize a learnable dictionary to extract pathological information of X via cross attention operation. Then our D 2 Bio groups instances into fine-grained pathological groups according to the similarity matrix. To hierarchically mine the interaction, Multi-head Self-attention (MSA) is first employed in each group. Features of X are updated by ungrouping these groups, which is further used to update the dictionary. After repeating the above steps L times, inter-group ViT is employed. Finally, our D 2 Bio assigns hard instance pseudo labels via unsupervised clustering to supervise the classfication head to reduce the bias in WSIs. The purple arrow indicates the classification branch and the orange arrow indicates the classifier debiasing branch.
CtransPath [ 50 ]
Method
TCGA-CRC-MSI
TCGA-CRC-BRAF
CPTAC-MSI
CPTAC-BRAF
TCGA-STAD-MSI
AUC
Acc
PRC
Bal
AUC
Acc
PRC
Bal
AUC
Acc
PRC
Bal
AUC
Acc
PRC
Bal
AUC
Acc
PRC
Bal
ABMIL [ 11 ]
88.00 ∗
80.80 ∗
64.80
86.20
68.40 ∗
80.80
24.80
54.90 ∗
87.50
76.20 ∗
49.10
50.00 ∗
85.20
85.70 ∗
56.90
50.00 ∗
77.80
65.70 ∗
46.80 ∗
67.80 ∗
(75.5-96.6)
(73.3-87.5)
(46.0-83.1)
(77.6-92.4)
(46.1-87.6)
(74.2-87.5)
(6.4-55.7)
(41.5-71.0)
(68.8-100.0)
(76.2-76.2)
(29.8-100.0)
(50.0-50.0)
(72.6-99.4)
(85.7-85.7)
(21.8-95.8)
(50.0-50.0)
(67.8-87.2)
(56.6-74.7)
(35.2-68.2)
(57.7-77.8)
DSMIL [ 12 ]
88.20 ∗
87.50 ∗
62.70
81.40
63.80 ∗
76.70 ∗
24.70
52.70 ∗
91.20
76.20
57.30
56.90 ∗
87.00
85.70
38.80
50.00
77.10
63.60 ∗
47.70 ∗
63.80 ∗
(76.1-97.5)
(81.7-93.3)
(40.2-87.9)
(68.6-92.4)
(40.5-83.8)
(70.0-83.3)
(5.6-56.8)
(39.3-69.2)
(76.2-100.0)
(61.9-85.7)
(34.4-100.0)
(40.6-76.9)
(78.1-92.2)
(85.7-85.7)
(22.1-75.2)
(50.0-50.0)
(66.7-86.6)
(54.5-72.7)
(35.2-69.3)
(53.1-73.8)
TABLE I: Comparisons with state-of-the-art MIL methods on five cohorts using two feature extractors, i.e. , CtransPath [ 50 ] and UNI v2 [ 51 ] . We report Mean (95% CI) in percentage (%). “*” denotes statistical significance (p < 0.05). Best results are shown in bold .
Fig. 4: Heatmap visualization of D 2 Bio on the MSI prediction task and identified pathological patterns.
Pathology Mining
Debiasing
TCGA-CRC-MSI
TCGA-STAD-MSI
×
×
88.63
64.05
×
✓
91.49
79.73
×
88.70
64.32
✓
×
93.46
80.00
✓
✓
96.70
81.73
TABLE II: Ablation study of two key components on TCGA-CRC-MSI cohort. w/o Mining: All instances are fed into two ViT layers without dictionary-guided instance grouping. The symbol “*” indicates only inter-group interaction mining is conducted without intra-group interaction mining. w/o Debiasing: Remove the hard-instance-assisted classifier debiasing module.
Cohort
Rand.
Text
K-means
k-NN (dist)
k-NN (cos)
Dic.
TCGA-CRC-MSI
89.46
91.75
92.83
92.00
90.86
96.70
TCGA-STAD-MSI
78.97
78.70
75.62
65.73
66.49
81.73
TABLE III: Ablation study of grouping strategies on the TCGA-CRC-MSI and TCGA-STAD-MSI cohorts. Rand.: Randomly dividing instances. K-means: Clustering instances using K-means. Text: using GPT-4 TO GENERATE AS MANY DESCRIPTIONS AS POSSIBLE, i.e. , 15 descriptions and grouping instances into 15 groups following [ 28 ] . k-NN (dist): k-NN grouping based on Euclidean distance. k-NN (cos): k-NN grouping based on cosine similarity. Dic.: Dictionary-guided grouping. The best result for each cohort is highlighted .
Fig. 5: Pathological group distribution and corresponding pathological patterns on a WSI of MSI cancer.
Fig. 6: Performance changes by varying the (a) dictionary size ( K ), (b) number of groups ( G ) and (c) number of dictionary updates ( L ) on the TCGA-CRC-MSI and TCGA-STAD-MSI cohort.
Fig. 7: t-SNE visualization of bag representations in different methods on the testing set of TCGA-CRC-MSI cohort. Our dictionary-based method constructs more discriminative representations of WSIs.
Fig. 8: (a) AUROC and (b) accuracy improvement on the TCGA-CRC-MSI cohort after employing dictionary-based hierarchical pathology mining to four MIL baselines, i.e. , ABMIL [ 11 ] , DSMIL [ 12 ] , CLAM [ 13 ] and TransMIL [ 14 ] . Performance improvement is marked red .
Fig. 9: Performance changes by varying the (a) hard threshold ( α ), (b) threshold for the number of hard instances ( β ) and (c) weight in loss function ( λ ) on the TCGA-CRC-MSI and the TCGA-STAD-MSI cohort.
Fig. 10: Distributions of ∣2p^i−1∣ on the (a) TCGA-CRC-MSI and (b) TCGA-STAD-MSI cohort.
Fig. 11: Heatmap visualizations of D 2 Bio w/o or w/ the hard-instance-assisted classifier debiasing module.
Fig. 12: (a) AUROC and (b) accuracy improvement on the TCGA-CRC-MSI cohort by employing hard-instance-assisted debiasing module to three MIL baselines: ABMIL [ 11 ] , CLAM [ 13 ] and TransMIL [ 14 ] . Performance improvement is marked red .
Extractor
TCGA-CRC-MSI
TCGA-STAD-MSI
DSMIL [ 12 ]
WiKG [ 25 ]
D 2 Bio (ours)
DSMIL [ 12 ]
WiKG [ 25 ]
D 2 Bio (ours)
AUROC
Acc.
AUROC
Acc.
AUROC
Acc.
AUROC
Acc.
AUROC
Acc.
AUROC
Acc.
CtransPath [ 50 ]
88.19
87.50
90.22
90.00
96.70
92.50
77.14
63.64
79.84
70.71
81.73
77.78
Virchow2 [ 59 ]
91.49
91.67
93.10
90.83
97.52
93.33
84.22
78.79
82.97
77.78
86.65
83.84
UNI [ 51 ]
90.54
89.17
90.92
90.83
94.35
94.17
65.46
72.73
82.49
73.74
86.32
74.75
UNI v2 [ 51 ]
89.21
90.83
95.49
94.10
98.48
96.67
84.05
74.75
84.22
72.73
86.16
82.83
TABLE IV: Performance of DSMIL [ 12 ] , WiKG [ 25 ] and our D 2 Bio using different pre-trained feature extractors on TCGA-CRC-MSI cohort and TCGA-SATD-MSI chohort. CtransPath [ 50 ] : Trained on over 32,000 WSIs of TCGA and PAIP datasets. Virchow2 [ 59 ] : Trained on 3.1 million WSIs from globally diverse institutions. UNI [ 51 ] : Trained on approximately 100,000 WSIs of the Mass-100k dataset. UNI v2 [ 51 ] : The extended version of UNI [ 51 ] , trained on approximately 350,000 WSIs. Best results are in bold and second-best results are underlined .
Fig. 13: KM plot of OS and PFS according to three methods, i.e. , our D 2 Bio, CLAM [ 13 ] and WiKG [ 25 ] , predicted patient-level “High MSI & Low SH-MSI” vs other subtypes. “SH-MSI” refers to spatial heterogeneity of MSI.
Method
Time per Epoch
CLAM-SB [ 13 ]
∼ 9 s
TransMIL [ 14 ]
∼ 10 s
ABMIL [ 11 ]
∼ 27 s
D 2 Bio (Ours)
∼ 60 s
TABLE V: Average training time per epoch on TCGA-CRC-MSI.
Molecular biomarker testing in pathology is often costly and tissue-consuming, limiting scalable clinical deployment. Artificial intelligence applied to hematoxylin and eosin (HE)-stained histology could enable rapid biomarker screening, but clinical translation requires models that are both accurate and interpretable. Here we introduce Hireca, a biomarker-focused pathology foundation model pretrained on more than 80,000 whole-slide images spanning 38 organ types from three medical centers, together with CytoMap, an interpretability module that localizes cellular-scale evidence underlying predictions. Across 10 biomarker tasks encompassing morphological, molecular, genetic, and spatial-transcriptomic-proxy readouts, Hireca ranked first in five tasks and outperformed comparable models overall. In evaluation by eight pathologists from two countries, CytoMap was consistently preferred over alternative visualization approaches and revealed error patterns in difficult cases. These results position Hireca and CytoMap as a transparent framework for clinically reviewable biomarker assessment directly from routine HE histology.
Jingsong Liu, Han Li, Zhengyang Xu +19
Institute of Pathology, Technical University of Munich, Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · Munich Data Science Institute (MDSI), Munich, Germany +6
Matched multi-omics can improve WSI-based biomarker and prognosis prediction, but most existing pipelines use omics as a paral lel feature stream or textual context rather than as an explicit retrieval constraint. HERO asks whether observed omics can be a testable mor phology hypothesis: a sparse pathway-to-morphology prior maps DNA methylation and miRNA into a K-dimensional intent vector m (K=16), TF-IDF retrieval over structured 10 captions selects endpoint-relevant regions, and a cosine gate c=cos(m,v) triggers deterministic deficit driven repair when c<τc. This closed-loop design bounds VLM calls, reduces reliance on embedding-based semantic matching, and makes every retrieval and verification step lexically auditable. On TCGA-BRCA (930WSIs, patient-level 5-fold CV), HERO sets new state-of-the-art across ER, PR, HER2, subtype, and risk prediction, outperforming both multimodal fusion and VLM-based baselines.
Xiangyu Li, Ran Su
College of Intelligence and Computing, Tianjin University, Tianjin 300072, China
Joint molecular phenotype prediction is complicated by small joint-positive populations and overlapping histological features across alternative molecular states. Existing computational pathology approaches typically predict biomarkers independently or formulate the joint-positive phenotype as a binary endpoint. Independent prediction does not model interactions between biomarker-specific histological representations, whereas binary joint prediction collapses the double-negative and two single-positive configurations into a single negative class. We propose State-Aware Interaction MIL, a weakly supervised method that preserves biomarker-specific histological representations, models their interaction, and supervises the complete four-state molecular configuration. We evaluate the proposed approach for joint BRAF+/MSI+ prediction in colorectal cancer and EGFR+/TP53+ prediction in lung adenocarcinoma using frozen UNI2-h and CONCH pathology foundation-model representations. With UNI2-h, State-Aware Interaction MIL achieved an average precision of 0.5566 in colorectal cancer (joint-positive prevalence 6.8%) compared with 0.5161 for NaiveMTL, and 0.2784 in lung adenocarcinoma (joint-positive prevalence 8.6%) compared with 0.2525 for IndependentPair. With CONCH, State-Aware achieved an average precision of 0.4410 compared with 0.3932 for DirectJoint in colorectal cancer and 0.1659 compared with 0.1226 for DirectJoint in lung adenocarcinoma. These results indicate that pathology foundation-model representations contain predictive information for rare joint molecular phenotypes and that preserving biomarker-specific representations within a structured molecular-state formulation can improve prediction of these phenotypes from histopathology.
Dasari Naga Raju, Tripti Bameta
Independent Researcher, Andhra Pradesh, India · Tata Memorial Centre-ACTREC, Navi Mumbai, India