GrapHist: Graph Self-Supervised Learning for Histopathology
Organizations: LTS4, EPFL, Lausanne, 1015, Vaud, Switzerland
Abstract
Self-supervised vision models have achieved notable success in digital pathology. However, their domain-agnostic transformer architectures are not originally designed to account for fundamental biological elements of histopathology images, namely cells and their complex interactions. In this work, we hypothesize that a biologically-informed modeling of tissues as cell graphs offers a more efficient representation learning. Thus, we introduce GrapHist, a novel graph-based self-supervised learning framework for histopathology, which learns generalizable and structurally-informed embeddings that enable diverse downstream tasks. GrapHist integrates masked autoencoders and heterophilic graph neural networks that are explicitly designed to capture the heterogeneity of tumor microenvironments. We pre-train GrapHist on a large collection of 11 million cell graphs derived from breast tissues and evaluate its transferability across in- and out-of-domain benchmarks. Our results show that GrapHist achieves competitive performance compared to its vision-based counterparts in slide-, region-, and cell-level tasks, while requiring four times fewer parameters. It also drastically outperforms fully-supervised graph models on cancer subtyping tasks. Finally, we also release five graph-based digital pathology datasets used in our study at https://huggingface.co/ogutsevda/datasets , establishing the first large-scale graph benchmark in this field. Our code is available at https://github.com/ogutsevda/graphist .
Figures & tables
| # Slides | # Patches | Avg # Nodes | Avg # Edge s | # Classes | |
| Slide-level | |||||
| TCGA-BRCA [ 43 ] | 998 | 11 149 500 | 43.94 | 119.16 | 2 |
| Region-level | |||||
| BACH [ 3 ] | 394 | 14 341 | 35.05 | 92.93 | 4 |
| BRACS [ 5 ] | 4493 | 96 153 | 46.96 | 128.09 | 7 |
| BreakHis [ 39 ] | 522 | 709 | 17.26 | 41.43 | 2 |
| Datasets | ||||
| Model | WSI | RoI | ||
| TCGA-BRCA | BACH | BRACS | BreakHis | |
| Supervised graph models | ||||
| ACM-bio | OOM | 38.37 2.16 | 21.55 2.62 | 55.82 10.33 |
| ACM-UNI | OOM | 29.03 9.92 | 19.95 3.27 | 76.27 9.23 |
| Self-supervised vision models | ||||
| Model | C-inde x | p-value |
|---|---|---|
| DINOv2 | 0.63 | 9.80 |
| MAE | 0.72 | 1.54 |
| GrapHist | 0.76 | 1.29 |
| Pre-training | Inference | |||
|---|---|---|---|---|
| Model | # Params (10 6 ) | Time (h) | Peak GPU mem (GB) | Mean ms/patch |
| DINOv2 | 22.01 | 180 | 0.380 | 0.885 |
| MAE | 47.58 | 350 | 0.309 | 1.053 |
| GrapHist | 9.50 | 50 | 0.142 | 0.221 |
| Model | PanNuke 20 | PanNuke 40 | NuCLS | |||
|---|---|---|---|---|---|---|
| Breast | PanCancer | Breast | PanCancer | main | super | |
| Supervised graph models | ||||||
| ACM-bio | 56.61 0.31 | 62.73 0.02 | 57.06 0.33 | 65.80 0.11 | 22.13 1.59 | 37.35 1.30 |
| ACM-UNI | 58.05 0.03 | 67.63 0.66 | 56.81 0.18 | 66.99 0.80 | 21.68 1.03 | 39.46 1.33 |
| Self-supervised vision models | ||||||
| DINOv2 | 54.82 0.48 | 50.49 0.37 | 53.86 0.54 | 49.27 0.42 | 21.42 1.49 | 41.17 2.26 |
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| Nuclei-Color | Nuclei-Morph | Nuclei-Texture |
|---|---|---|
| min intensity R | probability | mean ASM |
| min intensity G | orientation | mean contrast |
| min intensity B | axis major length | mean correlation |
| max intensity R | axis minor length | mean dissimilarity |
| max intensity G | eccentricity | mean energy |
| max intensity B | area | mean homogeneity |
| Model | Number of Parameters ( ) |
|---|---|
| DINOv2 | 22.01 |
| MAE | 47.58 |
| GrapHist ( ) | 9.50 |
| GrapHist ( ) | 21.12 |
| GrapHist ( ) | 37.34 |
| Stage | Device | Mean sec/patch |
|---|---|---|
| Cell segmentation | GPU | 0.127 |
| Cell feature extraction | CPU | 0.369 |
| Graph construction | CPU | 0.009 |
| Total pre-processing | — | 0.505 |
| Datasets | |||||
| Model | Aggregation | WSI | RoI | ||
| TCGA-BRCA | BACH | BRACS | BreakHis | ||
| Supervised graph models | |||||
| ACM-bio | ABMIL | — | 19.56 5.32 | 11.80 1.76 | 55.82 10.33 |
| add-ABMIL | — | 38.37 2.16 | 21.55 2.62 | 49.30 2.53 | |
| conj-ABMIL | — | 19.63 3.89 | 13.89 4.23 | 53.61 10.41 | |
| Datasets | |||||
| Model | Aggregation | WSI | RoI | ||
| TCGA-BRCA | BACH | BRACS | BreakHis | ||
| Supervised graph models | |||||
| ACM-bio | ABMIL | — | 26.23 4.41 | 18.69 3.05 | 58.50 8.11 |
| add-ABMIL | — | 41.00 1.40 | 25.16 2.80 | 50.06 2.13 | |
| conj-ABMIL | — | 24.68 4.86 | 18.27 3.10 | 56.89 7.31 | |
| Datasets | |||||
| Model | Aggregation | WSI | RoI | ||
| TCGA-BRCA | BACH | BRACS | BreakHis | ||
| Supervised graph models | |||||
| ACM-bio | ABMIL | — | 50.21 5.22 | 57.13 3.99 | 60.76 8.81 |
| add-ABMIL | — | 66.44 2.58 | 63.08 2.48 | 49.64 4.38 | |
| conj-ABMIL | — | 46.73 6.13 | 55.42 3.16 | 61.98 12.31 | |
| Datasets | |||||
| Model | Aggregation | WSI | RoI | ||
| TCGA-BRCA | BACH | BRACS | BreakHis | ||
| Supervised graph models | |||||
| ACM-bio | ABMIL | — | 30.20 5.67 | 19.63 2.77 | 67.43 7.26 |
| add-ABMIL | — | 42.91 1.49 | 23.35 2.35 | 59.13 3.98 | |
| conj-ABMIL | — | 26.45 3.75 | 17.98 2.10 | 66.39 8.75 | |
| Model | PanNuke 20 | PanNuke 40 | NuCLS | |||
|---|---|---|---|---|---|---|
| Breast | PanCancer | Breast | PanCancer | main | super | |
| Supervised graph models | ||||||
| ACM-bio | 56.51 0.41 | 61.61 0.33 | 56.41 0.38 | 65.26 0.59 | 24.68 0.96 | 39.25 1.36 |
| ACM-UNI | 57.38 0.01 | 66.72 0.41 | 56.96 0.19 | 65.64 0.87 | 25.29 0.31 | 40.53 1.01 |
| Self-supervised vision models | ||||||
| DINOv2 | 56.12 0.50 | 60.09 1.15 | 55.17 0.57 | 58.91 1.35 | 22.34 1.76 | 41.20 2.42 |
| Model | PanNuke 20 | PanNuke 40 | NuCLS | |||
|---|---|---|---|---|---|---|
| Breast | PanCancer | Breast | PanCancer | main | super | |
| Supervised graph models | ||||||
| ACM-bio | 81.12 4.53 | 91.07 0.07 | 82.80 8.75 | 91.48 0.23 | 75.35 0.69 | 79.34 0.43 |
| ACM-UNI | 91.39 2.35 | 92.92 0.21 | 89.05 3.28 | 92.38 0.09 | 72.02 0.73 | 76.13 1.16 |
| Self-supervised vision models | ||||||
| DINOv2 | 87.58 2.28 | 84.79 0.45 | 87.36 1.98 | 84.03 0.44 | 63.44 0.52 | 68.09 2.02 |
| Model | PanNuke 20 | PanNuke 40 | NuCLS | |||
|---|---|---|---|---|---|---|
| Breast | PanCancer | Breast | PanCancer | main | super | |
| Supervised graph models | ||||||
| ACM-bio | 61.49 0.23 | 67.80 0.17 | 62.44 0.02 | 70.37 0.06 | 28.29 0.46 | 47.78 0.35 |
| ACM-UNI | 64.11 0.12 | 73.59 0.66 | 62.40 0.06 | 72.76 0.43 | 26.93 0.44 | 45.20 0.31 |
| Self-supervised vision models | ||||||
| DINOv2 | 59.52 0.73 | 55.64 1.35 | 58.69 0.74 | 54.25 1.17 | 24.63 1.31 | 43.97 2.09 |