SRED: Sparse Regional Evidence Distillation Across Magnifications in Whole-Slide Imaging
Organizations: Department of Computer Science and Software Engineering (CSSE), Concordia University, Montreal, Canada · Axe Cancer, Centre de recherche du CHUM, Université de Montréal, Montréal, Canada · Mila - Quebec AI Institute, Montreal, Canada
Abstract
In computational pathology, high-magnification whole-slide image analysis preserves tissue detail but requires encoding many patches, whereas lower magnification reduces computation by covering a higher field of view per patch while sacrificing detail. Existing cross-magnification distillation matches features that often encode detail unreachable by low-magnification input, sometimes requiring slide-level labels that limit scalability. To address those issues, we introduce Sparse Regional Evidence Distillation (SRED), which transfers regional event occurrences that encode reachable high-level interactions between details without slide-level labels. We use a sparse autoencoder (SAE) to learn an event dictionary from 20x teacher features. Logical OR pooling records whether each event is detected in any spatially aligned patch, retaining both localized and shared evidence within the corresponding 5x region. A pretrained 5x encoder is fine-tuned to predict these regional targets without slide-level labels. Visual analysis identifies dictionary components responsive to recognizable histological patterns, supporting target interpretation. Compared with native 5x features, SRED improves average macro-F1 by 1.1 points across seven classification and grading cohorts and average C-index by 1.7 points across five survival cohorts. Runtime matches native 5x, with an average 11.58x speedup over 20x.
Figures & tables
| Native | Cross-magnification KD methods | |||||
| Dataset | Metric | XMAG [ 52 ] | LRMIL [ 50 ] | SRED (ours) | ||
| CAM16 | AUROC | |||||
| CAM17 | Macro-F1 | |||||
| Private-CRC | Macro-F1 | |||||
| BRACS | Macro-F1 | |||||
| PANDA | QW | |||||
| Width | MSE | FVU | Observed (%) | Unobserved (%) |
|---|---|---|---|---|
| Positive activations (%) | |||||
|---|---|---|---|---|---|
| Active limit | FVU | ||||
| Encoder | CAM16 | IMP | TCGA-STAD |
|---|---|---|---|
| AUROC | Macro-F1 | C-index | |
| Native | |||
| Native | |||
| Contrastive fine-tuning | |||
| SRED |
| Design | CAM16 | IMP | TCGA-STAD |
|---|---|---|---|
| AUROC | Macro-F1 | C-index | |
| AND | |||
| OR AND | |||
| Occupancy | |||
| OR |
| Design | CAM16 | IMP | TCGA-STAD |
|---|---|---|---|
| AUROC | Macro-F1 | C-index | |
| No anchor | |||
| Parameters | |||
| Relations | |||
| Squared L2 | |||
| Cosine |
| Native | SRED (ours) | ||
| MIL | |||
| CAM16 (AUROC) | |||
| TransMIL | |||
| nnMIL | |||
| RRT-MIL | |||
| Private-CRC (Macro-F1) | |||
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | Value |
|---|---|
| Dictionary size | 16,384 |
| TopK sparsity | 32 |
| Optimizer | Adam |
| Learning rate | , constant |
| Adam | |
| Adam |
| Setting | Value |
|---|---|
| Student initialization | Pretrained UNI2-h |
| MLP hidden dimension | 1,024 |
| Optimizer | AdamW |
| Encoder learning rate | |
| MLP learning rate | |
| AdamW |
| Dataset | Slides | features |
|---|---|---|
| TCGA | 9,501 | 132,633,661 |
| CPTAC | 6,738 | 35,702,601 |
| EBRAINS | 3,114 | 33,283,668 |
| MUT-HET-RCC | 781 | 8,679,542 |
| OV-Bevacizumab | 286 | 5,608,897 |
| BC-therapy | 166 | 340,677 |
| Partition | Slides | Groups | regions |
|---|---|---|---|
| Training | 15,963 | 10,392 | 11,313,866 |
| Validation | 3,486 | 2,227 | 2,496,874 |
| Evaluation | 3,940 | 2,619 | 2,919,381 |
| Native | Cross-magnification KD methods | |||||
|---|---|---|---|---|---|---|
| Dataset | Metric | XMAG [ 52 ] | LRMIL [ 50 ] | SRED (ours) | ||
| CAM16 | Accuracy | |||||
| AUROC | ||||||
| Macro-F1 | ||||||
| CAM17 | Accuracy | |||||
| AUROC | ||||||
| System | Time (s/slide) | Speedup | Network (TFLOPs/slide) | GPU (GiB) |
|---|---|---|---|---|
| Native | 70.04 | 1.00 | 5,015.38 | 3.81 |
| Native | 6.03 | 11.61 | 330.89 | 3.51 |
| SRED (ours) | 6.05 | 11.58 | 330.89 | 3.51 |
| CAM16 | IMP | TCGA-STAD | |
|---|---|---|---|
| AUROC | Macro-F1 | C-index | |
| (Default) |
| Design | CAM16 | IMP | TCGA-STAD |
|---|---|---|---|
| AUROC | Macro-F1 | C-index | |
| ASL | |||
| BCE |
| Latent | Occurrence (%) | Pathologist’s description and comments |
| Additional tissue structures | ||
| 1385 | 27.6106 | Glandular structures from different organs. |
| 4668 | 46.6330 | Large glandular structures with large lumens. |
| 2874 | 42.9318 | Large glandular structures on an eosinophilic background. |
| 3514 | 35.7496 | Glandular and parenchymal cells. |
| 2070 | 27.2946 | Well-defined clusters of epithelial cells. |
Whole-Slide Image Analysis
Whole-slide image analysis uses AI to interpret gigapixel pathology slides for diagnosis, prognosis, tissue mapping, and report generation.
See all 238 papers