FOCUS: A Subspecialty Foundation Model for Intraoperative Central Nervous System Pathology
Organizations: Department of Neuropathology, Beijing Neurosurgical Institute, Capital Medical University, No. 119 South Fourth Ring West Road, Fengtai District, Beijing,100070, P.R.China. · CAS Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, No. 95 Zhongguancun East Road, Haidian District, Beijing, 100190, P.R. China. · School of Artificial Intelligence, University of Chinese Academy of Sciences, No. 1 Yanqihu East Road, Huairou District, Beijing, 101408, P.R. China. · Department of Pathology, The First Hospital of Hebei Medical University, No. 89 Donggang Road, Shijiazhuang City, Hebei Province, 050031, P.R. China. · Department of Neurosurgery, The First Hospital of Hebei Medical University, No. 89 Donggang Road, Shijiazhuang City, Hebei Province, 050031, P.R. China. · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, No. 1 Yanqihu East Road, Huairou District, Beijing, 101408, P.R. China. · Guangdong Provincial People’s Hospital, No. 106 Zhongshan 2nd Road, Guangzhou, Guangdong Province, 510180, P.R. China. · Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Engineering Medicine, Beihang University, No. 37 Xueyuan Road, Haidian District, Beijing 100191, P.R. China.
Abstract
Intraoperative frozen-section diagnosis of central nervous system (CNS) lesions guides surgical decisions but is challenged by artefacts, diagnostic complexity and time constraints. Most pathology foundation models have limited exposure to CNS frozen tissue. Here we present FOCUS, a subspecialty foundation model pretrained on 27,930 frozen-sections enriched for CNS tissue. FOCUS led 11 internal diagnostic and 12 biomarker-prediction tasks, with mean AUCs of 0.969 and 0.877, respectively, retaining leading performance in temporal and external cohorts. To provide more comprehensive diagnostic information, we further organized ten diagnostic tasks into a WHO-aligned diagnostic hierarchy and separately evaluated FOCUS for lymphoma detection, a critical task for intraoperative surgical decision-making. In reader assessments by six neuropathologists, FOCUS assistance enabled junior readers to approach or exceed unassisted senior readers in hierarchical diagnosis and raised junior readers' lymphoma sensitivity from 81.6% to 94.7%. FOCUS could improve diagnostic accuracy and granularity, inform lymphoma-related resection decisions and provide provisional molecular information. Such capabilities highlight the potential of FOCUS to provide robust, clinically actionable support for intraoperative CNS pathology.
Figures & tables
Appendix figures & tables95 assets
Supplementary material from the paper’s appendix.
Appendix
| Data source | Slides | Percentage (%) |
|---|---|---|
| TCGA | 13,807 | 49.43 |
| Ivy-GAP | 11,230 | 40.21 |
| HEST | 1,228 | 4.40 |
| CGCI | 763 | 2.73 |
| HCMI-CMDC | 439 | 1.57 |
| CPTAC | 414 | 1.48 |
| Organ | Slides | Percentage (%) |
|---|---|---|
| Brain | 13,689 | 49.01 |
| Kidney | 1,725 | 6.18 |
| Bronchus and lungs | 1,672 | 5.99 |
| Breast | 1,525 | 5.46 |
| Bowel | 1,159 | 4.15 |
| Ovary | 1,048 | 3.75 |
| Tasks | Training cohort | Test cohort | Temporal cohort | External cohort |
|---|---|---|---|---|
| Non-neoplastic brain tissue/tumour | 2777 (200/2577) | 694 (54/640) | 309 (29/280) | 246 (9/237) |
| Metastasis/ other CNS tumours | 2577 (112/2465) | 640 (28/612) | 280 (14/266) | 237 (13/224) |
| Neuroepithelial/ non-neuroepithelial tumour | 2291 (1876/415) | 576 (472/104) | 255 (202/53) | 221 (58/163) |
| Glioma or GNT/embryonal tumour/ ependymoma | 1876 (1700/90/86) | 472 (427/25/20) | 202 (176/17/9) | 58 (52/0/6) |
| Glioma/GNT | 1700 (1553/147) | 427 (390/37) | 176 (162/14) | 52 (50/2) |
| Meningioma/ other non-neuroepithelial tumours | 415 (246/169) | 104 (62/42) | 53 (35/18) | 163 (72/91) |
| Data source | Slides | Percentage (%) |
|---|---|---|
| TCGA | 6,336 | 53.89 |
| CPTAC | 5,049 | 42.94 |
| HCMI-CMDC | 165 | 1.40 |
| CGCI | 208 | 1.77 |
| Total | 11758 | 100.00 |
| Task | Samples | Label distribution | External validation |
|---|---|---|---|
| BLCA (Benign vs. malignant) | 469 | 37 / 432 | – |
| BRCA (Histological subtype) | 1417 | 1173 / 244 | 111 (100 / 11) |
| BRCA subset (Histological subtype) | 408 | 331 / 77 | 115 (100 / 11 / 4) |
| BRCA (Benign vs. malignant) | 1976 | 398 / 1578 | – |
| CESC (tumour grading) | 283 | 19 / 139 / 125 | 208 (10 / 102 / 96) |
| COADREAD (Benign vs. malignant) | 1347 | 134 / 1213 | 165 (33 / 132) |
| Hyper-parameter | Value |
|---|---|
| Backbone architecture | ViT-G/14 |
| Levels | 40 |
| Heads | 24 |
| Patch size | 14 |
| FFN layer | SwiGLU |
| Embedding dimension | 1536 |
| Task | Definition |
|---|---|
| T1 | Non-neoplastic brain tissue versus tumour classification |
| T2 | Metastasis versus other CNS tumours classification |
| T3 | Neuroepithelial versus non-neuroepithelial tumour classification |
| T4 | Glioma or glioneuronal tumour versus embryonal tumour versus ependymoma classification |
| T5 | Glioma versus glioneuronal tumour classification |
| T6 | Meningioma versus other non-neuroepithelial tumours classification |
| Task | DSMIL | ABMIL | CLAM | TransMIL |
|---|---|---|---|---|
| T1 | 0.991 (0.984–0.998) | 0.990 (0.982–0.998) | 0.991 (0.983–0.998) | 0.989 (0.981–0.997) |
| T2 | 0.992 (0.986–0.999) | 0.976 (0.954–0.997) | 0.990 (0.981–0.998) | 0.994 (0.989–0.999) |
| T3 | 0.999 (0.998–1.000) | 0.996 (0.991–1.000) | 0.997 (0.993–1.000) | 0.999 (0.998–1.000) |
| T4 | 0.974 (0.929–0.997) | 0.975 (0.941–0.995) | 0.979 (0.948–0.996) | 0.985 (0.960–0.999) |
| T5 | 0.946 (0.915–0.976) | 0.947 (0.918–0.977) | 0.952 (0.923–0.981) | 0.946 (0.914–0.978) |
| T6 | 0.998 (0.994–1.000) | 0.995 (0.984–1.000) | 0.994 (0.982–1.000) | 0.996 (0.989–1.000) |
| Task | DSMIL | ABMIL | CLAM | TransMIL |
|---|---|---|---|---|
| T1 | 0.964 (0.943–0.985) | 0.974 (0.960–0.989) | 0.970 (0.954–0.986) | 0.968 (0.951–0.986) |
| T2 | 0.995 (0.991–1.000) | 0.983 (0.964–1.000) | 0.993 (0.985–1.000) | 0.995 (0.989–1.000) |
| T3 | 0.992 (0.986–0.998) | 0.991 (0.984–0.999) | 0.992 (0.984–0.999) | 0.988 (0.979–0.997) |
| T4 | 0.987 (0.977–0.995) | 0.985 (0.971–0.995) | 0.985 (0.973–0.995) | 0.985 (0.971–0.995) |
| T5 | 0.898 (0.861–0.936) | 0.902 (0.865–0.939) | 0.917 (0.883–0.950) | 0.908 (0.869–0.947) |
| T6 | 0.992 (0.979–1.000) | 0.988 (0.972–1.000) | 0.987 (0.970–1.000) | 0.987 (0.973–1.000) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.964 (0.943–0.985) | 0.947 (0.928–0.963) | 0.963 (0.946–0.976) | 0.759 (0.635–0.878) | 0.971 (0.960–0.980) |
| H-optimus-1 | 0.959 (0.930–0.988) | 0.935 (0.915–0.954) | 0.941 (0.921–0.959) | 0.870 (0.771–0.957) | 0.964 (0.952–0.974) |
| H-optimus-0 | 0.954 (0.923–0.986) | 0.937 (0.916–0.954) | 0.942 (0.922–0.960) | 0.870 (0.771–0.957) | 0.965 (0.953–0.975) |
| GigaPath | 0.958 (0.931–0.984) | 0.935 (0.915–0.952) | 0.944 (0.923–0.961) | 0.833 (0.727–0.929) | 0.964 (0.952–0.974) |
| CRISP | 0.957 (0.929–0.985) | 0.935 (0.915–0.952) | 0.947 (0.927–0.964) | 0.796 (0.678–0.900) | 0.964 (0.953–0.974) |
| Virchow2 | 0.958 (0.927–0.989) | 0.954 (0.937–0.968) | 0.969 (0.954–0.981) | 0.778 (0.653–0.889) | 0.975 (0.965–0.983) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.995 (0.991–1.000) | 0.984 (0.975–0.994) | 0.997 (0.992–1.000) | 0.714 (0.520–0.879) | 0.992 (0.987–0.997) |
| H-optimus-1 | 0.992 (0.985–0.999) | 0.983 (0.972–0.992) | 0.995 (0.989–1.000) | 0.714 (0.538–0.875) | 0.991 (0.985–0.996) |
| H-optimus-0 | 0.989 (0.980–0.998) | 0.981 (0.970–0.991) | 0.995 (0.989–1.000) | 0.679 (0.500–0.850) | 0.990 (0.984–0.995) |
| GigaPath | 0.981 (0.968–0.994) | 0.973 (0.961–0.984) | 0.995 (0.989–1.000) | 0.500 (0.308–0.690) | 0.986 (0.980–0.992) |
| CRISP | 0.986 (0.973–0.998) | 0.980 (0.969–0.989) | 0.998 (0.995–1.000) | 0.571 (0.385–0.750) | 0.989 (0.984–0.994) |
| Virchow2 | 0.980 (0.963–0.997) | 0.983 (0.972–0.992) | 0.998 (0.995–1.000) | 0.643 (0.458–0.815) | 0.991 (0.985–0.996) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.992 (0.986–0.998) | 0.973 (0.959–0.988) | 0.923 (0.867–0.971) | 0.985 (0.972–0.996) | 0.928 (0.888–0.964) |
| H-optimus-1 | 0.989 (0.981–0.997) | 0.973 (0.959–0.986) | 0.933 (0.880–0.979) | 0.983 (0.969–0.993) | 0.928 (0.889–0.963) |
| H-optimus-0 | 0.985 (0.974–0.996) | 0.968 (0.952–0.982) | 0.913 (0.854–0.967) | 0.980 (0.966–0.991) | 0.913 (0.868–0.951) |
| GigaPath | 0.985 (0.976–0.995) | 0.966 (0.950–0.980) | 0.923 (0.867–0.971) | 0.976 (0.960–0.989) | 0.910 (0.864–0.948) |
| CRISP | 0.985 (0.973–0.996) | 0.973 (0.959–0.986) | 0.933 (0.880–0.980) | 0.983 (0.969–0.994) | 0.928 (0.889–0.962) |
| Virchow2 | 0.986 (0.973–0.998) | 0.972 (0.957–0.984) | 0.923 (0.867–0.972) | 0.983 (0.970–0.993) | 0.923 (0.882–0.958) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.987 (0.977–0.995) | 0.974 (0.959–0.986) | 0.972 (0.958–0.985) | 0.779 (0.668–0.887) | 0.971 (0.953–0.985) |
| H-optimus-1 | 0.983 (0.972–0.993) | 0.958 (0.938–0.974) | 0.956 (0.934–0.972) | 0.638 (0.502–0.764) | 0.951 (0.926–0.971) |
| H-optimus-0 | 0.977 (0.958–0.991) | 0.964 (0.946–0.980) | 0.962 (0.943–0.979) | 0.678 (0.550–0.803) | 0.957 (0.933–0.976) |
| GigaPath | 0.977 (0.959–0.991) | 0.966 (0.950–0.980) | 0.964 (0.947–0.979) | 0.718 (0.590–0.835) | 0.961 (0.940–0.979) |
| CRISP | 0.974 (0.945–0.992) | 0.960 (0.942–0.976) | 0.958 (0.939–0.975) | 0.658 (0.529–0.782) | 0.954 (0.930–0.973) |
| Virchow2 | 0.977 (0.955–0.992) | 0.962 (0.944–0.978) | 0.960 (0.941–0.977) | 0.658 (0.535–0.783) | 0.953 (0.929–0.974) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.898 (0.861–0.936) | 0.899 (0.871–0.925) | 0.378 (0.216–0.533) | 0.949 (0.926–0.969) | 0.394 (0.235–0.531) |
| H-optimus-1 | 0.869 (0.822–0.917) | 0.909 (0.881–0.934) | 0.216 (0.094–0.359) | 0.974 (0.957–0.990) | 0.291 (0.130–0.453) |
| H-optimus-0 | 0.855 (0.804–0.906) | 0.902 (0.874–0.927) | 0.162 (0.056–0.286) | 0.972 (0.954–0.987) | 0.222 (0.078–0.370) |
| GigaPath | 0.863 (0.816–0.911) | 0.888 (0.857–0.916) | 0.135 (0.029–0.256) | 0.959 (0.938–0.977) | 0.172 (0.038–0.306) |
| CRISP | 0.887 (0.845–0.928) | 0.892 (0.862–0.918) | 0.514 (0.345–0.677) | 0.928 (0.901–0.951) | 0.452 (0.311–0.575) |
| Virchow2 | 0.880 (0.836–0.924) | 0.892 (0.859–0.920) | 0.459 (0.294–0.621) | 0.933 (0.907–0.957) | 0.425 (0.276–0.556) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.992 (0.979–1.000) | 0.971 (0.933–1.000) | 0.976 (0.921–1.000) | 0.968 (0.919–1.000) | 0.965 (0.919–1.000) |
| H-optimus-1 | 0.982 (0.961–1.000) | 0.952 (0.904–0.990) | 0.929 (0.844–1.000) | 0.968 (0.919–1.000) | 0.940 (0.880–0.987) |
| H-optimus-0 | 0.974 (0.947–1.000) | 0.923 (0.865–0.971) | 0.905 (0.809–0.978) | 0.935 (0.866–0.985) | 0.905 (0.831–0.961) |
| GigaPath | 0.984 (0.967–1.000) | 0.952 (0.904–0.990) | 0.952 (0.879–1.000) | 0.952 (0.889–1.000) | 0.941 (0.878–0.987) |
| CRISP | 0.988 (0.975–1.000) | 0.923 (0.865–0.971) | 0.905 (0.809–0.978) | 0.935 (0.869–0.985) | 0.905 (0.831–0.963) |
| Virchow2 | 0.986 (0.971–1.000) | 0.942 (0.894–0.981) | 0.905 (0.806–0.977) | 0.968 (0.919–1.000) | 0.927 (0.857–0.978) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.977 (0.944–0.997) | 0.925 (0.867–0.972) | 0.833 (0.714–0.929) | 0.938 (0.884–0.979) | 0.830 (0.707–0.929) |
| H-optimus-1 | 0.971 (0.937–0.993) | 0.900 (0.834–0.954) | 0.786 (0.667–0.905) | 0.914 (0.853–0.962) | 0.775 (0.627–0.899) |
| H-optimus-0 | 0.958 (0.917–0.986) | 0.881 (0.815–0.935) | 0.738 (0.595–0.857) | 0.899 (0.833–0.948) | 0.729 (0.573–0.856) |
| GigaPath | 0.968 (0.927–0.994) | 0.921 (0.859–0.969) | 0.810 (0.667–0.929) | 0.939 (0.885–0.980) | 0.803 (0.655–0.923) |
| CRISP | 0.976 (0.944–0.997) | 0.927 (0.869–0.972) | 0.833 (0.714–0.929) | 0.956 (0.917–0.987) | 0.836 (0.723–0.931) |
| Virchow2 | 0.971 (0.930–0.995) | 0.894 (0.822–0.950) | 0.762 (0.619–0.881) | 0.933 (0.889–0.971) | 0.762 (0.625–0.881) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.956 (0.933–0.979) | 0.921 (0.893–0.948) | 0.600 (0.473–0.733) | 0.976 (0.957–0.991) | 0.688 (0.575–0.787) |
| H-optimus-1 | 0.945 (0.916–0.974) | 0.906 (0.874–0.935) | 0.491 (0.360–0.623) | 0.976 (0.959–0.991) | 0.600 (0.474–0.713) |
| H-optimus-0 | 0.939 (0.905–0.972) | 0.914 (0.882–0.942) | 0.527 (0.393–0.661) | 0.979 (0.963–0.994) | 0.637 (0.511–0.741) |
| GigaPath | 0.948 (0.919–0.977) | 0.914 (0.885–0.942) | 0.600 (0.467–0.731) | 0.966 (0.945–0.985) | 0.667 (0.548–0.766) |
| CRISP | 0.939 (0.910–0.968) | 0.908 (0.880–0.937) | 0.655 (0.524–0.778) | 0.951 (0.928–0.973) | 0.673 (0.560–0.768) |
| Virchow2 | 0.935 (0.902–0.967) | 0.908 (0.880–0.935) | 0.618 (0.491–0.750) | 0.957 (0.934–0.978) | 0.660 (0.548–0.759) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.932 (0.912–0.950) | 0.869 (0.843–0.893) | 0.761 (0.716–0.804) | 0.884 (0.860–0.906) | 0.750 (0.699–0.796) |
| H-optimus-1 | 0.911 (0.889–0.932) | 0.856 (0.830–0.881) | 0.734 (0.688–0.780) | 0.876 (0.855–0.897) | 0.716 (0.663–0.765) |
| H-optimus-0 | 0.893 (0.868–0.918) | 0.852 (0.825–0.876) | 0.728 (0.679–0.774) | 0.869 (0.846–0.891) | 0.709 (0.655–0.760) |
| GigaPath | 0.916 (0.894–0.936) | 0.861 (0.833–0.885) | 0.749 (0.700–0.793) | 0.876 (0.851–0.899) | 0.737 (0.683–0.784) |
| CRISP | 0.908 (0.884–0.930) | 0.845 (0.817–0.870) | 0.719 (0.670–0.765) | 0.861 (0.836–0.884) | 0.696 (0.641–0.746) |
| Virchow2 | 0.905 (0.880–0.928) | 0.850 (0.822–0.876) | 0.731 (0.682–0.777) | 0.865 (0.838–0.889) | 0.715 (0.660–0.766) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.988 (0.965–1.000) | 0.930 (0.860–0.982) | 0.923 (0.750–1.000) | 0.932 (0.848–1.000) | 0.857 (0.667–0.970) |
| H-optimus-1 | 0.963 (0.918–1.000) | 0.930 (0.860–0.982) | 0.769 (0.500–1.000) | 0.977 (0.927–1.000) | 0.833 (0.625–0.968) |
| H-optimus-0 | 0.930 (0.862–0.998) | 0.877 (0.789–0.965) | 0.692 (0.400–0.923) | 0.932 (0.850–1.000) | 0.720 (0.454–0.897) |
| GigaPath | 0.965 (0.921–1.000) | 0.930 (0.860–0.982) | 0.846 (0.611–1.000) | 0.955 (0.881–1.000) | 0.846 (0.640–0.970) |
| CRISP | 0.962 (0.914–1.000) | 0.895 (0.807–0.965) | 0.769 (0.500–1.000) | 0.932 (0.848–1.000) | 0.769 (0.533–0.923) |
| Virchow2 | 0.932 (0.857–1.000) | 0.895 (0.807–0.965) | 0.692 (0.400–0.929) | 0.955 (0.886–1.000) | 0.750 (0.500–0.917) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.973 (0.937–1.000) | 0.987 (0.976–0.996) | 0.400 (0.100–0.714) | 1.000 (1.000–1.000) | 0.571 (0.182–0.833) |
| H-optimus-1 | 0.963 (0.913–1.000) | 0.989 (0.978–0.998) | 0.500 (0.167–0.833) | 1.000 (1.000–1.000) | 0.667 (0.286–0.909) |
| H-optimus-0 | 0.929 (0.816–1.000) | 0.989 (0.978–0.998) | 0.500 (0.167–0.833) | 1.000 (1.000–1.000) | 0.667 (0.286–0.909) |
| GigaPath | 0.952 (0.888–1.000) | 0.989 (0.978–0.998) | 0.500 (0.167–0.833) | 1.000 (1.000–1.000) | 0.667 (0.286–0.909) |
| CRISP | 0.956 (0.891–1.000) | 0.993 (0.985–1.000) | 0.700 (0.400–1.000) | 1.000 (1.000–1.000) | 0.824 (0.571–1.000) |
| Virchow2 | 0.924 (0.809–1.000) | 0.985 (0.971–0.993) | 0.500 (0.182–0.833) | 0.995 (0.989–1.000) | 0.588 (0.250–0.833) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.834 (0.779–0.890) | 0.831 (0.791–0.868) | 0.592 (0.474–0.697) | 0.897 (0.860–0.933) | 0.604 (0.500–0.688) |
| H-optimus-1 | 0.819 (0.760–0.878) | 0.831 (0.791–0.868) | 0.539 (0.427–0.653) | 0.912 (0.877–0.943) | 0.582 (0.482–0.675) |
| H-optimus-0 | 0.811 (0.750–0.872) | 0.834 (0.794–0.871) | 0.539 (0.427–0.654) | 0.916 (0.881–0.947) | 0.586 (0.481–0.676) |
| GigaPath | 0.824 (0.770–0.879) | 0.828 (0.785–0.865) | 0.513 (0.395–0.627) | 0.916 (0.880–0.946) | 0.565 (0.453–0.658) |
| CRISP | 0.817 (0.761–0.874) | 0.825 (0.785–0.863) | 0.500 (0.387–0.611) | 0.916 (0.880–0.947) | 0.555 (0.448–0.650) |
| Virchow2 | 0.808 (0.751–0.866) | 0.822 (0.782–0.862) | 0.421 (0.309–0.535) | 0.934 (0.903–0.961) | 0.508 (0.393–0.612) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.967 (0.951–0.984) | 0.911 (0.883–0.940) | 0.720 (0.625–0.813) | 0.970 (0.948–0.989) | 0.792 (0.719–0.859) |
| H-optimus-1 | 0.954 (0.931–0.978) | 0.909 (0.877–0.937) | 0.720 (0.618–0.815) | 0.966 (0.944–0.985) | 0.787 (0.708–0.855) |
| H-optimus-0 | 0.946 (0.921–0.971) | 0.889 (0.854–0.920) | 0.610 (0.500–0.716) | 0.974 (0.954–0.992) | 0.719 (0.628–0.800) |
| GigaPath | 0.951 (0.929–0.974) | 0.897 (0.866–0.929) | 0.683 (0.581–0.784) | 0.963 (0.939–0.982) | 0.757 (0.672–0.827) |
| CRISP | 0.933 (0.905–0.962) | 0.891 (0.860–0.923) | 0.622 (0.518–0.727) | 0.974 (0.954–0.992) | 0.729 (0.642–0.808) |
| Virchow2 | 0.924 (0.891–0.957) | 0.877 (0.843–0.911) | 0.610 (0.506–0.720) | 0.959 (0.933–0.981) | 0.699 (0.611–0.783) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.905 (0.858–0.951) | 0.945 (0.919–0.968) | 0.176 (0.000–0.375) | 0.985 (0.970–0.997) | 0.240 (0.000–0.455) |
| H-optimus-1 | 0.815 (0.722–0.908) | 0.945 (0.919–0.968) | 0.059 (0.000–0.188) | 0.991 (0.979–1.000) | 0.095 (0.000–0.273) |
| H-optimus-0 | 0.816 (0.719–0.913) | 0.942 (0.916–0.965) | 0.059 (0.000–0.188) | 0.988 (0.975–0.997) | 0.091 (0.000–0.267) |
| GigaPath | 0.833 (0.750–0.917) | 0.948 (0.922–0.968) | 0.176 (0.000–0.375) | 0.988 (0.975–0.997) | 0.250 (0.000–0.471) |
| CRISP | 0.791 (0.697–0.884) | 0.957 (0.934–0.977) | 0.294 (0.083–0.533) | 0.991 (0.979–1.000) | 0.400 (0.125–0.621) |
| Virchow2 | 0.753 (0.638–0.867) | 0.945 (0.919–0.968) | 0.176 (0.000–0.375) | 0.985 (0.970–0.997) | 0.240 (0.000–0.457) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.981 (0.962–1.000) | 0.986 (0.971–0.997) | 0.250 (0.000–1.000) | 0.994 (0.986–1.000) | 0.286 (0.000–0.667) |
| H-optimus-1 | 0.974 (0.941–1.000) | 0.986 (0.971–0.994) | 0.000 (0.000–0.000) | 0.997 (0.991–1.000) | 0.000 (0.000–0.000) |
| H-optimus-0 | 0.966 (0.918–1.000) | 0.983 (0.969–0.994) | 0.000 (0.000–0.000) | 0.994 (0.986–1.000) | 0.000 (0.000–0.000) |
| GigaPath | 0.965 (0.907–1.000) | 0.989 (0.977–0.997) | 0.000 (0.000–0.000) | 1.000 (1.000–1.000) | 0.000 (0.000–0.000) |
| CRISP | 0.981 (0.965–0.998) | 0.989 (0.977–0.997) | 0.000 (0.000–0.000) | 1.000 (1.000–1.000) | 0.000 (0.000–0.000) |
| Virchow2 | 0.978 (0.953–1.000) | 0.989 (0.977–0.997) | 0.000 (0.000–0.000) | 1.000 (1.000–1.000) | 0.000 (0.000–0.000) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.836 (0.722–0.950) | 0.946 (0.920–0.968) | 0.133 (0.000–0.333) | 0.982 (0.967–0.994) | 0.174 (0.000–0.385) |
| H-optimus-1 | 0.808 (0.693–0.922) | 0.954 (0.931–0.974) | 0.067 (0.000–0.222) | 0.994 (0.985–1.000) | 0.111 (0.000–0.333) |
| H-optimus-0 | 0.786 (0.656–0.916) | 0.957 (0.934–0.977) | 0.000 (0.000–0.000) | 1.000 (1.000–1.000) | 0.000 (0.000–0.000) |
| GigaPath | 0.825 (0.706–0.943) | 0.960 (0.937–0.980) | 0.133 (0.000–0.333) | 0.997 (0.991–1.000) | 0.222 (0.000–0.467) |
| CRISP | 0.800 (0.672–0.928) | 0.954 (0.931–0.974) | 0.000 (0.000–0.000) | 0.997 (0.991–1.000) | 0.000 (0.000–0.000) |
| Virchow2 | 0.790 (0.665–0.916) | 0.957 (0.934–0.977) | 0.000 (0.000–0.000) | 1.000 (1.000–1.000) | 0.000 (0.000–0.000) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.823 (0.774–0.872) | 0.814 (0.774–0.854) | 0.517 (0.417–0.624) | 0.916 (0.881–0.950) | 0.586 (0.486–0.675) |
| H-optimus-1 | 0.817 (0.767–0.867) | 0.797 (0.754–0.840) | 0.427 (0.325–0.528) | 0.923 (0.889–0.955) | 0.517 (0.415–0.610) |
| H-optimus-0 | 0.808 (0.757–0.859) | 0.794 (0.751–0.837) | 0.438 (0.337–0.544) | 0.916 (0.880–0.947) | 0.520 (0.422–0.614) |
| GigaPath | 0.802 (0.751–0.853) | 0.797 (0.754–0.840) | 0.427 (0.326–0.533) | 0.923 (0.888–0.955) | 0.517 (0.415–0.610) |
| CRISP | 0.802 (0.750–0.853) | 0.780 (0.737–0.823) | 0.236 (0.152–0.326) | 0.966 (0.942–0.985) | 0.353 (0.245–0.457) |
| Virchow2 | 0.801 (0.749–0.854) | 0.777 (0.734–0.823) | 0.247 (0.159–0.337) | 0.958 (0.932–0.981) | 0.361 (0.244–0.466) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.866 (0.826–0.907) | 0.820 (0.777–0.857) | 0.737 (0.659–0.809) | 0.871 (0.825–0.912) | 0.757 (0.700–0.809) |
| H-optimus-1 | 0.844 (0.800–0.888) | 0.803 (0.760–0.846) | 0.654 (0.570–0.735) | 0.894 (0.852–0.932) | 0.716 (0.648–0.777) |
| H-optimus-0 | 0.837 (0.792–0.881) | 0.803 (0.760–0.846) | 0.654 (0.576–0.733) | 0.894 (0.853–0.932) | 0.716 (0.651–0.775) |
| GigaPath | 0.843 (0.800–0.886) | 0.800 (0.757–0.840) | 0.662 (0.576–0.742) | 0.885 (0.841–0.925) | 0.715 (0.645–0.774) |
| CRISP | 0.834 (0.790–0.878) | 0.791 (0.749–0.831) | 0.752 (0.675–0.820) | 0.816 (0.763–0.865) | 0.733 (0.672–0.786) |
| Virchow2 | 0.829 (0.783–0.875) | 0.786 (0.743–0.829) | 0.602 (0.520–0.682) | 0.899 (0.857–0.936) | 0.681 (0.611–0.745) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.955 (0.935–0.975) | 0.885 (0.848–0.919) | 0.917 (0.872–0.957) | 0.855 (0.800–0.907) | 0.885 (0.844–0.920) |
| H-optimus-1 | 0.922 (0.892–0.952) | 0.863 (0.823–0.898) | 0.891 (0.839–0.936) | 0.837 (0.778–0.893) | 0.863 (0.818–0.900) |
| H-optimus-0 | 0.920 (0.889–0.950) | 0.848 (0.807–0.888) | 0.897 (0.849–0.940) | 0.801 (0.737–0.861) | 0.851 (0.807–0.891) |
| GigaPath | 0.939 (0.915–0.963) | 0.860 (0.820–0.898) | 0.904 (0.856–0.946) | 0.819 (0.757–0.877) | 0.862 (0.818–0.900) |
| CRISP | 0.927 (0.900–0.954) | 0.848 (0.804–0.885) | 0.865 (0.810–0.918) | 0.831 (0.772–0.887) | 0.846 (0.796–0.888) |
| Virchow2 | 0.927 (0.900–0.955) | 0.851 (0.811–0.891) | 0.865 (0.810–0.918) | 0.837 (0.780–0.894) | 0.849 (0.801–0.891) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.736 (0.677–0.794) | 0.680 (0.625–0.735) | 0.779 (0.714–0.843) | 0.554 (0.472–0.644) | 0.732 (0.673–0.782) |
| H-optimus-1 | 0.734 (0.676–0.792) | 0.655 (0.600–0.709) | 0.721 (0.653–0.791) | 0.570 (0.483–0.658) | 0.700 (0.642–0.756) |
| H-optimus-0 | 0.705 (0.644–0.767) | 0.655 (0.600–0.709) | 0.701 (0.631–0.773) | 0.595 (0.513–0.680) | 0.695 (0.632–0.748) |
| GigaPath | 0.727 (0.668–0.787) | 0.658 (0.604–0.713) | 0.721 (0.652–0.789) | 0.579 (0.492–0.667) | 0.703 (0.642–0.757) |
| CRISP | 0.707 (0.646–0.768) | 0.644 (0.589–0.698) | 0.773 (0.708–0.837) | 0.479 (0.392–0.565) | 0.708 (0.650–0.760) |
| Virchow2 | 0.718 (0.658–0.778) | 0.673 (0.618–0.727) | 0.786 (0.721–0.848) | 0.529 (0.440–0.618) | 0.729 (0.674–0.781) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.884 (0.846–0.923) | 0.838 (0.798–0.879) | 0.273 (0.157–0.395) | 0.955 (0.927–0.978) | 0.366 (0.229–0.494) |
| H-optimus-1 | 0.836 (0.791–0.881) | 0.807 (0.763–0.850) | 0.127 (0.043–0.217) | 0.947 (0.919–0.973) | 0.184 (0.063–0.299) |
| H-optimus-0 | 0.830 (0.782–0.878) | 0.810 (0.766–0.850) | 0.109 (0.035–0.196) | 0.955 (0.928–0.977) | 0.164 (0.056–0.277) |
| GigaPath | 0.837 (0.790–0.884) | 0.816 (0.773–0.860) | 0.091 (0.019–0.173) | 0.966 (0.943–0.985) | 0.145 (0.032–0.263) |
| CRISP | 0.831 (0.779–0.884) | 0.822 (0.782–0.863) | 0.091 (0.019–0.173) | 0.974 (0.953–0.992) | 0.149 (0.032–0.267) |
| Virchow2 | 0.825 (0.770–0.880) | 0.826 (0.782–0.866) | 0.109 (0.035–0.200) | 0.974 (0.954–0.989) | 0.176 (0.060–0.301) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.855 (0.790–0.920) | 0.888 (0.854–0.919) | 0.574 (0.425–0.712) | 0.942 (0.913–0.967) | 0.600 (0.472–0.706) |
| H-optimus-1 | 0.828 (0.758–0.899) | 0.866 (0.826–0.900) | 0.489 (0.341–0.643) | 0.931 (0.899–0.957) | 0.517 (0.381–0.634) |
| H-optimus-0 | 0.813 (0.738–0.889) | 0.866 (0.829–0.900) | 0.447 (0.302–0.591) | 0.938 (0.910–0.964) | 0.494 (0.354–0.614) |
| GigaPath | 0.843 (0.773–0.913) | 0.872 (0.835–0.907) | 0.468 (0.333–0.618) | 0.942 (0.912–0.968) | 0.518 (0.382–0.636) |
| CRISP | 0.797 (0.719–0.874) | 0.872 (0.835–0.907) | 0.319 (0.189–0.455) | 0.967 (0.944–0.986) | 0.423 (0.271–0.558) |
| Virchow2 | 0.797 (0.718–0.876) | 0.875 (0.838–0.910) | 0.277 (0.151–0.420) | 0.978 (0.960–0.993) | 0.394 (0.233–0.537) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.877 (0.840–0.915) | 0.823 (0.780–0.866) | 0.805 (0.741–0.867) | 0.840 (0.782–0.897) | 0.818 (0.769–0.863) |
| H-optimus-1 | 0.856 (0.816–0.897) | 0.764 (0.720–0.807) | 0.818 (0.756–0.875) | 0.712 (0.640–0.778) | 0.774 (0.724–0.821) |
| H-optimus-0 | 0.828 (0.784–0.873) | 0.752 (0.705–0.798) | 0.811 (0.750–0.871) | 0.693 (0.622–0.765) | 0.763 (0.709–0.811) |
| GigaPath | 0.865 (0.826–0.904) | 0.786 (0.739–0.829) | 0.767 (0.701–0.830) | 0.804 (0.742–0.863) | 0.780 (0.725–0.828) |
| CRISP | 0.852 (0.811–0.892) | 0.755 (0.705–0.801) | 0.774 (0.707–0.838) | 0.736 (0.667–0.804) | 0.757 (0.703–0.807) |
| Virchow2 | 0.829 (0.785–0.873) | 0.742 (0.696–0.789) | 0.799 (0.736–0.860) | 0.687 (0.615–0.762) | 0.754 (0.701–0.802) |
| Task | FOCUS | H-optimus-1 | H-optimus-0 | GigaPath | CRISP | Virchow2 | Virchow | UNI | Phikon-v2 |
|---|---|---|---|---|---|---|---|---|---|
| T1 | 0.954 (0.920–0.987) | 0.950 (0.920–0.980) | 0.949 (0.915–0.982) | 0.946 (0.911–0.980) | 0.942 (0.905–0.979) | 0.953 (0.923–0.983) | 0.941 (0.906–0.976) | 0.945 (0.911–0.978) | 0.935 (0.899–0.971) |
| T2 | 0.961 (0.923–0.998) | 0.924 (0.835–1.000) | 0.916 (0.813–1.000) | 0.917 (0.820–1.000) | 0.908 (0.837–0.978) | 0.895 (0.769–1.000) | 0.947 (0.901–0.992) | 0.907 (0.779–1.000) | 0.890 (0.795–0.985) |
| T3 | 0.967 (0.928–1.000) | 0.952 (0.904–0.999) | 0.942 (0.888–0.997) | 0.957 (0.915–1.000) | 0.959 (0.919–0.999) | 0.938 (0.883–0.992) | 0.922 (0.863–0.982) | 0.954 (0.909–0.999) | 0.967 (0.927–1.000) |
| T4 | 0.953 (0.898–0.988) | 0.943 (0.868–0.987) | 0.950 (0.894–0.988) | 0.950 (0.899–0.984) | 0.952 (0.893–0.989) | 0.944 (0.895–0.980) | 0.952 (0.887–0.989) | 0.943 (0.875–0.984) | 0.949 (0.904–0.980) |
| T5 | 0.956 (0.916–0.996) | 0.952 (0.907–0.998) | 0.952 (0.916–0.988) | 0.957 (0.919–0.995) | 0.955 (0.906–1.000) | 0.951 (0.906–0.995) | 0.948 (0.909–0.987) | 0.951 (0.912–0.990) | 0.909 (0.852–0.966) |
| T6 | 0.948 (0.885–1.000) | 0.951 (0.892–1.000) | 0.941 (0.875–1.000) | 0.952 (0.900–1.000) | 0.949 (0.888–1.000) | 0.937 (0.864–1.000) | 0.921 (0.843–0.998) | 0.938 (0.871–1.000) | 0.897 (0.813–0.981) |
| Task | FOCUS | H-optimus-1 | H-optimus-0 | GigaPath | CRISP | Virchow2 | Virchow | UNI | Phikon-v2 |
|---|---|---|---|---|---|---|---|---|---|
| T1 | 0.969 (0.932–1.000) | 0.967 (0.937–0.997) | 0.964 (0.936–0.993) | 0.941 (0.891–0.991) | 0.954 (0.910–0.998) | 0.956 (0.915–0.998) | 0.950 (0.901–0.999) | 0.943 (0.889–0.997) | 0.895 (0.786–1.000) |
| T2 | 0.942 (0.895–0.988) | 0.869 (0.760–0.978) | 0.858 (0.739–0.977) | 0.836 (0.750–0.921) | 0.939 (0.900–0.978) | 0.842 (0.725–0.958) | 0.805 (0.697–0.913) | 0.922 (0.874–0.969) | 0.814 (0.675–0.954) |
| T3 | 0.969 (0.950–0.989) | 0.965 (0.944–0.987) | 0.961 (0.939–0.983) | 0.904 (0.866–0.942) | 0.935 (0.903–0.967) | 0.971 (0.952–0.990) | 0.849 (0.799–0.899) | 0.921 (0.886–0.956) | 0.833 (0.778–0.888) |
| T4 | 0.929 (0.842–0.991) | 0.913 (0.790–1.000) | 0.904 (0.818–0.972) | 0.567 (0.236–0.921) | 0.718 (0.495–0.913) | 0.603 (0.375–0.840) | 0.878 (0.775–0.965) | 0.644 (0.457–0.838) | 0.638 (0.348–0.895) |
| T5 | 0.930 (0.784–1.000) | 0.930 (0.844–1.000) | 0.870 (0.764–0.976) | 0.840 (0.593–1.000) | 0.910 (0.726–1.000) | 0.920 (0.789–1.000) | 0.930 (0.844–1.000) | 0.910 (0.817–1.000) | 0.900 (0.812–0.988) |
| T6 | 0.867 (0.812–0.922) | 0.786 (0.715–0.856) | 0.819 (0.753–0.885) | 0.876 (0.824–0.929) | 0.955 (0.928–0.983) | 0.754 (0.681–0.827) | 0.785 (0.716–0.853) | 0.822 (0.759–0.884) | 0.670 (0.587–0.753) |
| Task | FOCUS | H-optimus-1 | H-optimus-0 | GigaPath | CRISP | Virchow2 | Virchow | UNI | Phikon-v2 |
|---|---|---|---|---|---|---|---|---|---|
| T12 | 0.841 (0.770–0.913) | 0.784 (0.710–0.857) | 0.765 (0.684–0.845) | 0.763 (0.673–0.852) | 0.811 (0.732–0.891) | 0.792 (0.709–0.876) | 0.739 (0.661–0.816) | 0.825 (0.749–0.901) | 0.749 (0.669–0.829) |
| T13 | 0.908 (0.858–0.959) | 0.836 (0.762–0.909) | 0.809 (0.734–0.885) | 0.884 (0.824–0.944) | 0.847 (0.780–0.914) | 0.844 (0.775–0.913) | 0.829 (0.757–0.900) | 0.865 (0.802–0.928) | 0.808 (0.733–0.883) |
| T14 | 0.847 (0.710–0.985) | 0.722 (0.546–0.898) | 0.716 (0.517–0.915) | 0.808 (0.675–0.941) | 0.801 (0.655–0.947) | 0.800 (0.683–0.918) | 0.762 (0.564–0.960) | 0.764 (0.600–0.928) | 0.684 (0.520–0.848) |
| T15 | 0.923 (0.798–1.000) | 0.933 (0.838–1.000) | 0.961 (0.911–1.000) | 0.946 (0.875–1.000) | 0.954 (0.879–1.000) | 0.923 (0.817–1.000) | 0.928 (0.794–1.000) | 0.951 (0.881–1.000) | 0.936 (0.880–0.991) |
| T16 | 0.886 (0.823–0.948) | 0.836 (0.755–0.916) | 0.819 (0.689–0.949) | 0.850 (0.754–0.946) | 0.847 (0.725–0.969) | 0.803 (0.675–0.930) | 0.785 (0.690–0.879) | 0.803 (0.692–0.913) | 0.763 (0.638–0.889) |
| T17 | 0.872 (0.811–0.933) | 0.833 (0.768–0.899) | 0.826 (0.755–0.897) | 0.830 (0.759–0.901) | 0.836 (0.769–0.903) | 0.801 (0.725–0.877) | 0.827 (0.759–0.895) | 0.789 (0.713–0.865) | 0.780 (0.704–0.855) |
| Task | FOCUS | H-optimus-1 | H-optimus-0 | GigaPath | CRISP | Virchow2 | Virchow | UNI | Phikon-v2 |
|---|---|---|---|---|---|---|---|---|---|
| T12 | 1.000 (1.000–1.000) | 0.917 (0.773–1.000) | 0.979 (0.921–1.000) | 0.583 (0.000–1.000) | 0.771 (0.467–1.000) | 0.896 (0.743–1.000) | 0.938 (0.815–1.000) | 0.854 (0.556–1.000) | 0.979 (0.921–1.000) |
| T13 | 1.000 (1.000–1.000) | 0.926 (0.798–1.000) | 0.926 (0.803–1.000) | 0.907 (0.775–1.000) | 0.704 (0.219–1.000) | 0.722 (0.479–0.965) | 1.000 (1.000–1.000) | 0.611 (0.319–0.904) | 0.593 (0.195–0.991) |
| T14 | 0.316 (0.154–0.540) | 0.105 (0.029–0.314) | 0.158 (0.055–0.376) | 0.000 (0.000–0.168) | 0.053 (0.009–0.246) | 0.000 (0.000–0.168) | 0.053 (0.009–0.246) | 0.105 (0.029–0.314) | 1.000 (0.832–1.000) |
| T15 | 1.000 (0.832–1.000) | 1.000 (0.832–1.000) | 1.000 (0.832–1.000) | 1.000 (0.832–1.000) | 1.000 (0.832–1.000) | 1.000 (0.832–1.000) | 1.000 (0.832–1.000) | 1.000 (0.832–1.000) | 1.000 (0.832–1.000) |
| T16 | 1.000 (1.000–1.000) | 0.950 (0.842–1.000) | 0.850 (0.684–1.000) | 0.700 (0.500–0.900) | 1.000 (1.000–1.000) | 0.950 (0.842–1.000) | 0.600 (0.368–0.800) | 0.600 (0.400–0.800) | 0.500 (0.299–0.737) |
| T17 | 1.000 (1.000–1.000) | 0.958 (0.867–1.000) | 0.979 (0.921–1.000) | 0.917 (0.773–1.000) | 0.979 (0.921–1.000) | 0.896 (0.677–1.000) | 0.896 (0.677–1.000) | 0.875 (0.617–1.000) | 0.917 (0.738–1.000) |
| Metric | FOCUS | H-optimus-1 | H-optimus-0 | GigaPath | CRISP | Virchow2 | Virchow | UNI | Phikon-v2 |
|---|---|---|---|---|---|---|---|---|---|
| AUC | 0.967 (0.936–0.998) | 0.943 (0.900–0.986) | 0.899 (0.832–0.965) | 0.924 (0.869–0.978) | 0.935 (0.887–0.982) | 0.907 (0.846–0.967) | 0.904 (0.832–0.975) | 0.956 (0.920–0.992) | 0.778 (0.689–0.868) |
| Accuracy | 0.902 (0.829–0.946) | 0.892 (0.817–0.939) | 0.873 (0.794–0.924) | 0.833 (0.749–0.893) | 0.892 (0.817–0.939) | 0.833 (0.749–0.893) | 0.814 (0.727–0.877) | 0.843 (0.760–0.901) | 0.794 (0.706–0.861) |
| F1 score | 0.940 (0.897–0.972) | 0.930 (0.884–0.968) | 0.923 (0.876–0.961) | 0.904 (0.854–0.946) | 0.933 (0.889–0.970) | 0.902 (0.848–0.944) | 0.869 (0.806–0.921) | 0.908 (0.855–0.949) | 0.885 (0.834–0.932) |
| Model | T1 internal | T1 temporal | T1 external | T11 pooled |
|---|---|---|---|---|
| FOCUS | 0.880 (0.800–0.980) | 0.862 (0.448–0.966) | 0.778 (0.556–1.000) | 0.813 (0.689–0.957) |
| H-Optimus-1 | 0.833 (0.741–0.944) | 0.759 (0.552–0.897) | 0.778 (0.444–1.000) | 0.680 (0.571–0.861) |
| H-Optimus-0 | 0.833 (0.741–0.944) | 0.759 (0.586–0.931) | 0.667 (0.333–1.000) | 0.688 (0.599–0.860) |
| GigaPath | 0.815 (0.704–0.907) | 0.724 (0.517–0.897) | 0.667 (0.333–1.000) | 0.803 (0.538–0.942) |
| CRISP | 0.796 (0.667–0.907) | 0.621 (0.448–0.862) | 0.778 (0.444–1.000) | 0.724 (0.536–0.930) |
| Virchow2 | 0.833 (0.722–0.926) | 0.724 (0.517–0.931) | 0.667 (0.333–1.000) | 0.786 (0.638–0.917) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.933 (0.911–0.951) | 0.931 (0.912–0.947) | 0.905 (0.884–0.921) | 0.865 (0.845–0.882) |
| H-optimus-1 | 0.922 (0.901–0.942) | 0.923 (0.902–0.940) | 0.891 (0.871–0.909) | 0.848 (0.828–0.866) |
| H-optimus-0 | 0.922 (0.898–0.942) | 0.921 (0.900–0.940) | 0.887 (0.867–0.907) | 0.844 (0.823–0.864) |
| GigaPath | 0.911 (0.887–0.933) | 0.913 (0.890–0.930) | 0.881 (0.859–0.899) | 0.842 (0.820–0.861) |
| CRISP | 0.899 (0.878–0.924) | 0.907 (0.887–0.927) | 0.871 (0.850–0.891) | 0.830 (0.810–0.850) |
| Virchow2 | 0.937 (0.916–0.956) | 0.934 (0.915–0.950) | 0.901 (0.881–0.918) | 0.858 (0.839–0.876) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.933 (0.911–0.951) | 0.931 (0.911–0.947) | 0.890 (0.869–0.908) | 0.825 (0.801–0.847) |
| H-optimus-1 | 0.922 (0.901–0.942) | 0.923 (0.903–0.940) | 0.873 (0.852–0.893) | 0.804 (0.781–0.826) |
| H-optimus-0 | 0.922 (0.898–0.942) | 0.921 (0.900–0.939) | 0.869 (0.846–0.891) | 0.800 (0.776–0.824) |
| GigaPath | 0.911 (0.887–0.933) | 0.913 (0.891–0.931) | 0.864 (0.840–0.884) | 0.803 (0.778–0.823) |
| CRISP | 0.899 (0.878–0.924) | 0.909 (0.890–0.929) | 0.853 (0.830–0.875) | 0.788 (0.765–0.810) |
| Virchow2 | 0.937 (0.916–0.956) | 0.932 (0.913–0.949) | 0.882 (0.860–0.901) | 0.814 (0.791–0.836) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.933 (0.911–0.951) | 0.905 (0.878–0.927) | 0.830 (0.800–0.856) | 0.702 (0.667–0.736) |
| H-optimus-1 | 0.922 (0.901–0.942) | 0.895 (0.867–0.917) | 0.804 (0.772–0.832) | 0.668 (0.632–0.705) |
| H-optimus-0 | 0.922 (0.898–0.942) | 0.891 (0.864–0.916) | 0.798 (0.765–0.830) | 0.659 (0.619–0.697) |
| GigaPath | 0.911 (0.887–0.933) | 0.881 (0.852–0.905) | 0.789 (0.757–0.818) | 0.665 (0.627–0.699) |
| CRISP | 0.899 (0.878–0.924) | 0.870 (0.844–0.898) | 0.757 (0.722–0.789) | 0.610 (0.570–0.647) |
| Virchow2 | 0.937 (0.916–0.956) | 0.907 (0.881–0.930) | 0.817 (0.784–0.846) | 0.679 (0.642–0.717) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.926 (0.893–0.955) | 0.904 (0.874–0.930) | 0.863 (0.831–0.893) | 0.816 (0.785–0.845) |
| H-optimus-1 | 0.893 (0.858–0.926) | 0.864 (0.832–0.894) | 0.820 (0.785–0.853) | 0.764 (0.731–0.797) |
| H-optimus-0 | 0.909 (0.877–0.942) | 0.888 (0.858–0.918) | 0.850 (0.817–0.882) | 0.787 (0.754–0.818) |
| GigaPath | 0.867 (0.829–0.903) | 0.853 (0.818–0.885) | 0.814 (0.779–0.847) | 0.757 (0.721–0.790) |
| CRISP | 0.884 (0.848–0.916) | 0.880 (0.849–0.909) | 0.842 (0.811–0.871) | 0.787 (0.759–0.816) |
| Virchow2 | 0.906 (0.874–0.935) | 0.862 (0.832–0.893) | 0.795 (0.761–0.826) | 0.754 (0.719–0.786) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.926 (0.893–0.955) | 0.896 (0.864–0.924) | 0.836 (0.799–0.870) | 0.761 (0.724–0.796) |
| H-optimus-1 | 0.893 (0.858–0.926) | 0.853 (0.820–0.886) | 0.790 (0.751–0.828) | 0.699 (0.662–0.738) |
| H-optimus-0 | 0.909 (0.877–0.942) | 0.881 (0.849–0.912) | 0.825 (0.786–0.861) | 0.721 (0.682–0.758) |
| GigaPath | 0.867 (0.829–0.903) | 0.848 (0.812–0.882) | 0.789 (0.749–0.827) | 0.694 (0.653–0.731) |
| CRISP | 0.884 (0.848–0.916) | 0.879 (0.847–0.909) | 0.822 (0.787–0.855) | 0.731 (0.696–0.766) |
| Virchow2 | 0.906 (0.874–0.935) | 0.846 (0.813–0.879) | 0.749 (0.707–0.787) | 0.688 (0.647–0.728) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.926 (0.893–0.955) | 0.854 (0.809–0.893) | 0.738 (0.689–0.786) | 0.576 (0.521–0.631) |
| H-optimus-1 | 0.893 (0.858–0.926) | 0.799 (0.754–0.845) | 0.683 (0.631–0.731) | 0.498 (0.443–0.553) |
| H-optimus-0 | 0.909 (0.877–0.942) | 0.835 (0.790–0.877) | 0.725 (0.670–0.777) | 0.518 (0.460–0.576) |
| GigaPath | 0.867 (0.829–0.903) | 0.793 (0.748–0.838) | 0.676 (0.625–0.728) | 0.492 (0.434–0.547) |
| CRISP | 0.884 (0.848–0.916) | 0.825 (0.780–0.867) | 0.696 (0.647–0.748) | 0.485 (0.430–0.541) |
| Virchow2 | 0.906 (0.874–0.935) | 0.790 (0.744–0.832) | 0.625 (0.570–0.676) | 0.518 (0.463–0.573) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.911 (0.874–0.943) | 0.895 (0.862–0.926) | 0.825 (0.791–0.857) | 0.772 (0.737–0.805) |
| H-optimus-1 | 0.915 (0.878–0.947) | 0.826 (0.791–0.859) | 0.706 (0.669–0.741) | 0.680 (0.640–0.717) |
| H-optimus-0 | 0.911 (0.874–0.943) | 0.877 (0.847–0.913) | 0.781 (0.747–0.813) | 0.729 (0.696–0.763) |
| GigaPath | 0.911 (0.874–0.943) | 0.855 (0.821–0.887) | 0.733 (0.696–0.766) | 0.687 (0.651–0.719) |
| CRISP | 0.911 (0.874–0.943) | 0.865 (0.831–0.897) | 0.787 (0.752–0.821) | 0.743 (0.708–0.777) |
| Virchow2 | 0.911 (0.874–0.943) | 0.869 (0.837–0.901) | 0.717 (0.684–0.749) | 0.679 (0.646–0.711) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.911 (0.874–0.943) | 0.889 (0.856–0.922) | 0.784 (0.745–0.819) | 0.690 (0.650–0.729) |
| H-optimus-1 | 0.915 (0.878–0.947) | 0.796 (0.756–0.834) | 0.624 (0.579–0.666) | 0.578 (0.531–0.628) |
| H-optimus-0 | 0.911 (0.874–0.943) | 0.866 (0.834–0.903) | 0.722 (0.681–0.760) | 0.632 (0.593–0.670) |
| GigaPath | 0.911 (0.874–0.943) | 0.835 (0.798–0.872) | 0.656 (0.613–0.695) | 0.580 (0.538–0.618) |
| CRISP | 0.911 (0.874–0.943) | 0.849 (0.812–0.884) | 0.736 (0.694–0.777) | 0.659 (0.617–0.699) |
| Virchow2 | 0.911 (0.874–0.943) | 0.855 (0.820–0.890) | 0.626 (0.587–0.664) | 0.563 (0.524–0.599) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.911 (0.874–0.943) | 0.837 (0.789–0.886) | 0.553 (0.484–0.610) | 0.411 (0.346–0.468) |
| H-optimus-1 | 0.915 (0.878–0.947) | 0.703 (0.646–0.756) | 0.358 (0.297–0.419) | 0.346 (0.289–0.402) |
| H-optimus-0 | 0.911 (0.874–0.943) | 0.805 (0.760–0.858) | 0.463 (0.402–0.524) | 0.325 (0.268–0.382) |
| GigaPath | 0.911 (0.874–0.943) | 0.760 (0.707–0.813) | 0.374 (0.313–0.431) | 0.264 (0.207–0.317) |
| CRISP | 0.911 (0.874–0.943) | 0.781 (0.728–0.829) | 0.504 (0.443–0.565) | 0.382 (0.321–0.439) |
| Virchow2 | 0.911 (0.874–0.943) | 0.789 (0.740–0.837) | 0.301 (0.244–0.362) | 0.203 (0.155–0.260) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.909 (0.844–0.961) | 0.902 (0.844–0.957) | 0.857 (0.797–0.915) | 0.813 (0.753–0.873) |
| H-optimus-1 | 0.909 (0.844–0.974) | 0.905 (0.846–0.960) | 0.842 (0.784–0.903) | 0.784 (0.726–0.840) |
| H-optimus-0 | 0.909 (0.844–0.961) | 0.899 (0.841–0.951) | 0.825 (0.768–0.884) | 0.761 (0.705–0.818) |
| GigaPath | 0.922 (0.857–0.974) | 0.916 (0.860–0.967) | 0.835 (0.780–0.891) | 0.785 (0.730–0.840) |
| CRISP | 0.961 (0.909–1.000) | 0.944 (0.902–0.981) | 0.856 (0.807–0.904) | 0.808 (0.761–0.858) |
| Virchow2 | 0.935 (0.870–0.987) | 0.910 (0.856–0.957) | 0.831 (0.772–0.887) | 0.777 (0.715–0.831) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.909 (0.844–0.961) | 0.900 (0.840–0.956) | 0.831 (0.765–0.899) | 0.761 (0.689–0.831) |
| H-optimus-1 | 0.909 (0.844–0.974) | 0.904 (0.844–0.960) | 0.808 (0.744–0.878) | 0.715 (0.647–0.783) |
| H-optimus-0 | 0.909 (0.844–0.961) | 0.896 (0.837–0.947) | 0.785 (0.717–0.857) | 0.682 (0.615–0.748) |
| GigaPath | 0.922 (0.857–0.974) | 0.914 (0.858–0.965) | 0.792 (0.728–0.858) | 0.715 (0.646–0.780) |
| CRISP | 0.961 (0.909–1.000) | 0.938 (0.891–0.982) | 0.808 (0.748–0.869) | 0.736 (0.672–0.801) |
| Virchow2 | 0.935 (0.870–0.987) | 0.901 (0.840–0.955) | 0.785 (0.719–0.854) | 0.701 (0.632–0.768) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.909 (0.844–0.961) | 0.857 (0.779–0.935) | 0.727 (0.636–0.831) | 0.623 (0.520–0.727) |
| H-optimus-1 | 0.909 (0.844–0.974) | 0.857 (0.766–0.935) | 0.675 (0.571–0.779) | 0.520 (0.416–0.623) |
| H-optimus-0 | 0.909 (0.844–0.961) | 0.844 (0.753–0.922) | 0.636 (0.533–0.753) | 0.455 (0.351–0.558) |
| GigaPath | 0.922 (0.857–0.974) | 0.870 (0.792–0.948) | 0.636 (0.533–0.740) | 0.507 (0.403–0.610) |
| CRISP | 0.961 (0.909–1.000) | 0.909 (0.844–0.974) | 0.662 (0.558–0.754) | 0.546 (0.442–0.649) |
| Virchow2 | 0.935 (0.870–0.987) | 0.857 (0.779–0.935) | 0.649 (0.546–0.753) | 0.520 (0.416–0.623) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.924 (0.864–0.985) | 0.879 (0.819–0.935) | 0.723 (0.661–0.781) | 0.637 (0.583–0.691) |
| H-optimus-1 | 0.697 (0.576–0.803) | 0.697 (0.587–0.792) | 0.566 (0.482–0.646) | 0.498 (0.421–0.569) |
| H-optimus-0 | 0.712 (0.591–0.818) | 0.703 (0.603–0.797) | 0.568 (0.486–0.650) | 0.506 (0.433–0.582) |
| GigaPath | 0.697 (0.576–0.818) | 0.700 (0.592–0.795) | 0.571 (0.485–0.652) | 0.510 (0.433–0.584) |
| CRISP | 0.682 (0.561–0.788) | 0.675 (0.564–0.771) | 0.552 (0.467–0.634) | 0.490 (0.411–0.566) |
| Virchow2 | 0.773 (0.667–0.879) | 0.755 (0.661–0.844) | 0.639 (0.551–0.721) | 0.562 (0.486–0.634) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.924 (0.864–0.985) | 0.864 (0.797–0.927) | 0.634 (0.561–0.705) | 0.504 (0.441–0.568) |
| H-optimus-1 | 0.697 (0.576–0.803) | 0.697 (0.587–0.791) | 0.493 (0.414–0.578) | 0.386 (0.321–0.456) |
| H-optimus-0 | 0.712 (0.591–0.818) | 0.699 (0.598–0.795) | 0.493 (0.411–0.575) | 0.399 (0.329–0.476) |
| GigaPath | 0.697 (0.576–0.818) | 0.701 (0.595–0.794) | 0.501 (0.420–0.583) | 0.408 (0.335–0.484) |
| CRISP | 0.682 (0.561–0.788) | 0.672 (0.560–0.769) | 0.482 (0.399–0.565) | 0.386 (0.315–0.464) |
| Virchow2 | 0.773 (0.667–0.879) | 0.749 (0.653–0.840) | 0.575 (0.486–0.664) | 0.450 (0.382–0.523) |
| Model | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| FOCUS | 0.924 (0.864–0.985) | 0.803 (0.712–0.894) | 0.394 (0.288–0.500) | 0.227 (0.136–0.333) |
| H-optimus-1 | 0.697 (0.576–0.803) | 0.591 (0.470–0.712) | 0.242 (0.152–0.349) | 0.121 (0.061–0.212) |
| H-optimus-0 | 0.712 (0.591–0.818) | 0.591 (0.470–0.712) | 0.242 (0.152–0.349) | 0.136 (0.061–0.227) |
| GigaPath | 0.697 (0.576–0.818) | 0.591 (0.469–0.712) | 0.242 (0.152–0.349) | 0.136 (0.061–0.227) |
| CRISP | 0.682 (0.561–0.788) | 0.561 (0.439–0.682) | 0.242 (0.152–0.349) | 0.152 (0.076–0.243) |
| Virchow2 | 0.773 (0.667–0.879) | 0.652 (0.530–0.773) | 0.349 (0.242–0.470) | 0.182 (0.106–0.288) |
| Task | Diagnostic task | Without clinical | With clinical | |
|---|---|---|---|---|
| T9 | Diffuse glioma molecular-subtype classification | 0.928 (0.896–0.952) | 0.956 (0.940–0.971) | +0.028 |
| T5 | Glioma vs GNT | 0.899 (0.862–0.936) | 0.914 (0.883–0.947) | +0.016 |
| T8 | Diffuse vs circumscribed glioma | 0.956 (0.931–0.977) | 0.971 (0.953–0.985) | +0.015 |
| T7 | Non-neuroepithelial tumour classification | 0.978 (0.949–0.997) | 0.985 (0.959–0.999) | +0.008 |
| T1 | Non-neoplastic brain tissue vs tumour | 0.964 (0.941–0.983) | 0.972 (0.954–0.987) | +0.008 |
| T10 | PA vs PXA | 0.988 (0.958–1.000) | 0.990 (0.965–1.000) | +0.002 |
| Metric | Without clinical | With clinical | |
|---|---|---|---|
| Hierarchical F1 score | 0.865 (0.845–0.882) | 0.888 (0.871–0.905) | +0.024 |
| Depth-weighted hierarchical F1 score | 0.825 (0.801–0.847) | 0.854 (0.833–0.876) | +0.029 |
| Leaf accuracy | 0.702 (0.667–0.736) | 0.755 (0.720–0.791) | +0.054 |
| Balanced accuracy | 0.578 (0.528–0.627) | 0.628 (0.580–0.678) | +0.050 |
| Leaf macro F1 score | 0.582 (0.530–0.623) | 0.633 (0.578–0.676) | +0.051 |
| Leaf weighted F1 score | 0.691 (0.655–0.728) | 0.746 (0.710–0.781) | +0.055 |
| Reader | H-F1 | H-F1 + AI | H-F1 | DH-F1 | DH-F1 + AI | DH-F1 | ||
|---|---|---|---|---|---|---|---|---|
| Junior 1 | 0.735 | 0.796 | +0.061 | 0.645 | 0.716 | +0.070 | 0.003 | |
| Junior 2 | 0.670 | 0.817 | +0.147 | 0.556 | 0.724 | +0.168 | ||
| Junior 3 | 0.682 | 0.746 | +0.064 | 0.580 | 0.649 | +0.069 | ||
| Senior 1 | 0.785 | 0.803 | +0.017 | 0.031 | 0.690 | 0.712 | +0.022 | 0.031 |
| Senior 2 | 0.776 | 0.862 | +0.086 | 0.680 | 0.799 | +0.119 | ||
| Senior 3 | 0.876 | 0.876 | +0.000 | 1.000 | 0.810 | 0.810 | +0.000 | 1.000 |
| Reader | H-F1 | H-F1 + AI | H-F1 | DH-F1 | DH-F1 + AI | DH-F1 | ||
|---|---|---|---|---|---|---|---|---|
| Junior 1 | 0.711 | 0.784 | +0.073 | 0.001 | 0.593 | 0.680 | +0.087 | 0.002 |
| Junior 2 | 0.654 | 0.707 | +0.053 | 0.039 | 0.529 | 0.593 | +0.064 | 0.024 |
| Junior 3 | 0.767 | 0.783 | +0.016 | 0.299 | 0.677 | 0.698 | +0.021 | 0.172 |
| Senior 1 | 0.719 | 0.767 | +0.048 | 0.004 | 0.629 | 0.686 | +0.057 | 0.003 |
| Senior 2 | 0.702 | 0.807 | +0.105 | 0.587 | 0.723 | +0.136 | ||
| Senior 3 | 0.817 | 0.834 | +0.017 | 0.064 | 0.754 | 0.774 | +0.020 | 0.064 |
| Reader | H-F1 | H-F1 + AI | H-F1 | DH-F1 | DH-F1 + AI | DH-F1 | ||
|---|---|---|---|---|---|---|---|---|
| Junior 1 | 0.670 | 0.682 | +0.012 | 0.408 | 0.553 | 0.566 | +0.013 | 0.433 |
| Junior 2 | 0.600 | 0.591 | -0.009 | 0.932 | 0.463 | 0.451 | -0.012 | 0.931 |
| Junior 3 | 0.419 | 0.476 | +0.057 | 0.010 | 0.326 | 0.387 | +0.061 | 0.010 |
| Senior 1 | 0.496 | 0.508 | +0.012 | 0.125 | 0.393 | 0.404 | +0.011 | 0.125 |
| Senior 2 | 0.466 | 0.568 | +0.102 | 0.013 | 0.360 | 0.468 | +0.108 | 0.013 |
| Senior 3 | 0.529 | 0.550 | +0.021 | 0.126 | 0.462 | 0.477 | +0.015 | 0.126 |
| Reader | Unaided | AI-assisted | Mean | SD | 95% CI | |
|---|---|---|---|---|---|---|
| Junior 1 | 100 | 2.14 | 2.36 | +0.22 | 0.61 | [0.10, 0.34] |
| Junior 2 | 100 | 2.08 | 2.54 | +0.46 | 0.86 | [0.29, 0.63] |
| Junior 3 | 100 | 2.20 | 2.35 | +0.15 | 0.44 | [0.06, 0.24] |
| Senior 1 | 100 | 2.36 | 2.42 | +0.06 | 0.34 | [-0.01, 0.13] |
| Senior 2 | 100 | 2.27 | 2.71 | +0.44 | 0.69 | [0.31, 0.57] |
| Senior 3 | 100 | 2.61 | 2.61 | +0.00 | 0.00 | [0.00, 0.00] |
| Reader | Unaided | AI-assisted | Mean | SD | 95% CI | |
|---|---|---|---|---|---|---|
| Junior 1 | 77 | 2.16 | 2.38 | +0.22 | 0.60 | [0.09, 0.35] |
| Junior 2 | 77 | 2.16 | 2.38 | +0.22 | 0.91 | [0.02, 0.42] |
| Junior 3 | 77 | 2.38 | 2.44 | +0.06 | 0.34 | [-0.01, 0.14] |
| Senior 1 | 77 | 2.19 | 2.39 | +0.19 | 0.56 | [0.07, 0.32] |
| Senior 2 | 77 | 2.06 | 2.56 | +0.49 | 0.75 | [0.33, 0.66] |
| Senior 3 | 77 | 2.52 | 2.57 | +0.05 | 0.22 | [0.00, 0.10] |
| Reader | Unaided | AI-assisted | Mean | SD | 95% CI | |
|---|---|---|---|---|---|---|
| Junior 1 | 66 | 2.02 | 2.06 | +0.05 | 0.27 | [-0.02, 0.11] |
| Junior 2 | 66 | 1.86 | 2.00 | +0.14 | 0.96 | [-0.09, 0.37] |
| Junior 3 | 66 | 1.14 | 1.29 | +0.15 | 0.44 | [0.05, 0.26] |
| Senior 1 | 66 | 1.44 | 1.47 | +0.03 | 0.25 | [-0.03, 0.09] |
| Senior 2 | 66 | 1.24 | 1.62 | +0.38 | 1.03 | [0.13, 0.63] |
| Senior 3 | 66 | 1.36 | 1.41 | +0.05 | 0.21 | [-0.01, 0.10] |
| Reader | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| Junior 1 | 0.899/0.899 (+0.000) | 0.747/0.848 (+0.101) | 0.545/0.667 (+0.121) | 0.232/0.313 (+0.081) |
| Junior 2 | 0.876/0.980 (+0.104) | 0.784/0.930 (+0.146) | 0.485/0.760 (+0.275) | 0.134/0.490 (+0.356) |
| Junior 3 | 0.879/0.920 (+0.041) | 0.758/0.830 (+0.072) | 0.566/0.680 (+0.114) | 0.222/0.340 (+0.118) |
| Senior 1 | 0.940/0.940 (+0.000) | 0.860/0.890 (+0.030) | 0.690/0.720 (+0.030) | 0.370/0.400 (+0.030) |
| Senior 2 | 0.949/0.980 (+0.031) | 0.838/0.940 (+0.102) | 0.667/0.790 (+0.123) | 0.273/0.610 (+0.337) |
| Senior 3 | 0.980/0.980 (+0.000) | 0.960/0.960 (+0.000) | 0.800/0.800 (+0.000) | 0.550/0.550 (+0.000) |
| Reader | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| Junior 1 | 0.948/0.961 (+0.013) | 0.792/0.883 (+0.091) | 0.416/0.558 (+0.143) | 0.143/0.273 (+0.130) |
| Junior 2 | 0.932/0.922 (-0.009) | 0.767/0.831 (+0.064) | 0.521/0.597 (+0.077) | 0.205/0.312 (+0.106) |
| Junior 3 | 0.935/0.935 (+0.000) | 0.831/0.844 (+0.013) | 0.597/0.649 (+0.052) | 0.312/0.351 (+0.039) |
| Senior 1 | 0.921/0.921 (+0.000) | 0.684/0.789 (+0.105) | 0.553/0.645 (+0.092) | 0.316/0.382 (+0.066) |
| Senior 2 | 0.933/0.947 (+0.014) | 0.747/0.895 (+0.148) | 0.467/0.724 (+0.257) | 0.173/0.421 (+0.248) |
| Senior 3 | 0.921/0.921 (+0.000) | 0.842/0.868 (+0.026) | 0.711/0.737 (+0.026) | 0.434/0.461 (+0.026) |
| Reader | Level 1 | Level 2 | Level 3 | Level 4 |
|---|---|---|---|---|
| Junior 1 | 0.909/0.909 (+0.000) | 0.773/0.788 (+0.015) | 0.333/0.364 (+0.030) | 0.121/0.167 (+0.045) |
| Junior 2 | 0.898/0.894 (-0.004) | 0.780/0.758 (-0.022) | 0.356/0.379 (+0.023) | 0.102/0.136 (+0.035) |
| Junior 3 | 0.591/0.621 (+0.030) | 0.439/0.530 (+0.091) | 0.182/0.273 (+0.091) | 0.167/0.227 (+0.061) |
| Senior 1 | 0.677/0.692 (+0.015) | 0.554/0.569 (+0.015) | 0.215/0.215 (+0.000) | 0.092/0.092 (+0.000) |
| Senior 2 | 0.662/0.742 (+0.081) | 0.477/0.591 (+0.114) | 0.123/0.288 (+0.165) | 0.046/0.121 (+0.075) |
| Senior 3 | 0.636/0.667 (+0.030) | 0.470/0.515 (+0.045) | 0.333/0.364 (+0.030) | 0.288/0.318 (+0.030) |
| Reader | Accuracy | Accuracy + AI | Accuracy | Lymphoma recall | Recall + AI | Recall | ||
|---|---|---|---|---|---|---|---|---|
| Junior 1 | 0.735 | 0.843 | +0.108 | 0.090 | 0.753 | 0.988 | +0.235 | |
| Junior 2 | 0.660 | 0.718 | +0.058 | 0.164 | 0.780 | 0.866 | +0.085 | 0.062 |
| Junior 3 | 0.843 | 0.941 | +0.098 | 0.062 | 0.914 | 0.988 | +0.074 | 0.062 |
| Junior mean | 0.746 | 0.834 | +0.088 | 0.816 | 0.947 | +0.131 | ||
| Senior 1 | 0.863 | 0.892 | +0.029 | 0.333 | 0.864 | 0.889 | +0.025 | 0.500 |
| Senior 2 | 0.725 | 0.784 | +0.059 | 0.338 | 0.753 | 0.877 | +0.123 | 0.062 |
| Hardware configuration | Preprocessing (s) | Prediction (s) | Total (s) |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8468V CPU + NVIDIA A100 80GB GPU | 96.47 | 0.97 | 97.44 |
| AMD EPYC 7K62 CPU + NVIDIA A6000 48GB GPU | 110.71 | 0.89 | 111.60 |
| Intel(R) Xeon(R) Platinum 8157 CPU + NVIDIA RTX 3090 GPU | 138.47 | 1.00 | 139.47 |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| GigaPath | 1.000 (1.000–1.000) | 0.993 (0.979–1.000) | 1.000 (1.000–1.000) | 0.909 (0.700–1.000) | 0.996 (0.988–1.000) |
| H-optimus-0 | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| H-optimus-1 | 1.000 (1.000–1.000) | 0.993 (0.979–1.000) | 1.000 (1.000–1.000) | 0.909 (0.700–1.000) | 0.996 (0.988–1.000) |
| Phikon-v2 | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| UNI | 1.000 (1.000–1.000) | 0.993 (0.979–1.000) | 1.000 (1.000–1.000) | 0.909 (0.700–1.000) | 0.996 (0.988–1.000) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 1.000 (1.000–1.000) | 0.998 (0.995–1.000) | 0.998 (0.993–1.000) | 1.000 (1.000–1.000) | 0.999 (0.997–1.000) |
| GigaPath | 1.000 (1.000–1.000) | 0.995 (0.988–1.000) | 0.994 (0.985–1.000) | 1.000 (1.000–1.000) | 0.997 (0.993–1.000) |
| H-optimus-0 | 1.000 (1.000–1.000) | 0.997 (0.992–1.000) | 0.996 (0.989–1.000) | 1.000 (1.000–1.000) | 0.998 (0.994–1.000) |
| H-optimus-1 | 1.000 (1.000–1.000) | 0.997 (0.992–1.000) | 0.996 (0.989–1.000) | 1.000 (1.000–1.000) | 0.998 (0.994–1.000) |
| Phikon-v2 | 1.000 (1.000–1.000) | 0.997 (0.992–1.000) | 0.996 (0.989–1.000) | 1.000 (1.000–1.000) | 0.998 (0.994–1.000) |
| UNI | 1.000 (1.000–1.000) | 0.998 (0.995–1.000) | 0.998 (0.993–1.000) | 1.000 (1.000–1.000) | 0.999 (0.997–1.000) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| GigaPath | 0.773 (0.708–0.829) | 0.620 (0.553–0.683) | 0.428 (0.392–0.462) | 0.752 (0.719–0.783) | 0.568 (0.487–0.642) |
| H-optimus-0 | 0.721 (0.656–0.783) | 0.591 (0.524–0.659) | 0.408 (0.370–0.448) | 0.735 (0.699–0.770) | 0.547 (0.468–0.622) |
| H-optimus-1 | 0.760 (0.699–0.820) | 0.611 (0.543–0.678) | 0.422 (0.385–0.458) | 0.747 (0.713–0.781) | 0.565 (0.488–0.639) |
| Phikon-v2 | 0.709 (0.646–0.769) | 0.601 (0.534–0.668) | 0.415 (0.379–0.450) | 0.740 (0.708–0.773) | 0.549 (0.471–0.623) |
| UNI | 0.767 (0.709–0.824) | 0.639 (0.572–0.702) | 0.443 (0.404–0.480) | 0.765 (0.730–0.800) | 0.602 (0.523–0.674) |
| Virchow | 0.805 (0.751–0.857) | 0.707 (0.639–0.764) | 0.493 (0.451–0.535) | 0.812 (0.773–0.848) | 0.688 (0.619–0.750) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.969 (0.945–0.993) | 0.848 (0.794–0.897) | 0.977 (0.949–1.000) | 0.333 (0.174–0.486) | 0.912 (0.875–0.943) |
| GigaPath | 0.947 (0.914–0.979) | 0.836 (0.782–0.891) | 0.970 (0.937–0.993) | 0.303 (0.152–0.464) | 0.905 (0.867–0.938) |
| H-optimus-0 | 0.959 (0.930–0.987) | 0.842 (0.782–0.897) | 0.985 (0.962–1.000) | 0.273 (0.132–0.429) | 0.909 (0.871–0.942) |
| H-optimus-1 | 0.962 (0.935–0.989) | 0.848 (0.794–0.897) | 0.977 (0.949–1.000) | 0.333 (0.174–0.486) | 0.912 (0.875–0.943) |
| Phikon-v2 | 0.965 (0.938–0.991) | 0.824 (0.764–0.879) | 0.992 (0.977–1.000) | 0.152 (0.036–0.289) | 0.900 (0.862–0.934) |
| UNI | 0.965 (0.939–0.992) | 0.842 (0.788–0.891) | 0.970 (0.937–0.993) | 0.333 (0.174–0.486) | 0.908 (0.871–0.940) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.978 (0.952–1.000) | 0.962 (0.939–0.985) | 0.994 (0.981–1.000) | 0.902 (0.840–0.957) | 0.971 (0.953–0.988) |
| GigaPath | 0.977 (0.948–1.000) | 0.970 (0.947–0.989) | 0.982 (0.960–1.000) | 0.946 (0.894–0.989) | 0.977 (0.959–0.991) |
| H-optimus-0 | 0.980 (0.955–1.000) | 0.962 (0.939–0.985) | 0.977 (0.952–0.994) | 0.935 (0.884–0.980) | 0.971 (0.952–0.988) |
| H-optimus-1 | 0.983 (0.960–1.000) | 0.985 (0.970–0.996) | 0.994 (0.981–1.000) | 0.967 (0.928–1.000) | 0.988 (0.975–0.997) |
| Phikon-v2 | 0.976 (0.954–0.998) | 0.654 (0.593–0.711) | 1.000 (1.000–1.000) | 0.011 (0.000–0.035) | 0.790 (0.745–0.829) |
| UNI | 0.954 (0.923–0.984) | 0.821 (0.776–0.867) | 0.965 (0.935–0.989) | 0.554 (0.452–0.655) | 0.875 (0.840–0.909) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.634 (0.553–0.709) | 0.462 (0.370–0.555) | 0.290 (0.229–0.358) | 0.764 (0.729–0.799) | 0.430 (0.333–0.525) |
| GigaPath | 0.606 (0.527–0.686) | 0.445 (0.353–0.538) | 0.264 (0.214–0.321) | 0.752 (0.719–0.786) | 0.397 (0.301–0.491) |
| H-optimus-0 | 0.612 (0.530–0.688) | 0.504 (0.412–0.588) | 0.284 (0.242–0.328) | 0.771 (0.743–0.800) | 0.424 (0.328–0.522) |
| H-optimus-1 | 0.629 (0.552–0.706) | 0.471 (0.378–0.555) | 0.306 (0.246–0.374) | 0.773 (0.738–0.810) | 0.444 (0.347–0.539) |
| Phikon-v2 | 0.568 (0.484–0.646) | 0.454 (0.361–0.538) | 0.259 (0.214–0.307) | 0.755 (0.723–0.787) | 0.391 (0.297–0.484) |
| UNI | 0.621 (0.544–0.698) | 0.445 (0.353–0.538) | 0.262 (0.212–0.319) | 0.751 (0.720–0.784) | 0.393 (0.295–0.494) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| GigaPath | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| H-optimus-0 | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| H-optimus-1 | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| Phikon-v2 | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| UNI | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.999 (0.998–1.000) | 0.995 (0.988–1.000) | 0.994 (0.986–1.000) | 1.000 (1.000–1.000) | 0.997 (0.993–1.000) |
| GigaPath | 0.998 (0.995–1.000) | 0.990 (0.981–0.998) | 0.992 (0.981–1.000) | 0.980 (0.933–1.000) | 0.994 (0.989–0.999) |
| H-optimus-0 | 0.999 (0.996–1.000) | 0.993 (0.983–1.000) | 0.994 (0.986–1.000) | 0.980 (0.933–1.000) | 0.996 (0.990–1.000) |
| H-optimus-1 | 0.999 (0.997–1.000) | 0.995 (0.988–1.000) | 0.994 (0.986–1.000) | 1.000 (1.000–1.000) | 0.997 (0.993–1.000) |
| Phikon-v2 | 0.998 (0.995–1.000) | 0.993 (0.983–1.000) | 0.994 (0.986–1.000) | 0.980 (0.933–1.000) | 0.996 (0.990–1.000) |
| UNI | 0.998 (0.994–1.000) | 0.990 (0.981–0.998) | 0.992 (0.981–1.000) | 0.980 (0.933–1.000) | 0.994 (0.989–0.999) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.968 (0.949–0.987) | 0.908 (0.880–0.936) | 0.959 (0.934–0.981) | 0.803 (0.734–0.871) | 0.934 (0.912–0.955) |
| GigaPath | 0.958 (0.934–0.982) | 0.891 (0.860–0.919) | 0.985 (0.969–0.996) | 0.693 (0.610–0.772) | 0.924 (0.901–0.946) |
| H-optimus-0 | 0.956 (0.932–0.979) | 0.906 (0.875–0.934) | 0.940 (0.909–0.968) | 0.835 (0.767–0.898) | 0.931 (0.908–0.953) |
| H-optimus-1 | 0.962 (0.941–0.983) | 0.916 (0.885–0.941) | 0.917 (0.883–0.949) | 0.913 (0.859–0.959) | 0.937 (0.913–0.957) |
| Phikon-v2 | 0.885 (0.849–0.920) | 0.684 (0.639–0.728) | 1.000 (1.000–1.000) | 0.024 (0.000–0.052) | 0.811 (0.777–0.842) |
| UNI | 0.923 (0.894–0.952) | 0.781 (0.740–0.822) | 0.981 (0.964–0.996) | 0.362 (0.279–0.450) | 0.859 (0.828–0.886) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.774 (0.731–0.818) | 0.505 (0.434–0.577) | 0.393 (0.331–0.455) | 0.867 (0.848–0.885) | 0.476 (0.399–0.551) |
| GigaPath | 0.743 (0.693–0.792) | 0.467 (0.396–0.538) | 0.338 (0.291–0.385) | 0.856 (0.838–0.873) | 0.417 (0.338–0.493) |
| H-optimus-0 | 0.730 (0.681–0.778) | 0.478 (0.407–0.549) | 0.333 (0.296–0.371) | 0.857 (0.840–0.874) | 0.400 (0.320–0.477) |
| H-optimus-1 | 0.744 (0.696–0.792) | 0.489 (0.418–0.560) | 0.348 (0.306–0.391) | 0.859 (0.842–0.876) | 0.415 (0.335–0.494) |
| Phikon-v2 | 0.689 (0.634–0.742) | 0.451 (0.379–0.522) | 0.309 (0.273–0.346) | 0.845 (0.830–0.862) | 0.356 (0.283–0.434) |
| UNI | 0.752 (0.704–0.800) | 0.505 (0.434–0.577) | 0.388 (0.331–0.446) | 0.865 (0.847–0.884) | 0.470 (0.395–0.546) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.990 (0.971–1.000) | 0.986 (0.968–1.000) | 1.000 (1.000–1.000) | 0.914 (0.810–1.000) | 0.992 (0.981–1.000) |
| GigaPath | 0.993 (0.982–1.000) | 0.972 (0.949–0.991) | 1.000 (1.000–1.000) | 0.829 (0.690–0.944) | 0.984 (0.969–0.995) |
| H-optimus-0 | 0.992 (0.984–1.000) | 0.963 (0.935–0.986) | 0.995 (0.983–1.000) | 0.800 (0.667–0.929) | 0.978 (0.962–0.992) |
| H-optimus-1 | 0.996 (0.990–1.000) | 0.968 (0.945–0.991) | 0.995 (0.983–1.000) | 0.829 (0.700–0.946) | 0.981 (0.966–0.994) |
| Phikon-v2 | 0.996 (0.991–1.000) | 0.963 (0.935–0.986) | 0.989 (0.972–1.000) | 0.829 (0.697–0.943) | 0.978 (0.961–0.992) |
| UNI | 0.997 (0.994–1.000) | 0.972 (0.949–0.991) | 1.000 (1.000–1.000) | 0.829 (0.700–0.946) | 0.984 (0.970–0.995) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| GigaPath | 0.999 (0.998–1.000) | 0.982 (0.965–0.996) | 0.995 (0.984–1.000) | 0.919 (0.818–1.000) | 0.990 (0.978–0.997) |
| H-optimus-0 | 0.999 (0.998–1.000) | 0.982 (0.965–0.996) | 0.995 (0.984–1.000) | 0.919 (0.818–1.000) | 0.989 (0.978–0.997) |
| H-optimus-1 | 0.999 (0.998–1.000) | 0.991 (0.978–1.000) | 0.989 (0.973–1.000) | 1.000 (1.000–1.000) | 0.995 (0.986–1.000) |
| Phikon-v2 | 1.000 (0.999–1.000) | 0.987 (0.969–1.000) | 1.000 (1.000–1.000) | 0.919 (0.828–1.000) | 0.992 (0.982–1.000) |
| UNI | 1.000 (0.999–1.000) | 0.987 (0.969–1.000) | 0.995 (0.984–1.000) | 0.946 (0.862–1.000) | 0.992 (0.982–1.000) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 1.000 (1.000–1.000) | 0.995 (0.984–1.000) | 0.994 (0.981–1.000) | 1.000 (1.000–1.000) | 0.997 (0.990–1.000) |
| GigaPath | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| H-optimus-0 | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| H-optimus-1 | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| Phikon-v2 | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| UNI | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) | 1.000 (1.000–1.000) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.985 (0.971–0.999) | 0.970 (0.955–0.983) | 0.975 (0.960–0.988) | 0.946 (0.893–0.989) | 0.982 (0.973–0.990) |
| GigaPath | 0.981 (0.971–0.992) | 0.843 (0.812–0.875) | 0.998 (0.993–1.000) | 0.098 (0.040–0.168) | 0.913 (0.894–0.932) |
| H-optimus-0 | 0.986 (0.971–1.000) | 0.966 (0.951–0.979) | 0.964 (0.946–0.980) | 0.978 (0.941–1.000) | 0.979 (0.969–0.988) |
| H-optimus-1 | 0.986 (0.968–1.000) | 0.961 (0.942–0.976) | 0.955 (0.935–0.973) | 0.989 (0.963–1.000) | 0.976 (0.965–0.985) |
| Phikon-v2 | 0.961 (0.940–0.982) | 0.847 (0.815–0.877) | 0.998 (0.993–1.000) | 0.120 (0.055–0.190) | 0.915 (0.896–0.933) |
| UNI | 0.970 (0.953–0.986) | 0.920 (0.897–0.942) | 0.968 (0.952–0.983) | 0.685 (0.594–0.779) | 0.952 (0.937–0.966) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.862 (0.840–0.884) | 0.762 (0.737–0.788) | 0.635 (0.579–0.691) | 0.808 (0.783–0.836) | 0.588 (0.540–0.632) |
| GigaPath | 0.872 (0.851–0.894) | 0.840 (0.820–0.862) | 0.444 (0.386–0.502) | 0.985 (0.976–0.993) | 0.598 (0.541–0.653) |
| H-optimus-0 | 0.884 (0.864–0.904) | 0.830 (0.808–0.852) | 0.430 (0.373–0.488) | 0.975 (0.964–0.985) | 0.574 (0.514–0.629) |
| H-optimus-1 | 0.875 (0.854–0.896) | 0.825 (0.802–0.847) | 0.580 (0.522–0.638) | 0.914 (0.894–0.932) | 0.639 (0.590–0.685) |
| Phikon-v2 | 0.841 (0.817–0.865) | 0.683 (0.655–0.711) | 0.805 (0.758–0.850) | 0.638 (0.607–0.672) | 0.576 (0.533–0.615) |
| UNI | 0.878 (0.857–0.898) | 0.823 (0.801–0.847) | 0.382 (0.325–0.439) | 0.984 (0.975–0.992) | 0.536 (0.474–0.593) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| GigaPath | 0.989 (0.982–0.997) | 0.975 (0.965–0.985) | 0.982 (0.971–0.991) | 0.965 (0.946–0.981) | 0.980 (0.972–0.987) |
| H-optimus-0 | 0.993 (0.987–0.998) | 0.983 (0.974–0.990) | 0.988 (0.979–0.995) | 0.974 (0.958–0.988) | 0.986 (0.979–0.992) |
| H-optimus-1 | 0.992 (0.986–0.998) | 0.985 (0.978–0.992) | 0.990 (0.981–0.997) | 0.979 (0.965–0.991) | 0.988 (0.982–0.993) |
| Phikon-v2 | 0.989 (0.981–0.997) | 0.967 (0.956–0.978) | 0.990 (0.981–0.997) | 0.932 (0.907–0.956) | 0.973 (0.964–0.982) |
| UNI | 0.989 (0.982–0.997) | 0.976 (0.967–0.985) | 0.985 (0.976–0.994) | 0.963 (0.944–0.980) | 0.981 (0.973–0.988) |
| Virchow | 0.992 (0.985–0.998) | 0.983 (0.974–0.990) | 0.991 (0.983–0.997) | 0.970 (0.952–0.984) | 0.986 (0.979–0.992) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.964 (0.947–0.980) | 0.920 (0.893–0.943) | 0.500 (0.340–0.658) | 0.960 (0.940–0.978) | 0.519 (0.369–0.651) |
| GigaPath | 0.935 (0.912–0.959) | 0.883 (0.850–0.911) | 0.725 (0.578–0.857) | 0.898 (0.867–0.927) | 0.518 (0.393–0.625) |
| H-optimus-0 | 0.961 (0.944–0.979) | 0.937 (0.913–0.959) | 0.575 (0.415–0.729) | 0.971 (0.955–0.986) | 0.613 (0.464–0.733) |
| H-optimus-1 | 0.960 (0.942–0.978) | 0.928 (0.902–0.950) | 0.425 (0.273–0.585) | 0.976 (0.961–0.990) | 0.507 (0.346–0.648) |
| Phikon-v2 | 0.956 (0.936–0.976) | 0.926 (0.900–0.948) | 0.700 (0.559–0.833) | 0.948 (0.925–0.967) | 0.622 (0.494–0.727) |
| UNI | 0.965 (0.949–0.981) | 0.917 (0.891–0.941) | 0.850 (0.733–0.952) | 0.924 (0.898–0.948) | 0.642 (0.524–0.737) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| FOCUS | 0.616 (0.572–0.660) | 0.510 (0.462–0.560) | 0.277 (0.254–0.305) | 0.771 (0.757–0.787) | 0.437 (0.387–0.495) |
| GigaPath | 0.541 (0.498–0.587) | 0.500 (0.455–0.548) | 0.260 (0.243–0.277) | 0.762 (0.750–0.774) | 0.403 (0.347–0.460) |
| H-optimus-0 | 0.598 (0.556–0.643) | 0.507 (0.457–0.555) | 0.274 (0.251–0.299) | 0.765 (0.751–0.780) | 0.428 (0.372–0.484) |
| H-optimus-1 | 0.614 (0.571–0.658) | 0.502 (0.452–0.552) | 0.268 (0.248–0.289) | 0.767 (0.753–0.782) | 0.429 (0.374–0.484) |
| Phikon-v2 | 0.660 (0.621–0.700) | 0.533 (0.486–0.581) | 0.304 (0.274–0.332) | 0.787 (0.770–0.803) | 0.479 (0.427–0.531) |
| UNI | 0.613 (0.571–0.658) | 0.495 (0.448–0.543) | 0.269 (0.246–0.295) | 0.770 (0.755–0.785) | 0.429 (0.374–0.483) |
| Model | AUC | Accuracy | Sensitivity | Specificity | F1 score |
|---|---|---|---|---|---|
| H-optimus-0 | 0.999 (0.997–1.000) | 0.975 (0.963–0.985) | 0.978 (0.964–0.990) | 0.968 (0.943–0.990) | 0.981 (0.972–0.989) |
| H-optimus-1 | 0.999 (0.999–1.000) | 0.979 (0.968–0.990) | 0.972 (0.955–0.986) | 0.995 (0.984–1.000) | 0.985 (0.975–0.992) |
| Phikon-v2 | 0.997 (0.995–0.999) | 0.966 (0.953–0.978) | 0.980 (0.967–0.992) | 0.936 (0.900–0.966) | 0.975 (0.965–0.984) |
| UNI | 0.999 (0.999–1.000) | 0.984 (0.974–0.993) | 0.993 (0.985–1.000) | 0.964 (0.937–0.986) | 0.988 (0.981–0.995) |
| Virchow2 | 1.000 (0.999–1.000) | 0.991 (0.984–0.997) | 0.996 (0.989–1.000) | 0.982 (0.961–0.996) | 0.993 (0.988–0.998) |
| Virchow | 0.999 (0.998–1.000) | 0.978 (0.966–0.988) | 0.974 (0.958–0.987) | 0.986 (0.970–1.000) | 0.984 (0.975–0.991) |
| Model | accuracy | Macro-F1 (%) | Balanced accuracy |
|---|---|---|---|
| FOCUS | 73.04 0.96 | 72.82 0.99 | 73.04 0.96 |
| UNI | 70.84 0.86 | 70.61 0.86 | 70.84 0.86 |
| Virchow2 | 70.68 0.83 | 70.38 0.85 | 70.68 0.83 |
| Virchow | 70.23 0.41 | 70.00 0.42 | 70.23 0.41 |
| H-Optimus-0 | 70.06 0.98 | 69.81 0.98 | 70.06 0.98 |
| GigaPath | 69.99 0.63 | 69.80 0.62 | 69.99 0.63 |
| Task | AUC, 224 px | AUC, 448 px | AUC, 896 px | AUC, 448–224 px | AUC, 896–224 px |
|---|---|---|---|---|---|
| T1 | 0.964 | 0.971 | 0.971 | +0.007 | +0.007 |
| T2 | 0.995 | 0.997 | 0.993 | +0.001 | -0.003 |
| T3 | 0.992 | 0.988 | 0.983 | -0.004 | -0.009 |
| T4 | 0.987 | 0.985 | 0.981 | -0.002 | -0.006 |
| T5 | 0.898 | 0.917 | 0.935 | +0.018 | +0.037 |
| T6 | 0.992 | 0.997 | 0.987 | +0.005 | -0.005 |
| Contrast (Full FOCUS minus counterpart) | Tasks with AUC | Mean AUC | Bootstrap 95% CI | Wilcoxon |
|---|---|---|---|---|
| FAMV | 17/23 | +0.0012 | +0.0000 to +0.0022 | 0.0327 |
| HCDC | 22/23 | +0.0125 | +0.0085 to +0.0173 |