CoRe-BT: A Multimodal Radiology-Pathology-Text Benchmark for Robust Brain Tumor Typing
Authors: Juampablo E. Heras Rivera, Daniel K. Low, Xavier Xiong, Jacob J. Ruzevick, Daniel D. Child, Wen-wai Yim, Mehmet Kurt, Asma Ben Abacha
Organizations: University of Washington, Seattle, Washington 98105, USA · University of Washington School of Medicine, Seattle, Washington 98105, USA · Microsoft Health AI, Redmond, Washington 98052-6399, USA
Accurate brain tumor typing requires integrating heterogeneous clinical evidence, including magnetic resonance imaging (MRI), histopathology, and pathology reports, which are often incomplete at the time of diagnosis. We introduce CoRe-BT, a cross-modal radiology-pathology-text benchmark for brain tumor typing, designed to study robust multimodal learning under missing modality conditions. The dataset comprises 310 patients with multi-sequence brain MRI (T1, T1c, T2, FLAIR), including 95 cases with paired H&E-stained whole-slide pathology images and pathology reports. All cases are annotated with tumor type and grade, and MRI volumes include expert-annotated tumor masks, enabling both region-aware modeling and auxiliary learning tasks. Tumors are categorized into six clinically relevant classes capturing the heterogeneity of common and rare glioma subtypes. We evaluate tumor typing under variable modality availability by comparing MRI-only models with multimodal approaches that incorporate pathology information when present. Baseline experiments demonstrate the feasibility of multimodal fusion and highlight complementary modality contributions across clinically relevant typing tasks. CoRe-BT provides a grounded testbed for advancing multimodal glioma typing and representation learning in realistic scenarios with incomplete clinical data.
Table 1: Hierarchical tumor classification labels present in the CoRe-BT dataset. Evaluation is performed at the Level 1 category.
Figure 1: Overview of the proposed CoRe-BT-Fusion framework. Pretrained histopathology and MRI foundation models generate modality-specific embeddings, which are fused for supervised tumor type prediction under variable modality availability.
Task
Model Configuration
Acc. (Macro)
Prec. (Macro)
Rec. (Macro)
F1 (Macro)
LGG-HGG
MRI only
0.812
0.727
0.812
0.757
Pathology only
0.812
0.727
0.812
0.757
Fusion
0.792
0.690
0.792
0.717
Fusion (MRI ablated)
0.792
0.690
0.792
0.717
Fusion (Pathology ablated)
0.833
0.778
0.833
0.801
Fusion (Both ablated)
0.812
0.727
0.812
0.757
Table 2: Modality ablation study for LGG-HGG classification.
Task
Model Configuration
Acc. (Macro)
Prec. (Macro)
Rec. (Macro)
F1 (Macro)
Level 1 Class
MRI only
0.470
0.488
0.470
0.467
Pathology only
0.470
0.488
0.470
0.467
Fusion
0.556
0.595
0.556
0.560
Fusion (MRI ablated)
0.525
0.523
0.525
0.520
Fusion (Pathology ablated)
0.269
0.292
0.269
0.255
Fusion (Both ablated)
0.470
0.488
0.470
0.467
Table 3: Modality ablation study for Level 1 Class and WHO Grade classification.
Guangdong Institute of Intelligence Science and Technology, Hengqin, China · Center for Brain-Inspired Computing Research (CBICR), Department of Precision Instrument, Tsinghua University, Beijing, China