Deep Learning-Based Prediction of HER2 Status from Whole-Brain MRI in Breast Cancer Brain Metastases
Authors: Sara Early, Chris Kang, Vibujithan Vigneshwaran, Kimberly Amador, Roger Y. Tsang, Nils D. Forkert
Organizations: Department of Radiology, University of Calgary, Calgary, AB, Canada · Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada · Alberta Children’s Hospital Research Institute, University of Calgary, Calgary, AB, Canada · Department of Biomedical Engineering, University of Calgary, Calgary, AB, Canada · Annie Charbonneau Cancer Institute, University of Calgary, Calgary, AB, Canada · Department of Oncology, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada · Arthur J.E. Child Comprehensive Cancer Centre, Calgary, AB, Canada · Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
Purpose: Human epidermal growth factor receptor 2 (HER2) status is a key biomarker relevant for prognostication and treatment decision making in metastatic breast cancer. Thus, the aim of this work was to explore whether HER2 status can be predicted directly from whole-brain post-contrast T1-weighted (T1w) MRI scans in breast cancer brain metastases (BCBM) using deep learning, and whether providing explicit tumour segmentation information improves prediction. Methods: Using the publicly available BCBM-Radiogenomics dataset, we analysed 135 patients, including 70 HER2+ and 65 HER2- patients. Three approaches were implemented and evaluated: a conventional radiomics model, a simple fully convolutional network (SFCN) using T1w MRI scans alone, and an SFCN using T1w MRI scans with an expert-derived tumour mask as an additional input channel. Results: The radiomics model achieved an area under the receiver operating characteristic curve (ROC-AUC) of 0.602 (95% confidence interval [CI]: 0.528--0.677) while the SFCN achieved a ROC-AUC of 0.667 (95% CI, 0.63--0.71) using T1w MRI scans alone, which increased to 0.692 (95% CI, 0.58--0.81) when an explicit tumour mask was additionally provided. Conclusion: However, the modest performance across all approaches demonstrates the difficulty of predicting HER2 status from structural MRI scans alone. The difference between the segmentation-free and mask-guided SFCN models suggests that much of the predictive signal captured by the deep-learning model may be learned directly from whole-brain T1w MRI without requiring explicit tumour delineation. These findings support further investigation of segmentation-free deep-learning approaches as a potential non-invasive strategy to support HER2 status determination in BCBM.
Figures & tables
Figure 1: Three approaches were evaluated for HER2 status classification: (a) a conventional radiomics model, (b) a simple fully convolutional network (SFCN) using T1w MRI scans alone, and (c) an SFCN using T1w MRI scans with an expert-delineated tumour mask as an additional input channel
Cohort variable
Overall ( n=135 )
HER2+ ( n=70 )
HER2- ( n=65 )
Age, years (mean ± SD)
56.9 ± 10.4
56.6 ± 10.4
57.2 ± 10.5
Number of metastases (mean ± SD; median)
3.3 ± 2.9; 2
3.1 ± 2.9; 2
3.4 ± 2.8; 2
Total lesion volume, cm 3 (mean ± SD; median)
6.7 ± 8.3; 3.6
5.8 ± 8.9; 3.1
7.6 ± 7.5; 5.3
1.5T scans, n
126
63
63
3T scans, n
9
7
2
Table 1: MRI acquisition characteristics of the study cohort stratified by HER2 status.
Feature representation
No. of features
Definition
Tumour burden
4
Number of lesions, total lesion volume, largest lesion volume, and log-transformed total lesion volume.
Largest-lesion radiomics
107
Complete radiomic feature vector from the largest lesion for each patient.
Mean-pooled radiomics
107
Mean of each radiomic feature across all lesions for each patient.
Max-pooled radiomics
107
Maximum value of each radiomic feature across all lesions for each patient.
Mean- and max-pooled radiomics
214
Concatenation of the mean- and max-pooled radiomic feature vectors.
Largest-lesion radiomics + age
108
Largest-lesion radiomic feature vector combined with patient age.
Table 2: Patient-level feature representations evaluated for HER2 classification.
Feature representation
No. of features
ROC-AUC
95% CI
Tumour burden
4
0.602
0.528–0.677
Largest-lesion radiomics
107
0.553
0.460–0.646
Mean+max radiomics
214
0.552
0.451–0.653
Mean-pooled radiomics
107
0.541
0.422–0.659
Max-pooled radiomics
107
0.536
0.431–0.642
Largest-lesion + clinical
108
0.525
0.305–0.745
Table 3: Patient-level HER2 classification performance of the seven feature representations evaluated using nested cross-validation. ROC-AUC, area under the receiver operating characteristic curve; CI, confidence interval.
Model
Input
Segmentation required
ROC-AUC
95% CI
Radiomics (Logistic Regression)
4 radiomic features
Yes
0.602
0.528–0.677
SFCN
T1w MRI
No
0.667
0.63–0.71
SFCN
T1w MRI + tumour mask
Yes
0.692
0.58–0.81
Table 4: Comparison of radiomics and SFCN approaches for patient-level HER2 classification performance.
Figure 2: Receiver operating characteristic (ROC) curves for HER2 status prediction. Curves are shown for the tumour-burden radiomics model, the T1w-only SFCN, and the T1w + tumour mask SFCN. The corresponding mean-of-folds AUCs were 0.602, 0.667, and 0.692, respectively
Prediction of pathological complete response before neoadjuvant chemotherapy may facilitate more tailored therapeutic planning for breast cancer patients. This work proposes a deep-learning model for pretreatment data only, combining apparent diffusion coefficient maps, dynamic contrast-enhanced magnetic resonance imaging, and clinical variables. The study uses the public ACRIN 6698/I-SPY2 multicenter dataset. The architecture employs EfficientNet-B0 pretrained encoders for image feature extraction and late fusion with clinical information. Multiple clinical variables were evaluated, including age, race, histological type, HR/HER2 subtype, SBR grade, and maximum diameter. Only HR/HER2 subtype improved the average area under the receiver operating characteristic curve (AUC) and was retained in the final model. Using stratified five-fold cross-validation, standalone apparent diffusion coefficient maps achieved a mean AUC of 0.79, whereas dynamic contrast-enhanced magnetic resonance imaging achieved 0.74. Adding HR/HER2 subtype improved performance to 0.83 and 0.81, respectively. The final configuration, using both imaging modalities and HR/HER2 subtype, achieved an AUC of 0.86. These results support pretreatment multimodal learning for response prediction, although external validation is required before clinical use.
Pablo García Marcos, Paula Puerta González, Guillermo Lorenzo +6
Computer Science Department, University of Oviedo, Asturias, Spain · Biomedical Engineering Center (BME), University of Oviedo, Asturias, Spain · Electrical Engineering Department, University of Oviedo, Asturias, Spain +7
We present HERMES (Hybrid Ensemble for Radiotherapy-target segmentation, Malignancy staging, and Event-free Survival), a single containerized algorithm for the three HECKTOR 2026 subtasks: segmentation of the primary tumor (GTVp) and pathological lymph nodes (GTVn), radiological T/N staging, and recurrence-free survival (RFS), computed from a paired FDG-PET/CT scan and an electronic health record. A 10-fold ensemble of STU-Net Small networks produces the segmentation; the predicted mask then drives two downstream tasks. Rather than pass a generic radiomics vector to the staging models, we derive from the predicted masks a compact set of geometry features aligned with the size and number axes of AJCC/UICC 7th-edition radiological N/T staging. On internal cross-validation these features raise N-stage balanced accuracy from 0.691 to 0.720 (+0.030), our largest single design gain, at lower feature dimensionality. For prognosis we combine complementary deep and clinical risk experts in an equal-weight ensemble, and train one deep expert with a concordance-tracking survival loss of our own, whose value approximates the concordance index during training. Every component was selected on honest out-of-fold predictions under a regularization-oriented protocol, with no tuning on the public validation set, and deployed as two decorrelated submissions. On the HECKTOR 2026 validation leaderboard, HERMES achieved a weighted score of 0.6454 (Mean Dice 0.641, T balanced accuracy 0.580, N balanced accuracy 0.642, RFS C-index 0.679) and qualified for the testing phase. Team: AMC_HNC.
Kai Wang, Meixu Chen, Elie Nasr +2
Department of Radiation Oncology, University of Colorado School of Medicine, Aurora, CO, USA
Efficient and explainable breast cancer (BC) risk prediction is critical for large-scale population-based screening. Breast MRI provides functional information for personalized risk assessment. Yet effective modeling remains challenging as fully 3D CNNs capture volumetric context at high computational cost, whereas lightweight 2D CNNs fail to model inter-slice continuity. Importantly, breast MRI modeling for shor- and long-term BC risk stratification remains underexplored. In this study, we propose LoGo-MR, a 2.5D local-global structural modeling framework for five-year BC risk prediction. Aligned with clinical interpretation, our framework first employs neighbor-slice encoding to capture subtle local cues linked to short-term risk. It then integrates transformer-enhanced multiple-instance learning (MIL) to model distributed global patterns related to long-term risk and provide interpretable slice importance. We further apply this framework across axial, sagittal, and coronal planes as LoGo3-MR to capture complementary volumetric information. This multi-plane formulation enables voxel-level risk saliency mapping, which may assist radiologists in localizing risk-relevant regions during breast MRI interpretation. Evaluated on a large breast MRI screening cohort (~7.5K), our method outperforms 2D/3D baselines and existing SOTA MIL methods, achieving AUCs of 0.77-0.69 for 1- to 5-year prediction and improving C-index by ~6% over 3D CNNs. LoGo3-MR further improves overall performance with interpretable localization across three planes, and validation across seven backbones shows consistent gains. These results highlight the clinical potential of efficient MRI-based BC risk stratification for large-scale screening. Code will be released publicly.
Xin Wang, Yuan Gao, George Yiasemis +13
Netherlands Cancer Institute · Radboud University Medical Center · University Medical Center Utrecht +1