AMO-ENE: Attention-based Multi-Omics Fusion Model for Outcome Prediction in Extra Nodal Extension and HPV-associated Oropharyngeal Cancer
Authors: Gautier Hénique, William Le, Gabriel Dayan, Coralie Brodeur, Kristoff Nelson, Apostolos Christopoulos, Edith Filion, Phuc-Felix Nguyen-Tan, Laurent Letourneau-Guillon, Houda Bahig, Samuel Kadoury
Organizations: MedICAL Laboratory, Polytechnique Montréeal, Montreal, Canada · Centre de recherche du CHUM (CRCHUM), Montréal, Canada · Centre Hospitalier de l’Université de Montréal (CHUM), Montreal Canada
Extranodal extension (ENE) is an emerging prognostic factor in human papillomavirus (HPV)-associated oropharyngeal cancer (OPC), although it is currently omitted as a clinical staging criteria. Recent works have advocated for the inclusion of iENE as a prognostic marker in HPV-positive OPC staging. However, several practical limitations continue to hinder its clinical integration, including inconsistencies in segmentation, low contrast in the periphery of metastatic lymph nodes on CT imaging, and laborious manual annotations. To address these limitations, we propose a fully automated end-to-end pipeline that uses computed tomography (CT) images with clinical data to assess the status of nodal ENE and predict treatment outcomes. Our approach includes a hierarchical 3D semi-supervised segmentation model designed to detect and delineate relevant iENE from radiotherapy planning CT scans. From these segmentations, a set of radiomics and deep features are extracted to train an imaging-detected ENE grading classifier. The predicted ENE status is then evaluated for its prognostic value and compared with existing staging criteria. Furthermore, we integrate these nodal features with primary tumor characteristics in a multimodal, attention-based outcome prediction model, providing a dynamic framework for outcome prediction. Our method is validated in an internal cohort of 397 HPV-positive OPC patients treated with radiation therapy or chemoradiotherapy between 2009 and 2020. For outcome prediction at the 2-year mark, our pipeline surpassed baseline models with 88.2% (4.8) in AUC for metastatic recurrence, 79.2% (7.4) for overall survival, and 78.1% (8.6) for disease-free survival. We also obtain a concordance index of 83.3% (6.5) for metastatic recurrence, 71.3% (8.9) for overall survival, and 70.0% (8.1) for disease-free survival, making it feasible for clinical decision making.
Figures & tables
Figure 1: Illustration for iENE status grading system. Nodes are presented with white borders, red healthy tissue background and black tumoral content. Grade 0 indicates metastatic pathological node with no signs of extension. Grade 1 indicates extension through the nodal capsule into surrounding fat. Grade 2 characterizes coalescent nodal masses with loss of internodal planes. Grade 3 is defined by overt invasion of surrounding structures and tissues. In case of multiple ENE, the maximum grade is reported.
Figure 2: Overall schematic diagram of the proposed framework for CT-based iENE status and prognosis prediction. First, an automated segmentation and selection of pathological node allows for the retrieval of the largest metastatic lymph node. Afterwards, radiomics and deep features are extracted from the structure to classify iENE grades in a binary problem. Finally, from the obtained predictions, the survival characteristics of predicted iENE groups are compared to demonstrate the influence of extra nodal extensions in HPV-associated OPC.
Outcome
Event type
Male
Female
Total
OS
Uncensored
27
11
38
Censored
290
69
359
DM
Uncensored
30
5
35
Censored
287
75
362
DFS
Uncensored
52
16
68
Censored
265
64
329
Table 1: Outcome representation in the patient cohort for Uncensored (Event presence at the latest available follow up) and Censored patients, stratified for different outcome measures 9OS, DM, DFS).
Figure 3: Graphical representation of the proposed attention-based modality fusion pipeline. In (A), multiple modality-specific feature encoders are trained in parallel, to extract features from each specific modality. The latent representation of each modality is concatenated and a multi-head scaled dot product attention is performed to evaluate interactions and combine latent representations. A final decision layer is adaptable to either predict a time dependent outcome target or multi-bin risk modeling. Details are provided in Sec. 3.4 for relevant notations. In (B), the proposed modality encoder module is shown, which is composed of multiple blocks containing parametric layers. (C) Detailed description of the operation performed in the attention layer.
Figure 4: Qualitative results for the iENE segmentation task on radiotherapy planning CT images. Each row presents the raw CT scan and the output segmentation maps for two distinct patients from test folds over a 5-fold cross validation. Comparative methods are shown in their respective columns : proposed model, nnUnet [ 20 ] , SwinUNETRV2 [ 16 ] , and SamMed3D [ 42 ] . GT: ground truth mask.
Features
Method
iENE - vs iENE +
0-1 / 2-3
0-1-2 / 3
Radiomics
RF + PCA
78.12 ± 4.67
76.93 ± 5.98
89.87 ± 5.24
Lasso + MLP
77.55 ± 2,65
74.98 ± 2.80
80.64 ± 17.31
PCA + XGBOOST
76.39 ± 2.56
80.60 ± 4.52
86.60 ± 8.40
Lasso + XGBOOST
79.92 ± 2.51
78.00 ± 1.61
88.85 ± 5.27
FMCIB
PCA + XGBOOST
70.33 ± 3.41
68.32 ± 5.51
80.52 ± 12.18
Lasso + XGBOOST
71.82 ± 2.32
64.98 ± 5.69
68.19 ± 10.36
Table 4: iENE grade classification results for 3 different dichotomization schemes (iENE - vs iENE + ; 0-1/2-3; 0-1-2/3). Results show the mean AUC ( ± standard deviation) over the 5 tests fold obtained in a stratified cross-validation. Features indicate the CT feature extraction strategy used as inputs to the classification method.
Figure 5: ROC curves obtained for the binary classification of dichotomized iENE classes, for 3 different dichotomization schemes (iENE - vs iENE + ; 0-1/2-3; 0-1-2/3). Individual patient logits were obtained in a 5-fold cross validation scheme with the L2 norm penalized XGBOOST classifier.
Figure 6: SHAP explanation analysis plots for the prediction of iENE- vs iENE + cases, with the top features obtained from the radiomics branch of the prediction framework.
Figure 7: Kaplan Meier curves obtained by stratifying all 397 patients with the predicted iENE + score. (a) Survival curves from ground-truth pathology. (b) Survival curves from predicted iENE. Blue lines constitute the real or predicted iENE- groups, red lines show the real or predicted iENE+ population. The selected node for grade classification is obtained as the largest predicted volume. The Kaplan Meyer fit function is unimodal and takes as input the tresholded (from AUC) binary prediction value. Each curve displays as legend the p-value obtained in a logrank test (DM: p=9.6×10−10 , DFS: p=9.83×10−5 ; OS: p=2.8×10−3 ).
Outcome
C
C1
C2
C3
SR
Proposed
DM
2.34×10−2
1.76×10−2
3.13×10−2
1.61×10−1
7.78×10−2
8.02×10−5
DFS
1.98×10−1
2.16×10−1
3.03×10−2
3.39×10−1
1.93×10−1
3.30×10−2
OS
2.62×10−1
1.82×10−1
1.86×10−1
3.32×10−1
2.82×10−1
2.90×10−2
Table 5: Log-rank statistical test for a separate set of 55 patients annotated by three separate radiologists. C : A collegial decision involved a three way consensus over the case definition, and served as the ground truth for model training. C1−3 refers to clinician 1, 2 or 3 respectively. SR: a single reader is simulated by randomly selecting one of the three annotations per patient, and bootstrapping the obtained results 10 000 times, giving the average score a unique reader could obtain.
Figure 8: Modality ablation experiment results for the 2-year therapeutic outcome prediction model. For readability, C indicates the clinical variables (including the TNM staging), P represents the primary tumor value (GTV radiomic characteristics) and N the predicted nodal volume (iENE) radiomics. Each result presents the mean AUC over the 5 tests fold obtained in a stratified k-fold validation. For each modality and combination of modality, we present the best mean AUC obtained over a grid search as described in 3.5 , as well as the standard deviation plotted in red dashed lines.
Model
Algorithm
DM 2 Y (6 % )
DFS 2Y (12 % )
OS 2Y (5 % )
Cox
Proportional Hazard Regression
69.7 ± 10.7
67.0 ± 8.5
63.4 ± 17.9
Kazmierski et al. [ 22 ]
Logistic Regression
68.2 ± 6.4
59.4 ± 17.9
57.1 ± 11.6
Rebaud et al. [ 34 ]
Bagged risk estimation
70.7 ± 5.0
62.9 ± 3.2
60.9 ± 12.7
Vallieres et al. [ 39 ]
Mrmre Random Forest
78.2 ± 7.2
67.2 ± 5.9
61.4 ± 12.4
Kazmierski et al. [ 22 ]
Fuzzy Logistic Mrmr
80.5 ± 7.9
63.6 ± 6.3
57.9 ± 15.1
AMO-ENE
Multi-Head Attention Classifier
88.2 ± 4.8
78.1 ± 8.6
79.2 ± 7.4
Table 6: Binary 2-year classification algorithms for distant metastasis (DM), disease-free survival (DFS) and overall survival (OS). Proportion of (uncensored) events at the end of the study are reported in parentheses. Results show the mean AUC ( ± standard deviation) over the 5 cross-validation folds. All presented algorithms incorporate the primary tumor, predicted nodal and clinical characteristics. For every parametric model, results presented are the best performance obtained via grid-search as proposed by the respective authors.
Model
Features
Algorithm
DM
DFS
OS
Clinical
Radiomics
Cox
✓
Cox Regression
66.0 ± 7.7
66.5 ± 6.3
64.4 ± 18.5
Rebaud et al. [ 34 ]
✓
✓
Bagged risk estimation
81.5 ± 8.1
68.8 ± 4.2
62.89 ± 8.1
Cox + XGBOOST iENE score
✓
✓
Cox Regression
80.1 ± 6.9
68.6 ± 5.1
70.2 ± 9.1
AMO-ENE (MHSA)
✓
✓
AMO-ENE + MTLR
83.3 ± 6.5
70.0 ± 8.1
71.3 ± 8.9
Table 7: Survival risk estimation for metastatic failure (DM), disease-free survival (DFS) and overall survival (OS). Figures are the mean concordance index (C-index) ( ± standard deviation) over the folds from the same 5-fold stratified cross-validation. For every parametric model, results presented are the best performance obtained via grid-search as proposed by the respective authors.
Fusion Type
Params (M)
Memory (MB)
Recall
Specificity
AUC
Early Fusion
1.95
15.4
60.3 ± 19.2
87.5 ± 3.5
76.4 ± 7.5
Late Concatenation
0.86
15.3
55.0 ± 28.6
86.6 ± 5.2
80.1 ± 7.1
Late Soft Attention
1.17
12.7
75.2 ± 23.2
88.4 ± 5.2
86.7 ± 6.6
AMO-ENE (MHSA)
2.37
15.4
75.4 ± 19.5
91.2 ± 6.2
88.2 ± 4.8
Table 8: Ablation study for the modality fusion module of AMO-ENE, tested on the two year DM classification task.
Model
Number of Heads
Memory (MB)
Recall
Specificity
AUC
AMO-ENE
2
15.62
71.2 ± 27.0
74.3 ± 21.5
75.4 ± 10.1
4
15.65
62.0 ± 12.9
83.2 ± 7.6
80.8 ± 7.0
8
15.68
75.4 ± 19.5
86.6 ± 6.5
88.2 ± 4.8
16
15.71
59.3 ± 27.6
87.2 ± 4.9
80.1 ± 13.4
Table 9: Ablation study for the number of heads used in our multi-head attention fusion module of AMO-ENE, tested on the two year DM classification task.
Head and neck cancers (HNC) represent a significant global health burden, with accurate tumor delineation being essential for effective radiotherapy planning. The complexity of the oropharyngeal anatomy, combined with the heterogeneous appearance of tumors on imaging, makes manual segmentation time-intensive and subject to inter-observer variability. Beyond segmentation, predicting long-term clinical outcomes, such as recurrence-free survival (RFS), and determining human papillomavirus (HPV) status from noninvasive imaging, remain challenging yet clinically valuable goals. The HECKTOR 2025 challenge addresses these needs by establishing a comprehensive benchmark for automated HNC analysis using multimodal PET/CT imaging and electronic health records. Building on previous editions (2020-2022), this challenge features an expanded multi-institutional dataset comprising over 1,100 patients from 10 centers worldwide. Participants were tasked with three complementary objectives: (1) segmenting primary gross tumor volumes (GTVp) and metastatic lymph nodes (GTVn), (2) predicting recurrence-free survival, and (3) classifying HPV status. The challenge attracted 35 registered teams, with 15 final submissions evaluated on a held-out test set. Top-performing algorithms achieved a mean Dice similarity coefficient of 0.75 for segmentation, a concordance index of 0.66 for survival prediction, and a balanced accuracy of 0.56 for HPV classification. This paper presents a comprehensive analysis of the submitted methodologies, evaluates their performance across different lesion characteristics, and discusses their implications for clinical translation in automated oncology workflows and decision support systems.
Numan Saeed, Salma Hassan, Shahad Hardan +27
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE. · Amsterdam UMC, Amsterdam, The Netherlands. · The Netherlands Cancer Institute, Amsterdam, The Netherlands. +22
Background: Early prediction of distant metastasis (DM) risk in head and neck cancer (HNC) can enable timely interventions that may improve treatment outcomes. Many current machine learning methods rely on prior knowledge of the region of interest such as tumor segmentations, which require expert knowledge, is time-consuming and introduces user-dependent variability. Medical image-based foundation models have recently been developed for specific imaging modalities to streamline down-stream prediction tasks by extracting modality-relevant features. Purpose: In this study, we evaluate the effectiveness of using a foundation model as the feature extractor to predict DM risk in HNC patients and compare its performance with traditional approaches that require prior knowledge on the regions of interest. Methods: Preoperative CT images of 2327 patients from the RADCURE dataset were used. Three features-sets were created including radiomics, deep-learning based features, and CT Foundation derived features. The feature-sets were used individually in a multi-layer perceptron (MLP) to predict DM risk. Results: The model using CT Foundation embeddings outperformed the radiomics and deep learning-based models, achieving a Receiver Operating Characteristic Area Under the Curve (AUC) of 0.791, compared to AUC values of 0.772 and 0.753 for the radiomics and deep learning-based models, respectively. The CT Foundation based model had similar performance to a model that combined the use of radiomics and deep learning-based features that achieved an AUC of 0.794. Conclusions: Features based on foundation models offer a promising alternative to traditional radiomics while reducing the need for domain expertise and extensively annotated datasets. Their minimal preprocessing requirements also make them a more accessible and scalable option.
Erich Schmitz, Meixu Chen, Bowen Jing +1
Advanced Imaging and Informatics for Radiation Therapy (AIRT) and Medical Artificial Intelligence and Automation (MAIA) Laboratory, Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, Texas
We present HERMES (Hybrid Ensemble for Radiotherapy-target segmentation, Malignancy staging, and Event-free Survival), a single containerized algorithm for the three HECKTOR 2026 subtasks: segmentation of the primary tumor (GTVp) and pathological lymph nodes (GTVn), radiological T/N staging, and recurrence-free survival (RFS), computed from a paired FDG-PET/CT scan and an electronic health record. A 10-fold ensemble of STU-Net Small networks produces the segmentation; the predicted mask then drives two downstream tasks. Rather than pass a generic radiomics vector to the staging models, we derive from the predicted masks a compact set of geometry features aligned with the size and number axes of AJCC/UICC 7th-edition radiological N/T staging. On internal cross-validation these features raise N-stage balanced accuracy from 0.691 to 0.720 (+0.030), our largest single design gain, at lower feature dimensionality. For prognosis we combine complementary deep and clinical risk experts in an equal-weight ensemble, and train one deep expert with a concordance-tracking survival loss of our own, whose value approximates the concordance index during training. Every component was selected on honest out-of-fold predictions under a regularization-oriented protocol, with no tuning on the public validation set, and deployed as two decorrelated submissions. On the HECKTOR 2026 validation leaderboard, HERMES achieved a weighted score of 0.6454 (Mean Dice 0.641, T balanced accuracy 0.580, N balanced accuracy 0.642, RFS C-index 0.679) and qualified for the testing phase. Team: AMC_HNC.
Kai Wang, Meixu Chen, Elie Nasr +2
Department of Radiation Oncology, University of Colorado School of Medicine, Aurora, CO, USA