AMO-ENE: Attention-based Multi-Omics Fusion Model for Outcome Prediction in Extra Nodal Extension and HPV-associated Oropharyngeal Cancer

Gautier Hénique; William Le; Gabriel Dayan; Coralie Brodeur; Kristoff Nelson; Apostolos Christopoulos; Edith Filion; Phuc-Felix Nguyen-Tan

eess.IVApr 10, 2026

AMO-ENE: Attention-based Multi-Omics Fusion Model for Outcome Prediction in Extra Nodal Extension and HPV-associated Oropharyngeal Cancer

Authors: Gautier Hénique, William Le, Gabriel Dayan, Coralie Brodeur, Kristoff Nelson, Apostolos Christopoulos, Edith Filion, Phuc-Felix Nguyen-Tan,

Organizations: MedICAL Laboratory, Polytechnique Montréeal, Montreal, Canada · Centre de recherche du CHUM (CRCHUM), Montréal, Canada · Centre Hospitalier de l’Université de Montréal (CHUM), Montreal Canada

Abstract

Extranodal extension (ENE) is an emerging prognostic factor in human papillomavirus (HPV)-associated oropharyngeal cancer (OPC), although it is currently omitted as a clinical staging criteria. Recent works have advocated for the inclusion of iENE as a prognostic marker in HPV-positive OPC staging. However, several practical limitations continue to hinder its clinical integration, including inconsistencies in segmentation, low contrast in the periphery of metastatic lymph nodes on CT imaging, and laborious manual annotations. To address these limitations, we propose a fully automated end-to-end pipeline that uses computed tomography (CT) images with clinical data to assess the status of nodal ENE and predict treatment outcomes. Our approach includes a hierarchical 3D semi-supervised segmentation model designed to detect and delineate relevant iENE from radiotherapy planning CT scans. From these segmentations, a set of radiomics and deep features are extracted to train an imaging-detected ENE grading classifier. The predicted ENE status is then evaluated for its prognostic value and compared with existing staging criteria. Furthermore, we integrate these nodal features with primary tumor characteristics in a multimodal, attention-based outcome prediction model, providing a dynamic framework for outcome prediction. Our method is validated in an internal cohort of 397 HPV-positive OPC patients treated with radiation therapy or chemoradiotherapy between 2009 and 2020. For outcome prediction at the 2-year mark, our pipeline surpassed baseline models with 88.2% (4.8) in AUC for metastatic recurrence, 79.2% (7.4) for overall survival, and 78.1% (8.6) for disease-free survival. We also obtain a concordance index of 83.3% (6.5) for metastatic recurrence, 71.3% (8.9) for overall survival, and 70.0% (8.1) for disease-free survival, making it feasible for clinical decision making.

Figures & tables

Figure 1: Illustration for iENE status grading system. Nodes are presented with white borders, red healthy tissue background and black tumoral content. Grade 0 indicates metastatic pathological node with no signs of extension. Grade 1 indicates extension through the nodal capsule into surrounding fat. Grade 2 characterizes coalescent nodal masses with loss of internodal planes. Grade 3 is defined by overt invasion of surrounding structures and tissues. In case of multiple ENE, the maximum grade is reported.

Figure 2: Overall schematic diagram of the proposed framework for CT-based iENE status and prognosis prediction. First, an automated segmentation and selection of pathological node allows for the retrieval of the largest metastatic lymph node. Afterwards, radiomics and deep features are extracted from the structure to classify iENE grades in a binary problem. Finally, from the obtained predictions, the survival characteristics of predicted iENE groups are compared to demonstrate the influence of extra nodal extensions in HPV-associated OPC.

Outcome	Event type	Male	Female	Total
OS	Uncensored	27	11	38
OS	Censored	290	69	359
DM	Uncensored	30	5	35
DM	Censored	287	75	362
DFS	Uncensored	52	16	68
DFS	Censored	265	64	329

Table 1: Outcome representation in the patient cohort for Uncensored (Event presence at the latest available follow up) and Censored patients, stratified for different outcome measures 9OS, DM, DFS).

Figure 3: Graphical representation of the proposed attention-based modality fusion pipeline. In (A), multiple modality-specific feature encoders are trained in parallel, to extract features from each specific modality. The latent representation of each modality is concatenated and a multi-head scaled dot product attention is performed to evaluate interactions and combine latent representations. A final decision layer is adaptable to either predict a time dependent outcome target or multi-bin risk modeling. Details are provided in Sec. 3.4 for relevant notations. In (B), the proposed modality encoder module is shown, which is composed of multiple blocks containing parametric layers. (C) Detailed description of the operation performed in the attention layer.

Figure 4: Qualitative results for the iENE segmentation task on radiotherapy planning CT images. Each row presents the raw CT scan and the output segmentation maps for two distinct patients from test folds over a 5-fold cross validation. Comparative methods are shown in their respective columns : proposed model, nnUnet [ 20 ] , SwinUNETRV2 [ 16 ] , and SamMed3D [ 42 ] . GT: ground truth mask.

Features	Method	iENE - vs iENE +	0-1 / 2-3	0-1-2 / 3
Radiomics	RF + PCA	78.12 $\pm$ 4.67	76.93 $\pm$ 5.98	89.87 $\pm$ 5.24
	Lasso + MLP	77.55 $\pm$ 2,65	74.98 $\pm$ 2.80	80.64 $\pm$ 17.31
	PCA + XGBOOST	76.39 $\pm$ 2.56	80.60 $\pm$ 4.52	86.60 $\pm$ 8.40
	Lasso + XGBOOST	79.92 $\pm$ 2.51	78.00 $\pm$ 1.61	88.85 $\pm$ 5.27
FMCIB	PCA + XGBOOST	70.33 $\pm$ 3.41	68.32 $\pm$ 5.51	80.52 $\pm$ 12.18
FMCIB	Lasso + XGBOOST	71.82 $\pm$ 2.32	64.98 $\pm$ 5.69	68.19 $\pm$ 10.36

Table 4: iENE grade classification results for 3 different dichotomization schemes (iENE - vs iENE + ; 0-1/2-3; 0-1-2/3). Results show the mean AUC (

\pm

standard deviation) over the 5 tests fold obtained in a stratified cross-validation. Features indicate the CT feature extraction strategy used as inputs to the classification method.

Figure 5: ROC curves obtained for the binary classification of dichotomized iENE classes, for 3 different dichotomization schemes (iENE - vs iENE + ; 0-1/2-3; 0-1-2/3). Individual patient logits were obtained in a 5-fold cross validation scheme with the L2 norm penalized XGBOOST classifier.

Figure 6: SHAP explanation analysis plots for the prediction of iENE- vs iENE + cases, with the top features obtained from the radiomics branch of the prediction framework.

Panel of Figure 7 — Figure 7: Kaplan Meier curves obtained by stratifying all 397 patients with the predicted iENE + score. (a) Survival curves from ground-truth pathology. (b) Survival curves from predicted iENE. Blue lines constitute the real or predicted iENE- groups, red lines show the real or predicted iENE+ population. The selected node for grade classification is obtained as the largest predicted volume. The Kaplan Meyer fit function is unimodal and takes as input the tresholded (from AUC) binary prediction value. Each curve displays as legend the p-value obtained in a logrank test (DM: $p=9.6\times 10^{-10}$ , DFS: $p=9.83\times 10^{-5}$ ; OS: $p=2.8\times 10^{-3}$ ).

Outcome	$\overline{C}$	$C_{1}$	$C_{2}$	$C_{3}$	SR	Proposed
DM	$2.34\times 10^{-2}$	$1.76\times 10^{-2}$	$3.13\times 10^{-2}$	$1.61\times 10^{-1}$	$7.78\times 10^{-2}$	$8.02\times 10^{-5}$
DFS	$1.98\times 10^{-1}$	$2.16\times 10^{-1}$	$3.03\times 10^{-2}$	$3.39\times 10^{-1}$	$1.93\times 10^{-1}$	$3.30\times 10^{-2}$
OS	$2.62\times 10^{-1}$	$1.82\times 10^{-1}$	$1.86\times 10^{-1}$	$3.32\times 10^{-1}$	$2.82\times 10^{-1}$	$2.90\times 10^{-2}$

Table 5: Log-rank statistical test for a separate set of 55 patients annotated by three separate radiologists.

\overline{C}

: A collegial decision involved a three way consensus over the case definition, and served as the ground truth for model training.

C_{1-3}

refers to clinician 1, 2 or 3 respectively. SR: a single reader is simulated by randomly selecting one of the three annotations per patient, and bootstrapping the obtained results 10 000 times, giving the average score a unique reader could obtain.

Figure 8: Modality ablation experiment results for the 2-year therapeutic outcome prediction model. For readability, C indicates the clinical variables (including the TNM staging), P represents the primary tumor value (GTV radiomic characteristics) and N the predicted nodal volume (iENE) radiomics. Each result presents the mean AUC over the 5 tests fold obtained in a stratified k-fold validation. For each modality and combination of modality, we present the best mean AUC obtained over a grid search as described in 3.5 , as well as the standard deviation plotted in red dashed lines.

Model	Algorithm	DM 2 Y (6 $\%$ )	DFS 2Y (12 $\%$ )	OS 2Y (5 $\%$ )
Cox	Proportional Hazard Regression	69.7 $\pm$ 10.7	67.0 $\pm$ 8.5	63.4 $\pm$ 17.9
Kazmierski et al. [ 22 ]	Logistic Regression	68.2 $\pm$ 6.4	59.4 $\pm$ 17.9	57.1 $\pm$ 11.6
Rebaud et al. [ 34 ]	Bagged risk estimation	70.7 $\pm$ 5.0	62.9 $\pm$ 3.2	60.9 $\pm$ 12.7
Vallieres et al. [ 39 ]	Mrmre Random Forest	78.2 $\pm$ 7.2	67.2 $\pm$ 5.9	61.4 $\pm$ 12.4
Kazmierski et al. [ 22 ]	Fuzzy Logistic Mrmr	80.5 $\pm$ 7.9	63.6 $\pm$ 6.3	57.9 $\pm$ 15.1
AMO-ENE	Multi-Head Attention Classifier	88.2 $\pm$ 4.8	78.1 $\pm$ 8.6	79.2 $\pm$ 7.4

Table 6: Binary 2-year classification algorithms for distant metastasis (DM), disease-free survival (DFS) and overall survival (OS). Proportion of (uncensored) events at the end of the study are reported in parentheses. Results show the mean AUC (

\pm

standard deviation) over the 5 cross-validation folds. All presented algorithms incorporate the primary tumor, predicted nodal and clinical characteristics. For every parametric model, results presented are the best performance obtained via grid-search as proposed by the respective authors.

Model	Features		Algorithm	DM	DFS	OS
	Clinical	Radiomics
Cox	$\checkmark$		Cox Regression	66.0 $\pm$ 7.7	66.5 $\pm$ 6.3	64.4 $\pm$ 18.5
Rebaud et al. [ 34 ]	$\checkmark$	$\checkmark$	Bagged risk estimation	81.5 $\pm$ 8.1	68.8 $\pm$ 4.2	62.89 $\pm$ 8.1
Cox + XGBOOST iENE score	$\checkmark$	$\checkmark$	Cox Regression	80.1 $\pm$ 6.9	68.6 $\pm$ 5.1	70.2 $\pm$ 9.1
AMO-ENE (MHSA)	$\checkmark$	$\checkmark$	AMO-ENE + MTLR	83.3 $\pm$ 6.5	70.0 $\pm$ 8.1	71.3 $\pm$ 8.9

Table 7: Survival risk estimation for metastatic failure (DM), disease-free survival (DFS) and overall survival (OS). Figures are the mean concordance index (C-index) (

\pm

standard deviation) over the folds from the same 5-fold stratified cross-validation. For every parametric model, results presented are the best performance obtained via grid-search as proposed by the respective authors.

Fusion Type	Params (M)	Memory (MB)	Recall	Specificity	AUC
Early Fusion	1.95	15.4	60.3 $\pm$ 19.2	87.5 $\pm$ 3.5	76.4 $\pm$ 7.5
Late Concatenation	0.86	15.3	55.0 $\pm$ 28.6	86.6 $\pm$ 5.2	80.1 $\pm$ 7.1
Late Soft Attention	1.17	12.7	75.2 $\pm$ 23.2	88.4 $\pm$ 5.2	86.7 $\pm$ 6.6
AMO-ENE (MHSA)	2.37	15.4	75.4 $\pm$ 19.5	91.2 $\pm$ 6.2	88.2 $\pm$ 4.8

Table 8: Ablation study for the modality fusion module of AMO-ENE, tested on the two year DM classification task.

Model	Number of Heads	Memory (MB)	Recall	Specificity	AUC
AMO-ENE	2	15.62	71.2 $\pm$ 27.0	74.3 $\pm$ 21.5	75.4 $\pm$ 10.1
	4	15.65	62.0 $\pm$ 12.9	83.2 $\pm$ 7.6	80.8 $\pm$ 7.0
	8	15.68	75.4 $\pm$ 19.5	86.6 $\pm$ 6.5	88.2 $\pm$ 4.8
	16	15.71	59.3 $\pm$ 27.6	87.2 $\pm$ 4.9	80.1 $\pm$ 13.4

Table 9: Ablation study for the number of heads used in our multi-head attention fusion module of AMO-ENE, tested on the two year DM classification task.

Clinical Outcome Prediction

Clinical outcome prediction uses patient records, imaging, signals, and biomarkers to estimate risks, disease trajectories, treatment response, and survival.

See all 265 papers

HEad and neCK TumOR (HECKTOR) 2025: Benchmark of Segmentation, Diagnosis, and Prognosis in Multimodal PET/CT
Jun 18, 2026Numan Saeed, Salma Hassan, Shahad Hardan +27Cancer Survival Prediction PET/CT Imaging
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE. · Amsterdam UMC, Amsterdam, The Netherlands. · The Netherlands Cancer Institute, Amsterdam, The Netherlands. +22
Comparing the Performance of Foundation Model Derived Embeddings with Traditional Approaches for Distant Metastasis Prediction in Head and Neck Cancer
Jul 28, 2026Erich Schmitz, Meixu Chen, Bowen Jing +1Medical Imaging Foundation Models Clinical Outcome Prediction
Advanced Imaging and Informatics for Radiation Therapy (AIRT) and Medical Artificial Intelligence and Automation (MAIA) Laboratory, Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, Texas
HERMES: A Hybrid Ensemble for Head-and-Neck Tumor Segmentation, TN Staging, and Recurrence-Free Survival on PET/CT
Jul 29, 2026Kai Wang, Meixu Chen, Elie Nasr +2Cancer Survival Prediction PET/CT Imaging
Department of Radiation Oncology, University of Colorado School of Medicine, Aurora, CO, USA

Jun 18, 2026J/K move · Enter open · S save

HEad and neCK TumOR (HECKTOR) 2025: Benchmark of Segmentation, Diagnosis, and Prognosis in Multimodal PET/CT

Numan Saeed, Salma Hassan, Shahad Hardan, Lishan Cai+26

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE. · Amsterdam UMC, Amsterdam, The Netherlands. · The Netherlands Cancer Institute, Amsterdam, The Netherlands.+22

Cancer Survival Prediction PET/CT Imaging Medical Image Segmentation Oncology Medical Image Benchmarks

Open paper AskPDF arXiv

Head and neck cancers (HNC) represent a significant global health burden, with accurate tumor delineation being essential for effective radiotherapy planning. The complexity of the oropharyngeal anatomy, combined with the heterogeneous appearance of tumors on imaging, makes manual segmentation time-intensive and subject to inter-observer variability. Beyond segmentation, predicting long-term clinical outcomes, such as recurrence-free survival (RFS), and determining human papillomavirus (HPV) status from noninvasive imaging, remain challenging yet clinically valuable goals. The HECKTOR 2025 challenge addresses these needs by establishing a comprehensive benchmark for automated HNC analysis using multimodal PET/CT imaging and electronic health records. Building on previous editions (2020-2022), this challenge features an expanded multi-institutional dataset comprising over 1,100 patients from 10 centers worldwide. Participants were tasked with three complementary objectives: (1) segmenting primary gross tumor volumes (GTVp) and metastatic lymph nodes (GTVn), (2) predicting recurrence-free survival, and (3) classifying HPV status. The challenge attracted 35 registered teams, with 15 final submissions evaluated on a held-out test set. Top-performing algorithms achieved a mean Dice similarity coefficient of 0.75 for segmentation, a concordance index of 0.66 for survival prediction, and a balanced accuracy of 0.56 for HPV classification. This paper presents a comprehensive analysis of the submitted methodologies, evaluates their performance across different lesion characteristics, and discusses their implications for clinical translation in automated oncology workflows and decision support systems.

Abstract

Clinical Outcome Prediction

Explore similar work

HEad and neCK TumOR (HECKTOR) 2025: Benchmark of Segmentation, Diagnosis, and Prognosis in Multimodal PET/CT

Comparing the Performance of Foundation Model Derived Embeddings with Traditional Approaches for Distant Metastasis Prediction in Head and Neck Cancer

HERMES: A Hybrid Ensemble for Head-and-Neck Tumor Segmentation, TN Staging, and Recurrence-Free Survival on PET/CT