Morphology-aware deep learning predicts colorectal cancer survival from whole-slide images

Hikmat Khan; Metin N. Gurcan; Anil Parwani; Wendy L. Frankel; Wei Chen; Muhammad Khalid Khan Niazi

cs.CVOct 13, 2026

Morphology-aware deep learning predicts colorectal cancer survival from whole-slide images

Authors: Hikmat Khan, Metin N. Gurcan, Anil Parwani, Wendy L. Frankel, Wei Chen, Muhammad Khalid Khan Niazi

Organizations: Department of Pathology, The Ohio State University Wexner Medical Center, Columbus, OH, USA. · Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA.

Abstract

Background: Five-year survival prediction from hematoxylin-and-eosin whole-slide images could support risk stratification, but the task is weakly supervised, and the tissue morphologies carrying prognostic information remain incompletely characterized. Methods: We conducted a retrospective secondary analysis using TCGA-COAD/READ (n=562) for morphology-specific analysis, Alliance/ CALGB 89803 (n=424) for model development, and SurGen (n=218) for external evaluation. We propose MorphContrast, a morphology-aware contrastive multiple-instance learning framework that uses tissue labels as structural supervision for five-year survival prediction. MorphContrast combines gated-attention pooling with three auxiliary objectives: patch-level morphology labels define within-patient contrasts, outcome concordance or discordance defines cross-patient contrasts within matched morphologies, and diversity regularization discourages representational collapse. Morphology-matched patches from patients with discordant outcomes serve as hard negatives, encouraging prognostic discrimination within comparable histologic contexts. Results: Morphology-specific models show weak but distributed prognostic discrimination: adipose tissue achieves an area under the receiver operating characteristic curve (AUC) of 0.622, stroma achieves 0.604, and both perform comparably to tumor epithelium (0.601-0.605). On Alliance/CALGB 89803, the single model achieves an AUC of 0.642, while the post hoc ensemble reaches a pooled out-of-fold AUC of 0.70 and a risk-group hazard ratio of 3.79 (95% confidence interval: 2.55-5.63). On the external SurGen colon cancer cohort (n=218), the single model and ensemble achieve AUCs of 0.669 and 0.678, respectively. Conclusions: These findings support tissue morphology as structural supervision for outcome-aware representation learning in colorectal cancer prognosis.

Figures & tables

Figure 1: Top-performing CRC tissue morphologies show modest prognostic discrimination. Each bar gives the five-fold cross-validated AUC of an attention-based multiple-instance learning (ABMIL) survival model trained on patches of a single morphology class in the TCGA-COAD/READ cohort (

n{=}562

; Table 1 ); the dashed line marks chance (

0.50

). Only the top-performing morphologies with AUC values above

0.50

are displayed. Non-tumor morphologies—adipose tissue, stroma, normal epithelium, and muscle—score comparably to tumor-epithelial classes, while the displayed morphology-specific models remain modest predictors. Because patch abundance, spatial context, and clinicopathologic correlates are uncontrolled, this analysis motivates the design of MorphContrast rather than supporting a biological interpretation.

Figure 2: Overview of MorphContrast. (a) A frozen MorphDistill encoder Khan et al. (2026b) assigns every patch one of $18$ CRC morphology classes, shown as a color overlay on a whole-slide image; this step is not trained here. (b) Tissue is segmented and tiled, and the same frozen encoder yields a feature vector and a morphology label $m_{bi}\in\{0,\ldots,17\}$ for each patch. A shared feature encoder then supplies two pathways. Along the predictive path (blue arrows), gated attention weights the patches and a fully connected network maps the pooled representation $\bar{h}_{b}$ (labeled “Slide Features”) to the five-year survival prediction under $\mathcal{L}_{\text{surv}}$ . Along the contrastive path (green arrows), a projection head is trained with intra-patient morphology contrast ( $\mathcal{L}_{\text{intra}}$ ), outcome-aware inter-patient contrast ( $\mathcal{L}_{\text{inter}}$ ), and diversity regularization ( $\mathcal{L}_{\text{div}}$ ); the figure writes these as $\mathcal{L}_{intra\text{-}patient}$ , $\mathcal{L}_{inter\text{-}patient}$ , and $\mathcal{L}_{diversity}$ . Contrastive pairs are drawn from the class-balanced patient pool (Section 3.3 ), from which one outcome-concordant and one outcome-discordant patient bag are sampled per anchor; losses from both pathways are accumulated across the three bags before a single backward pass. Snowflakes mark frozen components. Only the feature encoder, attention pooler, and survival classifier are retained at inference.

Morphology	AUC	Sen	Spec
Fat (adipose)	$\mathbf{0.622}$	$0.573$	$0.609$
Adenocarcinoma high-grade	$\underline{0.605}$	$0.698$	$0.447$
Adenoma high-grade dysp.	$\underline{0.605}$	$0.684$	$0.387$
Stroma	$0.604$	$0.581$	$0.528$
Adenocarcinoma low-grade	$0.601$	$0.637$	$0.491$
Normal epithelium	$0.596$	$0.608$	$0.541$

Table 1: Morphology-specific five-year survival prediction. Five-fold cross-validated ABMIL models trained independently using patches from a single morphology class in TCGA-COAD/READ.

Model	AUC $\uparrow$	Bal-Acc $\uparrow$	Sen $\uparrow$	Spec $\uparrow$
MorphContrast	${\underline{0.642}\pm 0.05}$	${0.594\pm 0.036}$	${0.635\pm 0.092}$	${0.554\pm 0.076}$
MorphContrast (ensemble)	$\mathbf{0.70}$ $(0.65,0.76)$	$0.68$ $(0.63,0.73)$	$0.68$ $(0.59,0.73)$	$0.68$ $(0.62,0.73)$

Table 2: Performance of the MorphContrast model and post hoc ensemble. Single-model values are five-fold cross-validation means

\pm

standard deviations. Ensemble values are computed from pooled OOF predictions, with

1{,}000

-sample bootstrap

95\%

confidence intervals.

Model	AUC $\uparrow$	Bal-Acc (%) $\uparrow$	Sen (%) $\uparrow$	Spec (%) $\uparrow$	C-index $\uparrow$
ABMIL Ilse et al. (2018)	$0.63\pm 0.06$	$58.54\pm 4.50$	$51.18\pm 9.83$	$65.89\pm 4.50$	$0.607$
CLAM Lu et al. (2021)	$0.66\pm 0.06$	$55.15\pm 3.37$	$25.61\pm 8.18$	$84.68\pm 8.89$	$0.603$
TransMIL Shao et al. (2021)	$0.67\pm 0.05$	$52.50\pm 4.64$	$5.25\pm 9.11$	$99.74\pm 0.05$	$0.647$
Nakanishi Nakanishi et al. (2023)	$0.61\pm 0.06$	$58.31\pm 5.77$	$53.49\pm 11.11$	$63.13\pm 2.75$	$0.604$
RRT-MIL Tang et al. (2024)	$0.59\pm 0.05$	$56.76\pm 4.46$	$48.58\pm 23.92$	$64.94\pm 22.19$	$0.575$
MorphDistill Khan et al. (2026b)	$\underline{0.68}\pm 0.08$	$63.41\pm 4.70$	$60.24\pm 5.38$	$66.57\pm 8.81$	$0.661$

Table 3: Comparison with MIL, morphology-aware, and task-specific CRC prognosis baselines. Baselines and the single MorphContrast model report mean

\pm

standard deviation over folds; the ensemble reports pooled OOF performance.

Encoder	AUC $\uparrow$	Bal-Acc (%) $\uparrow$	Sen (%) $\uparrow$	Spec (%) $\uparrow$
CONCH v1.5 Lu et al. (2024)	$0.51\pm 0.07$	$50.81\pm 3.83$	$41.80\pm 15.76$	$59.81\pm 16.66$
CTransPath Wang et al. (2022)	$0.58\pm 0.02$	$50.63\pm 1.97$	$21.90\pm 39.22$	$79.36\pm 37.38$
Lunit ViT-S/8 Kang et al. (2023)	$0.58\pm 0.04$	$53.22\pm 3.14$	$54.24\pm 15.52$	$52.21\pm 12.69$
Phikon v2 Filiot et al. (2024)	$0.58\pm 0.04$	$58.05\pm 5.67$	$59.50\pm 15.38$	$56.61\pm 11.42$
Virchow2 Zimmermann et al. (2024)	$0.58\pm 0.02$	$55.73\pm 3.84$	$52.71\pm 12.97$	$58.75\pm 19.85$
Kaiko ViT-B/8 kaiko.ai et al. (2024)	$0.59\pm 0.11$	$58.38\pm 8.91$	$46.45\pm 30.02$	$70.31\pm 13.85$

Table 4: Frozen pathology foundation encoders under an identical survival head. Each encoder’s patch features are aggregated using the same gated-attention survival head. Values are mean

\pm

standard deviation over five folds; the ensemble reports pooled OOF performance.

Model	Fold AUC $\uparrow$	Pooled AUC [95% CI] $\uparrow$	C-index $\uparrow$
All fourteen variables	$0.548\pm 0.066$	$0.541$ $(0.474,0.608)$	$0.556$
Nodal burden, T category, grade	$0.554\pm 0.052$	$0.550$ $(0.487,0.613)$	$0.553$
MorphContrast	$0.642\pm 0.050$	—	$0.616$
MorphContrast (ensemble)	—	$0.70$ $(0.65,0.76)$	$0.665$

Table 5: Clinical-variable models and MorphContrast on Alliance/CALGB 89803 (

n=424

;

103

events). Fold AUC reports the mean

\pm

standard deviation across five test folds. Pooled AUC uses out-of-fold predictions, with DeLong

95\%

CIs for the clinical models and a

1{,}000

-sample bootstrap

95\%

CI for the ensemble. Variable coding and imputation are detailed in Supplementary Table 3.

Figure 3: Kaplan–Meier risk stratification for MorphContrast and comparison models. Patients are assigned to low- and high-risk groups at each model’s operating threshold; group sizes are shown in each panel. Hazard ratios and

95\%

CIs are obtained from univariate Cox models, and

p

-values from log-rank tests.

Figure 4: Discrimination and risk-group separation across MIL and morphology-aware models. Left: C-index. Right: univariate Cox hazard ratios with

95\%

CIs for high- versus low-risk groups.

Model	AUC $\uparrow$	Bal-Acc (%) $\uparrow$	Sen. (%) $\uparrow$	Spec. (%) $\uparrow$
TransMIL Shao et al. (2021)	$0.558$ $(0.47,0.64)$	$55.06$ $(45.1,57.7)$	$65.15$ $(52.2,76.4)$	$44.97$ $(37.6,53.0)$
FR-MIL Chikontwe et al. (2022)	$\underline{0.676}$ $(0.59,0.75)$	$58.80$ $(46.5,60.0)$	$83.33$ $(73.3,92.0)$	$40.27$ $(32.6,48.6)$
MHIM-MIL Tang et al. (2023)	$0.558$ $(0.48,0.64)$	$53.61$ $(47.4,61.4)$	$51.52$ $(40.0,63.2)$	$55.70$ $(47.9,64.5)$
MorphContrast	$0.669$ $(0.58,0.75)$	$63.98$ $(60.1,72.3)$	$57.58$ $(30.3,85.1)$	$70.39$ $(44.0,96.2)$
MorphContrast (ensemble)	$\mathbf{0.678}$ $(0.60,0.76)$	$65.80$ $(60.7,73.4)$	$65.15$ $(43.9,85.5)$	$66.45$ $(45.4,87.4)$

Table 6: External validation performance for five-year overall survival prediction in the SurGen colon cancer cohort. Models were trained using the Alliance cohort and externally evaluated on the independent SurGen cohort. The best AUC point estimate is shown in bold, and the second-best AUC point estimate is underlined.

	Loss terms		Five-year survival classification
Configuration	$\mathcal{L}_{\text{intra}}$	$\mathcal{L}_{\text{inter}}$	AUC	Bal. Acc. (%)	Sens. (%)	Spec. (%)
Survival only	✗	✗	$0.60\pm 0.07$	$52.43\pm 1.32$	$82.12\pm 7.06$	$22.71\pm 7.01$
$+\,\mathcal{L}_{\text{intra}}$	✓	✗	$\underline{0.62}\pm 0.09$	$53.62\pm 2.42$	$72.01\pm 1.75$	$35.04\pm 1.09$
$+\,\mathcal{L}_{\text{inter}}$	✗	✓	$0.61\pm 0.08$	$56.62\pm 5.20$	$80.53\pm 18.41$	$32.70\pm 19.02$
Full	✓	✓	$\mathbf{0.64}\pm 0.05$	$59.43\pm 3.61$	$63.52\pm 9.23$	$55.40\pm 7.63$

Table 7: Ablation of the intra- and inter-patient contrastive losses (Alliance/CALGB 89803). Each configuration adds the indicated contrastive term to the shared survival (

\mathcal{L}_{\text{surv}}

) and diversity (

\mathcal{L}_{\text{div}}

) objectives under identical five-fold splits; ✓/✗ mark inclusion. “Survival only” omits both contrastive terms; “Full” is complete MorphContrast. Weights are fixed at the established values (

\lambda_{\text{surv}}{=}1.0

\lambda_{\text{div}}{=}0.1

;

\lambda_{\text{intra}}{=}0.25

\lambda_{\text{inter}}{=}0.1

when enabled). Values are mean

\pm

standard deviation across folds; in the AUC column the best value is shown in bold, and the second-best is underlined. AUC is highest for the full objective, whereas either contrastive term alone is within noise of the survival-only baseline.

Supplementary Fig. 1: Kaplan–Meier risk stratification using frozen pathology foundation encoders and MorphContrast. Each panel reports low- and high-risk group sizes, the hazard ratio with

95\%

CI, and the log-rank

p

-value. The foundation-model features are aggregated using the same gated-attention survival head.

Alliance cohort characteristics
Characteristic	Category	Value
Number of slides		424
Number of patients		424
Mean age (years)		60.47
Mean household income median (USD)		43,194.59
Race	White	391

Supplementary Table 1: Clinicopathologic characteristics of the Alliance/CALGB 89803 cohort.

Parameter	Value(s)	Setting
Hyperparameter search space
Feature-encoder hidden dimension	$\{768,1024\}$	Searched
Projection dimension	$\{128,256\}$	Searched
Attention hidden dimension	$\{512,1024,1536\}$	Searched
Initial learning rate	$\{10^{-4},10^{-5}\}$	Searched
Decision threshold	$\{0.4,0.5\}$	Searched

Supplementary Table 2: Hyperparameter search space and fixed training settings for MorphContrast. Parameters with multiple candidate values were varied during hyperparameter search; single-valued parameters were fixed.

Variable	Model coding
Age at enrolment	years, continuous
Sex	male vs female
Histological grade	III (poorly differentiated) vs I–II
T category (depth of invasion)	T4 vs T1–T3
Positive lymph nodes	number, continuous
Small blood/lymphatic vessel invasion	present vs absent

Supplementary Table 3: Clinicopathological variables used in the clinical models. The fourteen variables were obtained from Alliance/CALGB 89803 trial case-report forms (

n=424

;

103

events). Both full and reduced baselines use

\ell_{2}

-regularized logistic regression to predict death within five years, evaluated using the same five patient-disjoint folds as MorphContrast. Missing values were imputed with medians estimated from the training patients in each fold. The reduced model includes positive lymph-node count, T category, and grade only. T category combines trial depth-of-invasion codes

4

and

5

as T4. FL denotes fluorouracil and leucovorin; IFL additionally includes irinotecan. No molecular markers were included.

Explore similar work

Apr 7, 2026

MorphDistill: Distilling Unified Morphological Knowledge from Pathology Foundation Models for Colorectal Cancer Survival Prediction

Background: Colorectal cancer (CRC) remains a leading cause of cancer-related mortality worldwide. Accurate survival prediction is essential for treatment stratification, yet existing pathology foundation models often overlook organ-specific features critical for CRC prognostication. Methods: We propose MorphDistill, a two-stage framework that distills complementary knowledge from multiple pathology foundation models into a compact CRC-specific encoder. In Stage I, a student encoder is trained using dimension-agnostic multi-teacher relational distillation with supervised contrastive regularization on large-scale colorectal datasets. This preserves inter-sample relationships from ten foundation models without explicit feature alignment. In Stage II, the encoder extracts patch-level features from whole-slide images, which are aggregated via attention-based multiple instance learning to predict five-year survival. Results: On the Alliance/CALGB 89803 cohort (n=424, stage III CRC), MorphDistill achieves an AUC of 0.68 (SD 0.08), an approximately 8% relative improvement over the strongest baseline (AUC 0.63). It also attains a C-index of 0.661 and a hazard ratio of 2.52 (95% CI: 1.73-3.65), outperforming all baselines. On an external TCGA cohort (n=562), it achieves a C-index of 0.628, demonstrating strong generalization across datasets and robustness across clinical subgroups. Conclusion: MorphDistill enables task-specific representation learning by integrating knowledge from multiple foundation models into a unified encoder. This approach provides an efficient strategy for prognostic modeling in computational pathology, with potential for broader oncology applications. Further validation across additional cohorts and disease stages is warranted.

Hikmat Khan, Usama Sajjad, Metin N. Gurcan +4

Department of Pathology, College of Medicine, The Ohio State University Wexner Medical Center, Columbus, OH 43210, USA · Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC 27101, USA

Cancer Survival Prediction Computational Pathology Medical Image Representation Learning Pathology Foundation Models

Ask PDF

Sep 18, 2026

MIST: Multimodal Survival Prediction with Genomic-Guided Histology Attention

Multimodal survival models can combine complementary prognostic information from whole-slide images and genomic profiles, but effective fusion remains challenging amid external cohort shift and computational complexity. To address these challenges, we propose MIST, multimodal survival prediction with genomic-guided histology attention. MIST represents genomic features as tokens and allows them to query compact foundation-model-derived histology context tokens before survival prediction. This design enriches molecular information with histology context rather than merging separately encoded modalities only at the final stage. Training combines discrete-time survival prediction with genomic feature masking, WSI dropout, and paired WSI-genomics contrastive alignment. Across four external evaluations in colon, renal, lung, and glioblastoma cohorts, MIST improves external C-index over standard fusion baselines in the primary comparisons. These results support genomic-guided histology attention as a compact and effective strategy for multimodal oncology outcome prediction. Our code is available at https://github.com/samiyavuuz/MIST .

Muhammet Sami Yavuz, Sabri Mustafa Kahya, Richard R. Chen +2

AI for Image-Guided Diagnosis and Therapy, School of Medicine and Health, Technical University of Munich (TUM), Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · Technical University of Munich (TUM), Munich, Germany +2

Cancer Survival Prediction Multimodal Survival Analysis Multimodal Fusion

Ask PDF

May 20, 2026

ProtoPathway: Biologically Structured Prototype-Pathway Fusion for Multimodal Cancer Survival Prediction

We introduce ProtoPathway, an interpretable-by-design multimodal framework for cancer survival prediction that unifies whole slide imaging and transcriptomics through encoders producing biologically grounded representations on both sides of the fusion. On the histopathology side,

K

learnable morphological prototypes, trained end-to-end with the survival objective, serve as the slide representation itself: patches flow into prototype tokens via soft assignment, compressing variable-length patch sets into fixed task-adaptive tokens. On the genomic side, a bipartite graph neural network encodes gene expression within the Reactome pathway hierarchy, producing pathway embeddings that reflect both constituent genes and their broader biological context through bidirectional message passing over a shared gene--pathway graph. Cross-modal attention then operates over a compact prototype

\times

pathway matrix in which prototypes query pathways, modeling the biological direction in which molecular programs give rise to tissue morphology. Because both axes carry stable task-learned identity, the attention matrix is itself an interpretability output, yielding native inference-time attribution across the full biological hierarchy, from genes through pathways and prototypes to spatial tissue maps. We evaluate on five TCGA cancer cohorts, demonstrating competitive or superior survival prediction with substantially improved biological interpretability and reduced computational cost, with interpretability claims validated through fold-stratified rank-based population-level analysis. Our source code, model weights, and Reactome pathways, together with a unified codebase reimplementing all multimodal survival baselines under identical preprocessing and evaluation, are available at: https://github.com/AmayaGS/ProtoPathway.

Amaya Gallagher-Syed, Costantino Pitzalis, Myles J. Lewis +2

Queen Mary University of London

Cancer Survival Prediction Oncology Multimodal Survival Analysis Explainable Medical Image Analysis Multimodal Fusion Whole-Slide Image Analysis

Ask PDF