An Automated Radiomics Framework for Postoperative Survival Prediction in Colorectal Liver Metastases using Preoperative MRI
Authors: Muhammad Alberb, Jianan Chen, Hossam El-rewaidy, Paul Karanicolas, Arun Seth, Yutaka Amemiya, Anne Martel, Helen Cheung
Organizations: Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada · Physical Sciences Platform, Sunnybrook Research Institute, Toronto, ON, Canada · UCL Cancer Institute, University College London, London, UK · Systems and Biomedical Engineering Department, Cairo University, Giza, Egypt · Department of Surgery, University of Toronto, Toronto, ON, Canada · Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada · Sunnybrook Health Sciences Centre, Toronto, ON, Canada · Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
While colorectal liver metastasis (CRLM) is potentially curable via hepatectomy, patient outcomes remain highly heterogeneous. Postoperative survival prediction is necessary to avoid non-beneficial surgeries and guide personalized therapy. In this study, we present an automated AI-based framework for postoperative CRLM survival prediction using pre- and post-contrast MRI. We performed a retrospective study of 227 CRLM patients who had gadoxetate-enhanced MRI prior to curative-intent hepatectomy between 2013 and 2020. We developed a survival prediction framework comprising an anatomy-aware segmentation pipeline followed by a radiomics pipeline. The segmentation pipeline learns liver, CRLMs, and spleen segmentation from partially-annotated data, leveraging promptable foundation models to generate pseudo-labels. To support this pipeline, we propose SAMONAI, a prompt propagation algorithm that extends Segment Anything Model to 3D point-based segmentation. Predicted pre- and post-contrast segmentations are then fed into our radiomics pipeline, which extracts per-tumor features and predicts survival using SurvAMINN, an autoencoder-based multiple instance neural network for time-to-event survival prediction. SurvAMINN jointly learns dimensionality reduction and survival prediction from right-censored data, emphasizing high-risk metastases. We compared our framework against established methods and biomarkers using univariate and multivariate Cox regression. Our segmentation pipeline achieves median Dice scores of 0.96 (liver) and 0.93 (spleen), driving a CRLM segmentation Dice score of 0.78 and a detection F1-score of 0.79. Accurate segmentation enables our radiomics pipeline to achieve a survival prediction C-index of 0.69. Our results show the potential of integrating segmentation algorithms with radiomics-based survival analysis to deliver accurate and automated CRLM outcome prediction.
Figures & tables
Figure 1: Dataset processing: (a) cohort definitions and exclusion criteria, (b) patient-level data splitting for training, validation, and testing. Patients with manual liver and spleen annotations were held out from segmentation model development, while patients with available genomics data were held out from both segmentation and survival prediction models development to be exclusively used for testing.
Figure 2: Segmentation pipeline: a promptable foundation model is used to complete partially-annotated masks for a small subset. A few-shot UNETR model is trained on the subset then used to complete the annotations for all cases selected for segmentation model development. Finally, A supervised UNETR model is retrained on the entire training dataset with completed labels.
Figure 3: Overview of SAMONAI applied to the liver: 3D objects are segmented from single points by propagating prompts from one view to another. Positive and negative points are displayed in green and red, respectively.
Figure 4: Radiomics pipeline: per-tumor radiomics are extracted. An autoencoder compresses features and a multiple instance regressor connected to the autoencoder’s bottleneck predicts per-tumor risk scores and pools them into patient hazards. The autoencoder and regressor are jointly trained using a weighted sum of mean squared error LMSE and cox proportional hazard loss LCoxPH .
Figure 5: Segmentation box plots. MedSAM, in blue, and SAMONAI, in orange, denote prompt-based segmentation performance on missing labels. Pipeline (MedSAM), in green, and Pipeline (SAMONAI), in red, denote final automated segmentation performance after integrating each promptable model into our segmentation pipeline. Black dots inside boxes represent mean Dice scores over patients, and gray dots outside boxes represent individual patients beyond the interquartile range.
Contrast
Precision
Recall
F1-score
TP
FP
FN
Pre-contrast
0.674
0.756
0.713
31
15
10
Post-contrast
0.720
0.878
0.791
36
14
5
Table 1: Tumor detection results of the SAMONAI-based segmentation pipeline
Figure 6: Survival prediction box plot showing C-index when using pre-contrast (pre), post-contrast (post), and combined contrast phases (both). Black dots inside boxes represent mean C-index over runs, and gray dots outside boxes represent individual runs beyond the interquartile range. Solid boxes correspond to ground-truth manual segmentations and striped boxes correspond to predicted segmentations. Statistical significance, assessed by Wilcoxon rank-sum test’s p-value, confirms that combining pre- and post-contrast MRI yields better performance than using either phase individually.
Pooling
C-Index
HR (95% CI)
p
mean
0.556
1.114 (1.040–1.216)
0.759
largest
0.594
1.263 (1.147–1.391)
0.618
max
0.623
1.568 (1.441–1.784)
0.162
LSE
0.691
1.929 (1.890–1.980)
< 0.001
Table 2: Comparison of pooling strategies
Model
Reduction
C-Index
HR (95% CI)
p
SVM
None
0.659
1.781 (1.693–1.876)
0.002
SVM
KBest
0.647
1.745 (1.652–1.842)
0.005
SVM
PCA
0.667
1.907 (1.793–2.036)
0.004
SVM
mRMR
0.634
1.568 (1.494–1.652)
0.011
RSF
None
0.632
1.447 (1.368–1.531)
0.072
RSF
KBest
0.637
1.543 (1.453–1.653)
0.056
Table 3: Comparison of SurvAMINN against common machine learning methods
Univariate
Multivariate
Biomarker
C-Index
HR (95% CI)
p
C-Index
HR (95% CI)
p
Clinical factors
0.751
Sex
0.476
0.93 (0.57–1.51)
0.768
1.16 (1.12–1.20)
0.691
Age
0.525
1.34 (0.81–2.22)
0.258
1.54 (1.50–1.58)
0.164
Risk scores
Fong [ 11 ]
0.545
1.17 (0.74–1.85)
0.500
0.91 (0.88–0.94)
0.741
Table 4: Comparison of SurvAMINN against existing clinical and genomic biomarkers using univariate and multivariate Cox regression
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Framework overview: liver, tumors, and spleen are segmented and used to extract radiomic features. An imaging risk score is then predicted from radiomics and combined with other biomarkers to predict patient prognosis.
Class
Category
Features
First-order
Statistical
18
Shape-based
Shape
14
GLCM
Textural
22
GLRLM
Textural
16
GLSZM
Textural
16
GLDM
Textural
14
Appendix
Table S1: Summary of extracted radiomic features
Figure S2: Qualitative assessment of the segmentation pipeline on a sample testing case.
Figure S3: Failure analysis of the segmentation pipeline illustrating two testing cases with false-positive detections: (a) two FPs located near a surgical resection margin in a patient who underwent hepatectomy prior to MRI acquisition, and (b) one FP corresponding to biliary duct dilatation.
Figure S4: Kaplan–Meier curves of our outcome prediction pipeline using (a) ground truth segmentations and (b) predicted segmentations. Five patients, for whom no tumors were predicted, were excluded from the analysis in (b), as hazard scores could not be computed.
Figure S5: Randomized testing of SurvAMINN. Blue bars represent the C-index distribution when training and testing on randomly shuffled labels. Red line represents the observed score without randomization.
Accurate segmentation of the future liver remnant (FLR) is critical for surgical planning in colorectal liver metastases (CRLM) to prevent fatal post-hepatectomy liver failure. However, this segmentation task is technically challenging due to complex resection boundaries, convoluted hepatic vasculature and diffuse metastatic lesions. A primary bottleneck in developing automated AI tools has been the lack of high-fidelity, validated data. We address this gap by manually refining all 197 volumes from the public CRLM-CT-Seg dataset, creating the first open-source, validated benchmark for this task. We then establish the first segmentation baselines, comparing cascaded (Liver->CRLM->FLR) and end-to-end (E2E) strategies using nnU-Net, SwinUNETR, and STU-Net. We find a cascaded nnU-Net achieves the best final FLR segmentation Dice (0.767), while the pretrained STU-Net provides superior CRLM segmentation (0.620 Dice) and is significantly more robust to cascaded errors. This work provides the first validated benchmark and a reproducible framework to accelerate research in AI-assisted surgical planning.
Anthony T. Wu, Arghavan Rezvani, Kela Liu +4
University of California, Irvine Donald Bren School of Information and Computer Science · University of California, Irvine School of Medicine
Accurate segmentation of colorectal liver metastases (CRLM) in contrast-enhanced computed tomography (CT) is important for response assessment, surgical planning, and follow-up. We propose two parameter-efficient spectral adapters for the Segment Anything Model (SAM): the Directional Spectral Adapter (DiSECT) and Spectral Instance-Guided Adapter (SiGA). DiSECT uses singular value decomposition of frozen weights to constrain residual updates to leading spectral directions, while SiGA adds global and input-conditioned gating through a multilayer perceptron. We evaluate these methods on 446 contrast-enhanced CT volumes (355 training, 91 testing) and compare them with LoRA, QLoRA, convolutional adapters (CAD), and a 3D nnU-Net baseline. Experiments consider single-point, three-point, bounding-box, and no-prompt regimes. SiGA achieves the best single-point performance with a Dice score of 0.77, IoU of 0.69, and HD95 of 35.39 mm. Under no-prompt inference, SiGA reaches 0.76 Dice, 0.68 IoU, and 46.76 mm HD95, comparable to the nnU-Net baseline (0.758 Dice). DiSECT uses only 0.14 million trainable parameters. These results show that spectral adapters can efficiently adapt SAM for CRLM segmentation while retaining strong accuracy with limited trainable parameters.
Ramtin Mojtahedi, Mohammad Hamghalam, Jacob J. Peoples +6
School of Computing, Queen’s University, Kingston, ON, Canada · Toronto General Hospital Research Institute, University Health Network, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada · Department of Electrical Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran +6
Background: Colorectal cancer (CRC) remains a leading cause of cancer-related mortality worldwide. Accurate survival prediction is essential for treatment stratification, yet existing pathology foundation models often overlook organ-specific features critical for CRC prognostication. Methods: We propose MorphDistill, a two-stage framework that distills complementary knowledge from multiple pathology foundation models into a compact CRC-specific encoder. In Stage I, a student encoder is trained using dimension-agnostic multi-teacher relational distillation with supervised contrastive regularization on large-scale colorectal datasets. This preserves inter-sample relationships from ten foundation models without explicit feature alignment. In Stage II, the encoder extracts patch-level features from whole-slide images, which are aggregated via attention-based multiple instance learning to predict five-year survival. Results: On the Alliance/CALGB 89803 cohort (n=424, stage III CRC), MorphDistill achieves an AUC of 0.68 (SD 0.08), an approximately 8% relative improvement over the strongest baseline (AUC 0.63). It also attains a C-index of 0.661 and a hazard ratio of 2.52 (95% CI: 1.73-3.65), outperforming all baselines. On an external TCGA cohort (n=562), it achieves a C-index of 0.628, demonstrating strong generalization across datasets and robustness across clinical subgroups. Conclusion: MorphDistill enables task-specific representation learning by integrating knowledge from multiple foundation models into a unified encoder. This approach provides an efficient strategy for prognostic modeling in computational pathology, with potential for broader oncology applications. Further validation across additional cohorts and disease stages is warranted.
Hikmat Khan, Usama Sajjad, Metin N. Gurcan +4
Department of Pathology, College of Medicine, The Ohio State University Wexner Medical Center, Columbus, OH 43210, USA · Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC 27101, USA