An Automated Radiomics Framework for Postoperative Survival Prediction in Colorectal Liver Metastases using Preoperative MRI
Authors: Muhammad Alberb, Jianan Chen, Hossam El-rewaidy, Paul Karanicolas, Arun Seth, Yutaka Amemiya, Anne Martel, Helen Cheung
Organizations: Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada · Physical Sciences Platform, Sunnybrook Research Institute, Toronto, ON, Canada · UCL Cancer Institute, University College London, London, UK · Systems and Biomedical Engineering Department, Cairo University, Giza, Egypt · Department of Surgery, University of Toronto, Toronto, ON, Canada · Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada · Sunnybrook Health Sciences Centre, Toronto, ON, Canada · Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
While colorectal liver metastasis (CRLM) is potentially curable via hepatectomy, patient outcomes remain highly heterogeneous. Postoperative survival prediction is necessary to avoid non-beneficial surgeries and guide personalized therapy. In this study, we present an automated AI-based framework for postoperative CRLM survival prediction using pre- and post-contrast MRI. We performed a retrospective study of 227 CRLM patients who had gadoxetate-enhanced MRI prior to curative-intent hepatectomy between 2013 and 2020. We developed a survival prediction framework comprising an anatomy-aware segmentation pipeline followed by a radiomics pipeline. The segmentation pipeline learns liver, CRLMs, and spleen segmentation from partially-annotated data, leveraging promptable foundation models to generate pseudo-labels. To support this pipeline, we propose SAMONAI, a prompt propagation algorithm that extends Segment Anything Model to 3D point-based segmentation. Predicted pre- and post-contrast segmentations are then fed into our radiomics pipeline, which extracts per-tumor features and predicts survival using SurvAMINN, an autoencoder-based multiple instance neural network for time-to-event survival prediction. SurvAMINN jointly learns dimensionality reduction and survival prediction from right-censored data, emphasizing high-risk metastases. We compared our framework against established methods and biomarkers using univariate and multivariate Cox regression. Our segmentation pipeline achieves median Dice scores of 0.96 (liver) and 0.93 (spleen), driving a CRLM segmentation Dice score of 0.78 and a detection F1-score of 0.79. Accurate segmentation enables our radiomics pipeline to achieve a survival prediction C-index of 0.69. Our results show the potential of integrating segmentation algorithms with radiomics-based survival analysis to deliver accurate and automated CRLM outcome prediction.
Figures & tables
Figure 1: Dataset processing: (a) cohort definitions and exclusion criteria, (b) patient-level data splitting for training, validation, and testing. Patients with manual liver and spleen annotations were held out from segmentation model development, while patients with available genomics data were held out from both segmentation and survival prediction models development to be exclusively used for testing.
Figure 2: Segmentation pipeline: a promptable foundation model is used to complete partially-annotated masks for a small subset. A few-shot UNETR model is trained on the subset then used to complete the annotations for all cases selected for segmentation model development. Finally, A supervised UNETR model is retrained on the entire training dataset with completed labels.
Figure 3: Overview of SAMONAI applied to the liver: 3D objects are segmented from single points by propagating prompts from one view to another. Positive and negative points are displayed in green and red, respectively.
Figure 4: Radiomics pipeline: per-tumor radiomics are extracted. An autoencoder compresses features and a multiple instance regressor connected to the autoencoder’s bottleneck predicts per-tumor risk scores and pools them into patient hazards. The autoencoder and regressor are jointly trained using a weighted sum of mean squared error LMSE and cox proportional hazard loss LCoxPH .
Figure 5: Segmentation box plots. MedSAM, in blue, and SAMONAI, in orange, denote prompt-based segmentation performance on missing labels. Pipeline (MedSAM), in green, and Pipeline (SAMONAI), in red, denote final automated segmentation performance after integrating each promptable model into our segmentation pipeline. Black dots inside boxes represent mean Dice scores over patients, and gray dots outside boxes represent individual patients beyond the interquartile range.
Contrast
Precision
Recall
F1-score
TP
FP
FN
Pre-contrast
0.674
0.756
0.713
31
15
10
Post-contrast
0.720
0.878
0.791
36
14
5
Table 1: Tumor detection results of the SAMONAI-based segmentation pipeline
Figure 6: Survival prediction box plot showing C-index when using pre-contrast (pre), post-contrast (post), and combined contrast phases (both). Black dots inside boxes represent mean C-index over runs, and gray dots outside boxes represent individual runs beyond the interquartile range. Solid boxes correspond to ground-truth manual segmentations and striped boxes correspond to predicted segmentations. Statistical significance, assessed by Wilcoxon rank-sum test’s p-value, confirms that combining pre- and post-contrast MRI yields better performance than using either phase individually.
Pooling
C-Index
HR (95% CI)
p
mean
0.556
1.114 (1.040–1.216)
0.759
largest
0.594
1.263 (1.147–1.391)
0.618
max
0.623
1.568 (1.441–1.784)
0.162
LSE
0.691
1.929 (1.890–1.980)
< 0.001
Table 2: Comparison of pooling strategies
Model
Reduction
C-Index
HR (95% CI)
p
SVM
None
0.659
1.781 (1.693–1.876)
0.002
SVM
KBest
0.647
1.745 (1.652–1.842)
0.005
SVM
PCA
0.667
1.907 (1.793–2.036)
0.004
SVM
mRMR
0.634
1.568 (1.494–1.652)
0.011
RSF
None
0.632
1.447 (1.368–1.531)
0.072
RSF
KBest
0.637
1.543 (1.453–1.653)
0.056
Table 3: Comparison of SurvAMINN against common machine learning methods
Univariate
Multivariate
Biomarker
C-Index
HR (95% CI)
p
C-Index
HR (95% CI)
p
Clinical factors
0.751
Sex
0.476
0.93 (0.57–1.51)
0.768
1.16 (1.12–1.20)
0.691
Age
0.525
1.34 (0.81–2.22)
0.258
1.54 (1.50–1.58)
0.164
Risk scores
Fong [ 11 ]
0.545
1.17 (0.74–1.85)
0.500
0.91 (0.88–0.94)
0.741
Table 4: Comparison of SurvAMINN against existing clinical and genomic biomarkers using univariate and multivariate Cox regression
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Framework overview: liver, tumors, and spleen are segmented and used to extract radiomic features. An imaging risk score is then predicted from radiomics and combined with other biomarkers to predict patient prognosis.
Class
Category
Features
First-order
Statistical
18
Shape-based
Shape
14
GLCM
Textural
22
GLRLM
Textural
16
GLSZM
Textural
16
GLDM
Textural
14
Appendix
Table S1: Summary of extracted radiomic features
Figure S2: Qualitative assessment of the segmentation pipeline on a sample testing case.
Figure S3: Failure analysis of the segmentation pipeline illustrating two testing cases with false-positive detections: (a) two FPs located near a surgical resection margin in a patient who underwent hepatectomy prior to MRI acquisition, and (b) one FP corresponding to biliary duct dilatation.
Figure S4: Kaplan–Meier curves of our outcome prediction pipeline using (a) ground truth segmentations and (b) predicted segmentations. Five patients, for whom no tumors were predicted, were excluded from the analysis in (b), as hazard scores could not be computed.
Figure S5: Randomized testing of SurvAMINN. Blue bars represent the C-index distribution when training and testing on randomly shuffled labels. Red line represents the observed score without randomization.
School of Computing, Queen’s University, Kingston, ON, Canada · Toronto General Hospital Research Institute, University Health Network, 200 Elizabeth Street, Toronto, ON M5G 2C4, Canada · Department of Electrical Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran +6
Department of Pathology, College of Medicine, The Ohio State University Wexner Medical Center, Columbus, OH 43210, USA · Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC 27101, USA