A four-step validation protocol for radiomic survival analysis in small cohorts: a body composition case study
Authors: Mariela Talavera-Muñoz, Cecilia Diana-Albelda, Miguel Antonio Sampedro-Nuñez, Victor Rodriguez-Laval, Jesus Bescos, Álvaro García-Martín
Organizations: Video Processing and Understanding Lab, Escuela Politécnica Superior, Universidad Autónoma de Madrid, Calle Francisco Tomás y Valiente 11, Madrid, 28049, Madrid, Spain · Department of Endocrinology and Nutrition, Hospital Universitario de La Princesa, Instituto de Investigación Sanitaria de La Princesa, Calle de Diego de León 62, Madrid, 28006, Madrid, Spain · Instituto de Investigación Sanitaria del Hospital Universitario de La Princesa, Calle de Diego de León 62, Madrid, 28006, Madrid, Spain · Department of Radiodiagnostics, Hospital Universitario de La Princesa, Calle de Diego de León 62, Madrid, 28006, Madrid, Spain
Gastroenteropancreatic neuroendocrine tumors (GEP-NET) lack prognostic biomarkers, and computed tomography (CT)-based body composition radiomics at the third lumbar vertebra (L3) is a promising candidate. Small cohorts make high-dimensional radiomic models prone to overfitting and single-split evaluations variable and optimistic. This methodological study proposes a four-step validation protocol for radiomic survival analysis in small cohorts: (1) false discovery rate (FDR)-corrected feature selection, (2) a dual-path comparison of prior feature selection versus embedded regularization, (3) repeated nested cross-validation (CV) and (4) a negative control with Gaussian noise features. In 78 GEP-NET patients, 535 radiomic features from five L3 compartments were used to evaluate five survival models with Harrell's concordance index (C-index), time-dependent area under the curve (AUC) and integrated Brier score (IBS). Step 1 selected at least one feature in 18% of outer folds under noise (nominal level 20%) versus 84% with radiomic features. The dual-path comparison showed no consistent difference between strategies. Repeated nested CV reduced variability across seeds (41% median reduction in C-index standard deviation), but not enough to distinguish models. The nested CV C-index exceeded chance by 0.07-0.15 with radiomic features but not clearly under noise (0.40-0.42 with prior feature selection, 0.47-0.55 with embedded regularization). Frequently selected features tentatively linked greater subcutaneous fat, skeletal muscle compactness and texture to better prognosis. These findings suggest a modest prognostic signal in L3 radiomics, with considerable uncertainty that the protocol quantifies. The protocol may transfer to other small-cohort radiomic survival studies. Code: https://github.com/marielatalaveram/radiomics-survival-validation-protocol.
Figures & tables
Figure 1: Workflow for radiomic feature extraction. A standardized volume consisting of ten central slices at the L3 vertebral level is selected from the abdominal CT scan. Automatic segmentations are refined by clinical experts to delineate four body composition compartments (skeletal muscle, subcutaneous fat, visceral fat, and intramuscular fat), alongside a fifth compartment, a spherical region of interest (ROI) placed manually in the paraspinal muscles (manual ROI). Finally, 107 radiomic features, encompassing shape, intensity and texture, are extracted from each of the five compartments per patient.
Figure 2: Proposed validation protocol for radiomic survival analysis. Step 1 selects features from the 535 radiomic features extracted across five compartments: a false discovery rate (FDR) filter (univariate Cox analysis with FDR correction, q≤0.20 ) followed by a redundancy filter (correlation clustering, ∣ρ∣≥0.80 ). Step 2 establishes a dual-path comparison of dimensionality-reduction strategies: Path A fits the survival models on the selected set of features, whereas Path B fits them on the full set of 535 features, relying on embedded regularization. Step 3 evaluates both paths with repeated nested cross-validation (CV): 5-fold CV stratified by event status, with hyperparameters tuned in an inner loop, repeated with ten random seeds to quantify the variability of the estimates. Finally, Step4 applies the analytical pipeline (Steps 1–3) to Gaussian noise features as a negative control.
Overall
Events
Censored
Study cohort, N (%)
78 (100.0%)
27 (34.6%)
51 (65.4%)
Sex, n (%)
Female
38 (48.7%)
16 (59.3%)
22 (43.1%)
Male
40 (51.3%)
11 (40.7%)
29 (56.9%)
Age (years), median (IQR)
63.0 (52.2–75.0)
72.0 (62.0–77.5)
58.0 (46.0–73.0)
Age groups (years), n (%)
Table 1: Demographic variables of the study cohort (N=78), stratified by survival outcome. Event denotes patients who died, whereas censored refers to patients who did not experience the event during the follow-up period. Overall survival time is the observed time from diagnosis to death or last clinical contact, so censored patients contribute their follow-up time. Age groups are defined at diagnosis. Values are presented as counts (%) or medians (interquartile range, IQR).
Figure 3: Kaplan–Meier survival curves for (a) the study cohort and (b) patients stratified by age group. Follow-up time is expressed in days, and survival probability is denoted as S(t)∈[0,1] . Age groups in (b) are color-coded: the youngest group ( < 55) is plotted in green, the 55–65 group in orange, 65–75 in blue, and ≥75 group in pink.
Figure 4: Univariate Cox analysis and correlation of the selected radiomic features in the training data of the single split. (a) Forest plot of the hazard ratios per SD (HR, Log scale) and 95% confidence intervals (CI), grouped by compartment. (b) Spearman correlation matrix of the 7 selected features, with all pairwise ∣ρ∣ below the redundancy filter threshold ( 0.80 ). GLNU: Gray-Level Non-Uniformity ⋅ Idmn: Inverse Difference Moment Normalized ⋅ GLCM: Gray-Level Co-occurrence Matrix ⋅ GLSZM: Gray-Level Size Zone Matrix ⋅ NGTDM: Neighborhood Gray-Tone Difference Matrix.
Feature a
Tissue compartment
Radiomic family b
HR
95% CI
Raw p -value
FDR p -value
C -index
Schoenfeld p
SF/CSF
Contrast
Subcutaneous fat
NGTDM
2.07
1.30–3.29
0.002
0.080
0.70
0.955
0.25 / 0.35
Major Axis Length
Shape
0.33
0.16–0.69
0.003
0.088
0.64
0.021
0.25 / 0.35
GLNU
GLSZM
0.35
0.17–0.73
0.005
0.130
0.66
0.140
0.53 / 0.74
Correlation
GLCM
2.07
1.23–3.50
0.007
0.132
0.68
0.475
0.07 / 0.10
Sphericity
Skeletal muscle
Shape
0.22
0.08–0.61
0.003
0.091
0.75
0.369
0.50 / 0.69
Contrast
NGTDM
0.41
0.21–0.79
0.007
0.132
0.63
0.195
0.41 / 0.57
Table 2: Radiomic features selected in Step 1 in the training data of the single split (FDR filter: univariate Cox analysis with Benjamini–Hochberg correction, q≤0.20 ; redundancy filter: correlation clustering, ∣ρ∣≥0.80 ), grouped by compartment. HR: hazard ratio per standard deviation ( HR<1 indicates a protective effect); CI: 95% confidence interval; raw p and FDR p : unadjusted and Benjamini–Hochberg-adjusted p -values from the univariate Cox analysis (Wald test); C -index: Harrell’s concordance index from the univariate Cox analysis; Schoenfeld p : test of the proportional hazards assumption ( p<0.05 indicates violation); SF and CSF: selection frequency and conditional selection frequency over 100 repetitions of Step 1 (range 0–1).
Figure 5: Performance of the survival models under the two validation schemes (a) and comparison between Path A (prior feature selection) and Path B (full set of features with embedded regularization) (b). (a) Harrell’s concordance index ( C -index) obtained with the single split (orange) and with repeated nested cross-validation (CV) (blue), for each model–path combination. Each gray line joins, for one seed, the single-split estimate and the nested CV estimate (mean across the five outer folds); black points and bars show the mean ± standard deviation (SD) between seeds, and the dashed line marks chance level ( C -index =0.5 ). Numerical values are given in Table 3 . (b) Per-seed paired difference between Path A and Path B for the C -index, the time-dependent area under the receiver operating characteristic curve (AUC) and the integrated Brier score (IBS). Gray points are individual seeds ( n=10 ) and black diamonds the mean across seeds ( ΔA−B ). Positive values (shaded area) indicate better performance of Path A (sign inverted for the IBS, which is undefined for SSVM). Horizontal axes are scaled independently for each metric. Numerical values are given in Table 4 .
Radiomic features
Gaussian noise features
Metric
Path
Model
Nested CV
Single split
Δ b, d
∣Δ∣ c, d
Nested CV
Margin e
C -index ↑
A
Cox Ridge
0.617±0.049
0.631±0.083
+0.014
0.058
0.405±0.048
0.231
Cox LASSO
0.606±0.050
0.625±0.084
+0.019
0.062
0.413±0.051
0.210
RSF
0.643±0.051
0.632±0.119
−0.011
0.076
0.423±0.073
0.245
GBSA
0.619±0.055
0.579±0.127
−0.040
0.080
0.412±0.037
0.226
SSVM
0.629±0.054
0.616±0.078
−0.013
0.074
0.399±0.037
0.250
Table 3: Performance of each model–path combination over ten seeds, estimated with repeated nested cross-validation (CV) and the single split on the radiomic features, and with repeated nested CV only on the Gaussian noise features. Performance was measured with Harrell’s concordance index ( C -index), the time-dependent area under the receiver operating characteristic curve (AUC) and the integrated Brier score (IBS). The nested CV and single split columns show the mean ± standard deviation (SD) between seeds, accounting for folds with an empty selection a . Upward arrows indicate metrics where higher values represent better performance, whereas the downward arrow indicates the opposite.
C -index ↑
AUC ↑
IBS c ↓
Model
ΔA−B
nA
ΔA−B
nA
ΔA−B
nA
Cox Ridge
−0.023±0.042
2/10
−0.023±0.051
4/10
+0.019±0.032
8/10
Cox LASSO
0.000±0.034
5/10
−0.002±0.051
3/10
−0.004±0.007
3/10
RSF
−0.009±0.051
6/10
−0.016±0.056
4/10
+0.009±0.005
10/10
GBSA
+0.033±0.067
7/10
+0.034±0.081
6/10
−0.001±0.015
7/10
SSVM
+0.057±0.082
8/10
+0.047±0.100
8/10
–
–
Table 4: Dual-path comparison under repeated nested cross-validation (CV) over ten seeds, in which Path A fits the models on the selected set of features (prior feature selection) and Path B on the full set of features (embedded regularization). Performance was measured with Harrell’s concordance index ( C -index), the time-dependent area under the receiver operating characteristic curve (AUC) and the integrated Brier score (IBS). ΔA−B a is the mean paired difference between paths ( ± standard deviation, SD, between seeds), and nA b is the number of seeds in which Path A performed better. Positive ΔA−B values indicate better performance of Path A for all three metrics. Upward arrows indicate metrics for which higher values are better, whereas the downward arrow indicates the opposite. The mean estimate of each path is reported in Table 3 .
Radiomics quantifies medical-image shape, intensity, and texture to develop biomarkers for diagnosis, prognosis, treatment response, and biological characterization.
IT Infrastructure for Translational Medical Research, University of Augsburg · Department of Neurosurgery, Río Hortega University Hospital · Specialized Group in Biomedical Imaging and Computational Analysis, Instituto de Investigación Biosanitaria de Valladolid +1
Division AI4Health, Carl von Ossietzky Universität Oldenburg, Ammerländer Heerstraße 114-118, Oldenburg 26129, Niedersachsen, Germany · University Institute for Diagnostic and Interventional Radiology, Klinikum Oldenburg AöR, Rahel-Straus-Straße 10, Oldenburg 26133, Niedersachsen, Germany