Radiomics--Foundation Fusion for Interpretable RCC Classification: Internal Benchmarking and Exploratory External Transfer
Authors: Yuan Liang, Fangyijie Wang, Kathleen M. Curran, Guénolé Silvestre, Sourav Bhattacharjee, Abraham Campbell
Organizations: Research Ireland Centre for Research Training in Machine Learning · School of Computer Science, University College Dublin, Dublin, Ireland · School of Medicine, University College Dublin, Dublin, Ireland · School of Veterinary Medicine, University College Dublin, Dublin, Ireland
Accurate preoperative subtype classification of renal cell carcinoma (RCC) from contrast-enhanced CT remains clinically challenging because clear cell RCC (ccRCC) and non-clear cell RCC often show overlapping imaging appearances. This study evaluates whether foundation representations reduce reliance on handcrafted radiomics, or whether radiomics remains complementary for interpretable tumour characterisation. We compared radiomics, conventional CNN features, MedicalNet-pretrained features, MedVAE representations, and fusion variants for binary ccRCC classification on KiTS23, reporting area under the receiver operating characteristic curve (AUC) with bootstrap confidence intervals and average precision (AP) as a complementary class-imbalance-sensitive metric. We further assessed branch-removal ablation, TCGA/AIMI external transfer, and interpretability using radiomics permutation importance and gate-level analysis. Internally, 3D MedVAE gated fusion achieved the best performance, with an AUC of 82.7% and AP of 92.2%. On the external TCGA cohort, the same model achieved an AUC of 79.5% and AP of 98.9%, although specificity remains uncertain because only two external non-ccRCC cases were available. Gate analysis showed a radiomics-dominant fusion regime, suggesting that foundation representations acted as case-dependent refinement signals rather than replacements for structured tumour descriptors. These findings support radiomics as a complementary and clinically interpretable component of CT-based RCC characterisation in the foundation-model era.
Figures & tables
Figure 1: Overview of the RCC framework: KiTS23 for internal evaluation, TCGA-KIRC/KIRP/KICH with AIMI segmentations for exploratory external transfer, and radiomics–image modelling with interpretability analyses.
Category
Model
AUC (95% CI)
AP
Accuracy
F1
Sensitivity
Specificity
Radiomics
SVC baseline
74.4 [61.2, 87.3]
84.8
62.0
65.6
52.6
83.8
CNN fusion
ResNet18 gated fusion
78.4 [63.5, 90.9]
86.5
75.0
81.9
81.0
61.1
CNN fusion
MedicalNet gated fusion
78.6 [63.9, 91.3]
86.4
70.0
75.0
64.3
83.3
Foundation only
2D MedVAE
59.5 [42.9, 75.4]
78.8
63.3
75.0
78.6
27.8
Foundation only
3D MedVAE
63.2 [46.0, 79.9]
77.2
71.7
82.5
95.2
16.7
Foundation fusion
2D MedVAE gated fusion
79.6 [67.0, 90.4]
90.1
76.7
84.4
90.5
44.4
Table 1: Internal held-out test performance for ccRCC versus non-ccRCC classification. AUC includes bootstrap 95% confidence intervals; all values are percentages.
Variant
Image branch
Radiomics branch
AUC (%)
Radiomics branch only
✗
✓
52.9
Image branch only
✓
✗
60.4
Full fusion
✓
✓
82.7
Table 2: Branch-removal ablation within the trained 3D MedVAE gated-fusion model. AUC is reported on the internal test set.
Category
Model
n
AUC (95% CI)
AP (95% CI)
F1
Sensitivity
Specificity
TN/FP/FN/TP
Radiomics
SVC baseline
41
83.3 [59.0, 100.0]
99.0 [97.5, 100.0]
97.4
97.4
50.0
1/1/1/38
CNN fusion
ResNet18 gated fusion
41
93.6 [79.5, 100.0]
99.7 [98.9, 100.0]
93.3
89.7
50.0
1/1/4/35
CNN fusion
MedicalNet gated fusion
41
96.2 [87.2, 100.0]
99.8 [99.3, 100.0]
90.1
82.1
100.0
2/0/7/32
Foundation only
2D MedVAE
41
70.5 [30.8, 100.0]
97.9 [94.7, 100.0]
87.3
79.5
50.0
1/1/8/31
Foundation only
3D MedVAE
41
64.1 [33.3, 92.3]
97.6 [95.1, 99.6]
92.1
89.7
0.0
0/2/4/35
Foundation fusion
2D MedVAE gated fusion
41
83.3 [65.4, 97.4]
99.1 [98.0, 99.9]
93.3
89.7
50.0
1/1/4/35
Table 3: Exploratory external transfer performance on the TCGA cohort with AIMI segmentations. Models were evaluated using locked internal-validation thresholds without external refitting. AUC and AP are reported with bootstrap 95% confidence intervals. All metric values are percentages.
Figure 2: Top radiomics features ranked by mean AUC decrease after repeated feature-wise permutation. Error bars show variation across repetitions.
Figure 3: Gate-level analysis of the 3D MedVAE gated-fusion model. Each point shows the case-level mean of the 512-dimensional gate vector; higher values indicate greater image weighting and lower values greater radiomics weighting. External class-stratified patterns should be interpreted cautiously due to only two non-ccRCC cases.
Accurate preoperative subtype classification of renal cell carcinoma (RCC) from contrast-enhanced computed tomography remains clinically challenging. Radiomics provides structured tumour descriptors, whereas foundation representations offer transferable image features. However, it remains unclear whether radiomics still adds value beyond pretrained representations, and how 2D and 3D MedVAE encoders compare in this setting. We compared handcrafted radiomics, 2D MedVAE, 3D MedVAE, and their fusion for binary clear-cell RCC versus non-clear-cell RCC classification on KiTS23 under a unified preprocessing pipeline. Concatenation, cross-attention, and gated fusion were evaluated as representative integration strategies, and radiomics feature importance was analysed to support decision-centric interpretability. Fusion consistently improved discrimination over image-only MedVAE branches. The best overall performance was achieved by 3D gated fusion, with an AUC of 82.7%, outperforming the best 2D fusion model (79.6%), the radiomics baseline (74.4%), and the single-modality MedVAE branches. Ablation analysis further showed clear gains of the full fusion model over both image-only and radiomics-only variants, indicating complementary contributions from radiomics and image representations. These findings suggest that radiomics remains relevant for RCC CT classification in the presence of foundation representations, and that its integration with MedVAE is more effective in the 3D setting. More broadly, the study supports a complementary role for radiomics and foundation representations in clinically meaningful imaging decision support.
Yuan Liang, Sourav Bhattacharjee, Abraham Campbell
Research Ireland Centre for Research Training in Machine Learning · School of Computer Science, University College Dublin, Dublin, Ireland · School of Veterinary Medicine, University College Dublin, Dublin, Ireland
Radiomics provides quantitative descriptions of tumour appearance that may complement disease-specific foundation models in small labelled cohorts. We investigate this complementarity for computed tomography-based classification of clear cell renal cell carcinoma. Our framework uses radiomics to modulate RenalCLIP features through feature-wise linear modulation (FiLM), while retaining a direct radiomics contribution. Internal testing and external validation compare it with conventional fusion strategies and reference classifiers. The FiLM model achieves an area under the receiver operating characteristic curve (AUC) of 0.804 internally and 0.854 externally, with the highest mean AUC among the evaluated RenalCLIP fusion strategies in both cohorts. Pathway ablations examine the contributions of conditional modulation and the direct radiomics residual, while feature permutation highlights the role of tumour texture. These findings support radiomics as a useful complement to RenalCLIP in a small labelled cohort and identify FiLM as an effective approach to integrating their representations for robust renal tumour classification.
Yuan Liang, Sourav Bhattacharjee, Abraham Campbell
Research Ireland Centre for Research Training in Machine Learning · School of Computer Science, University College Dublin, Dublin, Ireland · School of Veterinary Medicine, University College Dublin, Dublin, Ireland
Gate coefficients describe how a fusion layer weights intermediate features, but they do not necessarily reflect the magnitude of branch contributions at the classifier output. We investigate this distinction in a fixed 3D MedVAE--radiomics model for clear cell versus non-clear cell renal cell carcinoma classification from contrast-enhanced CT. Because the model uses a linear classification head, the binary decision logit can be decomposed exactly into a gated image term, a direct radiomics term, and a bias. We define effective radiomics dominance as the direct radiomics term's share of the summed absolute non-bias branch contributions and compare it with structural radiomics weighting. Internally, mean structural radiomics weighting was 0.823, whereas mean effective radiomics dominance was 0.148, with a modest case-wise association (\r{ho} = 0.290). The gated image term retained substantially stronger discriminative ranking information than the direct radiomics term. The same structural-to-effective discrepancy was observed in exploratory external evaluation. Branch-targeted permutation, integrated-gradient, and Shapley-style analyses further identified radiomics descriptors influencing the direct radiomics term. Because radiomics also generates the gate, the gated image term remains radiomics-dependent, so additive branch contribution should not be interpreted as total input influence. These findings show that structural gate weighting and decision-level branch contribution represent distinct aspects of gated feature fusion and should be interpreted separately.
Yuan Liang, Sourav Bhattacharjee, Abraham Campbell
Research Ireland Centre for Research Training in Machine Learning · School of Computer Science, University College Dublin, Dublin, Ireland · School of Veterinary Medicine, University College Dublin, Dublin, Ireland