Beyond Gate Weights: Decision-Level Interpretation of Radiomics-Image Feature Fusion for RCC Classification
Organizations: Research Ireland Centre for Research Training in Machine Learning · School of Computer Science, University College Dublin, Dublin, Ireland · School of Veterinary Medicine, University College Dublin, Dublin, Ireland
Abstract
Gate coefficients describe how a fusion layer weights intermediate features, but they do not necessarily reflect the magnitude of branch contributions at the classifier output. We investigate this distinction in a fixed 3D MedVAE--radiomics model for clear cell versus non-clear cell renal cell carcinoma classification from contrast-enhanced CT. Because the model uses a linear classification head, the binary decision logit can be decomposed exactly into a gated image term, a direct radiomics term, and a bias. We define effective radiomics dominance as the direct radiomics term's share of the summed absolute non-bias branch contributions and compare it with structural radiomics weighting. Internally, mean structural radiomics weighting was 0.823, whereas mean effective radiomics dominance was 0.148, with a modest case-wise association (\r{ho} = 0.290). The gated image term retained substantially stronger discriminative ranking information than the direct radiomics term. The same structural-to-effective discrepancy was observed in exploratory external evaluation. Branch-targeted permutation, integrated-gradient, and Shapley-style analyses further identified radiomics descriptors influencing the direct radiomics term. Because radiomics also generates the gate, the gated image term remains radiomics-dependent, so additive branch contribution should not be interpreted as total input influence. These findings show that structural gate weighting and decision-level branch contribution represent distinct aspects of gated feature fusion and should be interpreted separately.
Figures & tables
| Cohort | Score | AUC (95% CI) | AP (95% CI) |
|---|---|---|---|
| Internal | Full score | 0.827 (0.710–0.923) | 0.922 (0.867–0.967) |
| 0.844 (0.738–0.938) | 0.928 (0.877–0.972) | ||
| 0.519 (0.352–0.689) | 0.720 (0.629–0.833) | ||
| External | Full score | 0.782 (0.641–0.897) | 0.988 (0.979–0.995) |
| 0.756 (0.615–0.872) | 0.987 (0.978–0.993) | ||
| 0.462 (0.000–0.974) | 0.926 (0.862–0.999) |
| Cohort | Relation | Errors/ (%) | Mean | Median | |
|---|---|---|---|---|---|
| Internal | Agreement | 43 | 11/43 (25.6) | 0.128 | 0.086 |
| Conflict | 17 | 6/17 (35.3) | 0.198 | 0.043 | |
| External | Agreement | 27 | 7/27 (25.9) | 0.060 | 0.035 |
| Conflict | 14 | 4/14 (28.6) | 0.112 | 0.039 |