Apr 7, 2025 · eess.IVJ/K move · Enter open · S save
Lisa Herzog, Jonas Brändli, Maurice Schneeberger, Loran Avci+6
University Hospital Zurich, Department of Neurology, Frauenklinikstrasse 26, 8091 Zurich, Switzerland · Zurich University of Applied Sciences, Institute of Data Analysis and Process Design (IDP), Technikumstrasse 9, 8401 Winterthur, Switzerland · University of Zurich, Epidemiology, Biostatistics and Prevention Institute (EBPI), Hirschengraben 84, 8001 Zurich, Switzerland · Thurgau Institute for Digital Transformation (TIDIT), Hauptstrasse 21a, 8280 Kreuzlingen, Switzerland
Multimodal prediction models based on imaging and clinical data are increasingly used for clinical decision support, yet their interpretability remains limited. We present multimodal Deep Transformation Models (DTMs) combining statistical approaches and neural networks to achieve strong predictive performance while preserving interpretability for tabular data. A key contribution of this work is the adaption of the xAI methods Grad-CAM and Occlusion to DTMs relying on 3D CNNs, enabling interpretation of the image branch through the generation of explanation maps. We developed DTMs to predict functional independence three months after stroke using diffusion-weighted imaging and clinical data from 407 patients. In a ten-fold cross-validation, the models achieved state-of-the-art predictive performance (AUC 0.81 [0.75, 0.87]) while maintaining interpretability for tabular features, with functional independence before stroke and stroke severity on admission emerging as the strongest predictors. Explanation maps from both xAI methods highlighted consistent regions, including frontal lobe areas which are known to be associated with age, a strong predictor of functional outcome. Notably, these regions disappeared once age was included as an explicit tabular predictor. Similarity analyses of explanation maps revealed distinct spatial patterns, providing meaningful insights into stroke pathophysiology, systematic error analysis and hypothesis generation.