Interpretable Multimodal MRI-Clinical Fusion for Functional Outcome Prediction After Acute Ischemic Stroke

Sina Raeisadigh; Henning Müller; Abderrahmane Hedjoudje

cs.LGNov 5, 2026

Interpretable Multimodal MRI-Clinical Fusion for Functional Outcome Prediction After Acute Ischemic Stroke

Authors: Sina Raeisadigh, Henning Müller, Abderrahmane Hedjoudje

Organizations: Department of Computer Science, University of Geneva, Switzerland · Informatics Institute, University of Applied Sciences Western Switzerland (HES-SO), Sierre, Switzerland · Service of Medical Informatics, University Hospital of Geneva, Switzerland · The Sense, Lausanne and Sion, Switzerland ·

Abstract

This study evaluates the added prognostic value of day-1 multimodal MRI for predicting three-month functional outcome after acute ischemic stroke (AIS). Sixty-six AIS patients with ADC, DWI, FLAIR, and clinical data were analyzed. Three-dimensional pretrained ResNet embeddings were extracted from each MRI modality, reduced using fold-isolated principal component analysis, and combined with structured clinical variables and ADC-derived lesion biomarkers. Linear support vector machines were evaluated using eight-fold stratified group cross-validation across MRI modality combinations. Clinical-only modeling achieved strong performance (AUC = $0.899 \pm 0.151$ ), while MRI-only models showed limited discrimination (AUC $\approx$ $0.67$ - $0.73$ ). Among multimodal configurations, FLAIR-clinical fusion achieved the highest mean cross-validated performance (AUC = $0.948 \pm 0.061$ ), followed by ADC-clinical fusion (AUC = $0.941 \pm 0.094$ ). Full ADC-DWI-FLAIR fusion did not consistently outperform single-modality clinical fusion. Feature-importance analysis identified age, NIHSS subscores, ADC lesion burden, and learned FLAIR representations as key contributors, while qualitative FLAIR activation maps showed responses over lesion-like regions. These findings suggest that compact day-1 FLAIR and ADC representations provide complementary prognostic information beyond clinical variables and support interpretable, data-efficient outcome prediction in AIS.

Figures & tables

Configuration	Train AUC	Val AUC	Val Acc	Val F1	$\Delta$ Val AUC
FLAIR + clinical + ADC lesions	$0.949\pm 0.018$	$\mathbf{0.948\pm 0.061}$	$0.802\pm 0.093$	$0.732\pm 0.175$	$\mathbf{+0.049}$
ADC + clinical + ADC lesions	$0.941\pm 0.012$	$0.941\pm 0.094$	$0.816\pm 0.097$	$0.776\pm 0.086$	$+0.042$
ADC + FLAIR + clinical + ADC lesions	$0.959\pm 0.006$	$0.932\pm 0.070$	$0.788\pm 0.105$	$0.747\pm 0.161$	$+0.033$
ADC + DWI + FLAIR + clinical + ADC lesions	$0.963\pm 0.011$	$0.926\pm 0.107$	$0.708\pm 0.130$	$0.636\pm 0.198$	$+0.027$
ADC + DWI + clinical + ADC lesions	$0.956\pm 0.014$	$0.922\pm 0.118$	$0.755\pm 0.118$	$0.674\pm 0.184$	$+0.023$
Clinical only, without ADC lesions	$0.913\pm 0.017$	$0.899\pm 0.151$	$\mathbf{0.845\pm 0.161}$	$\mathbf{0.820\pm 0.145}$	$0.000$

Table 1: Cross-validation performance for selected configurations. Values are mean

\pm

standard deviation across folds. Delta AUC is computed relative to the clinical-only model without ADC lesion biomarkers.

Figure 1: Filtered global feature importance from the direct linear probe. Bars show average SVM coefficients across folds after excluding rare binary variables from visualization. Positive coefficients indicate association with unfavorable outcome (mRS $>1$ ), while negative coefficients indicate association with favorable outcome. Age, NIHSS-related variables, ADC lesion burden, and learned MRI features contributed to the prediction.

Figure 2: Qualitative FLAIR CNN activation visualization for representative patients. Each row shows the raw FLAIR slice, the CNN activation map, and the activation overlaid on FLAIR. The selected activation channel responds over lesion-like FLAIR hyperintense regions. These maps are qualitative activation visualizations and should not be interpreted as lesion segmentation or causal attribution.

Clinical Outcome Prediction

Clinical outcome prediction uses patient records, imaging, signals, and biomarkers to estimate risks, disease trajectories, treatment response, and survival.

See all 274 papers

Explainability in mulimodal deep transformation models for stroke outcome prediction
Apr 7, 2025Lisa Herzog, Jonas Brändli, Maurice Schneeberger +7Clinical Outcome Prediction Explainable Medical Image Analysis
University Hospital Zurich, Department of Neurology, Frauenklinikstrasse 26, 8091 Zurich, Switzerland · Zurich University of Applied Sciences, Institute of Data Analysis and Process Design (IDP), Technikumstrasse 9, 8401 Winterthur, Switzerland · University of Zurich, Epidemiology, Biostatistics and Prevention Institute (EBPI), Hirschengraben 84, 8001 Zurich, Switzerland +1
Vision-Core Guided Contrastive Learning for Balanced Multi-modal Prognosis Prediction of Stroke
May 14, 2026Liren Chen, Lidong Sun, Mingyan Huang +5Cross-Modal Alignment Clinical Outcome Prediction
School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237, PR China
Federated Learning for MRI-based BrainAGE: a multicenter study on post-stroke functional outcome prediction
Jun 18, 2025Vincent Roca, Marc Tommasi, Paul Andrey +8Brain Age Prediction Clinical Outcome Prediction
Univ. Lille, CNRS, Inserm, CHU Lille, Institut Pasteur de Lille, US 41 - UAR 2014 - PLBS, F-59000 Lille, France · Univ. Lille, Inria, F-59000 Lille, France · Univ. Montpellier, Inria, F-34090 Montpellier, France +3

Apr 7, 2025J/K move · Enter open · S save

Explainability in mulimodal deep transformation models for stroke outcome prediction

Lisa Herzog, Jonas Brändli, Maurice Schneeberger, Loran Avci+6

University Hospital Zurich, Department of Neurology, Frauenklinikstrasse 26, 8091 Zurich, Switzerland · Zurich University of Applied Sciences, Institute of Data Analysis and Process Design (IDP), Technikumstrasse 9, 8401 Winterthur, Switzerland · University of Zurich, Epidemiology, Biostatistics and Prevention Institute (EBPI), Hirschengraben 84, 8001 Zurich, Switzerland+1

Clinical Outcome Prediction Explainable Medical Image Analysis Multimodal Model Interpretability Multimodal Learning

Open paper AskPDF arXiv

Multimodal prediction models based on imaging and clinical data are increasingly used for clinical decision support, yet their interpretability remains limited. We present multimodal Deep Transformation Models (DTMs) combining statistical approaches and neural networks to achieve strong predictive performance while preserving interpretability for tabular data. A key contribution of this work is the adaption of the xAI methods Grad-CAM and Occlusion to DTMs relying on 3D CNNs, enabling interpretation of the image branch through the generation of explanation maps. We developed DTMs to predict functional independence three months after stroke using diffusion-weighted imaging and clinical data from 407 patients. In a ten-fold cross-validation, the models achieved state-of-the-art predictive performance (AUC 0.81 [0.75, 0.87]) while maintaining interpretability for tabular features, with functional independence before stroke and stroke severity on admission emerging as the strongest predictors. Explanation maps from both xAI methods highlighted consistent regions, including frontal lobe areas which are known to be associated with age, a strong predictor of functional outcome. Notably, these regions disappeared once age was included as an explicit tabular predictor. Similarity analyses of explanation maps revealed distinct spatial patterns, providing meaningful insights into stroke pathophysiology, systematic error analysis and hypothesis generation.

Abstract

Clinical Outcome Prediction

Explore similar work

Explainability in mulimodal deep transformation models for stroke outcome prediction

Vision-Core Guided Contrastive Learning for Balanced Multi-modal Prognosis Prediction of Stroke

Federated Learning for MRI-based BrainAGE: a multicenter study on post-stroke functional outcome prediction