cs.LGOct 23, 2026

DIGOM: Disentangled Interaction Graphs for Outcome prediction in Multimodal healthcare

Authors: Hakima Laribi, Olivier Lefebvre, Dan Poenaru, Martin Vallières

Organizations: Department of Computer Science, Université de Sherbrooke, 2500 Boulevard de l’Université, Sherbrooke, Québec, J1K 2R1, Canada · Department of Pediatric Surgery, Montreal Children’s Hospital, 1001 Boulevard Décarie, Montreal, Québec, H4A 3J1, Canada · Centre for Outcomes Research and Evaluation (CORE), Research Institute of the McGill University Health Centre, 5252 Boulevard de Maisonneuve Ouest, Montreal, Québec, H4A 3S5, Canada · Medical Physics Unit, Department of Oncology, McGill University, 1001 Boulevard Décarie, Montreal, Québec, H4A 3J1, Canada

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Multimodal healthcare data contain relational information that extends beyond individual patient observations. Effective clinical outcome prediction therefore requires not only integrating information from multiple modalities, but also modeling the relationships between them. Standard fusion approaches do not explicitly capture these interactions, while existing graph-based approaches typically model only a subset of them. We propose DIGOM (Disentangled Interaction Graphs for Outcome prediction in Multimodal healthcare), a graph-based framework that jointly models three complementary interaction types: Intra-patient cross-modal, Cross-patient same-modal, and Cross-patient cross-modal interactions. DIGOM learns a sparse multiplex graph with disentangled views, each encoding interactions between pairs of modalities across patients. Discrete adjacency matrices with patient-adaptive connectivity are sampled via a differentiable mechanism, enabling end-to-end learning and allowing the model to select informative relationships without imposing a fixed number of neighbors. A regularization objective maintains overall graph sparsity, while message passing integrates this relational context into patient representations for outcome prediction. We evaluate DIGOM on two multimodal healthcare datasets, MIMIC-IV and NIH-TB, across five clinical prediction tasks. DIGOM achieves the highest AUROC on all five tasks while using fewer connections than existing multi-interaction approaches. Ablation studies reveal that each interaction type's contribution varies across tasks, highlighting the need to adapt relational structures per task. DIGOM also demonstrates greater robustness to missing modalities, suggesting learned relationships provide complementary information when modalities are unavailable. Post-hoc analyses further reveal task- and outcome-specific connectivity patterns in the learned graphs.

Figures & tables

Figure 1: Illustration of the three interaction types modeled by DIGOM and their role in clinical reasoning. (a) Existing methods typically capture only one of Intra-patient cross-modal or Cross-patient same-modal interactions, whereas DIGOM jointly models both along with Cross-patient cross-modal interactions, capturing a richer relational structure across patients and modalities. (b) A clinical case illustrating how a newly admitted patient (green) can benefit from information from a previously seen patient (red) with similar clinical findings who experienced in-hospital mortality. Shared observations help identify the related patient, while their remaining clinical information and outcome provide additional context for understanding the current patient’s condition and anticipating their outcome.
Figure 2: Overview of the DIGOM framework. ( Top ) The patient-adaptive sparse graph learning module produces two latent projections HH and ZZ for each modality, computes pairwise similarity matrices, and samples discrete adjacency matrices to construct a sparse multimodal multi-interaction graph organized into M2M^{2} disentangled views, each encoding directed interactions from a source modality to a target modality across all patients. Views are grouped by target modality, such that each group contains all incoming interactions for a given modality. ( Bottom ) The multimodal multi-interaction message passing module propagates information across the learned graph for LL layers. At each layer, representations of target modality mlm_{l} are updated by aggregating neighboring representations from all MM source modality views and combining them with the node’s own representation. The resulting enriched modality representations are concatenated and passed through a fully connected layer to produce the final outcome prediction Y^\hat{Y} . The model is jointly optimized using the cross-entropy loss LCE\mathcal{L}_{\mathrm{CE}} , the soft-HGR correlation loss LsHGR\mathcal{L}_{\mathrm{sHGR}} , and the density regularization loss Ldensity\mathcal{L}_{\mathrm{density}} .
MethodIntra-patientCross-patientCross-patient
 cross-modalsame-modalcross-modal
Early Fusion✗✗✗
Joint Fusion✗✗✗
Late Fusion✗✗✗
STGNN✗✓✗
SGNN✗✓✗
Table 1: Interaction types modeled by the considered baselines and DIGOM. ✓\checkmark indicates that the interaction type is explicitly modeled, ∼\sim indicates that it is captured but not modeled separately, and ✗ indicates that it is not modeled.
Figure 3: Evaluation protocol of each model based on nested five-fold cross-validation. The inner loops are used for hyperparameter optimization, and the same data splits are used for all models.
 MIMIC-IVNIH-TB
 48h Mortality48h LOS30d ReadmissionTB OutcomeTB Mortality
Early Fusion0.870±0.02∗0.870\pm 0.02^{*}0.802±0.02†0.802\pm 0.02^{\dagger}0.675±0.01∗0.675\pm 0.01^{*}0.726±0.01∗0.726\pm 0.01^{*}0.734±0.01∗0.734\pm 0.01^{*}
Joint Fusion0.871±0.01∗0.871\pm 0.01^{*}0.813±0.020.813\pm 0.020.670±0.02∗0.670\pm 0.02^{*}0.745±0.01†0.745\pm 0.01^{\dagger}0.741±0.02∗0.741\pm 0.02^{*}
Late Fusion0.865±0.01∗0.865\pm 0.01^{*}0.789±0.02∗0.789\pm 0.02^{*}0.672±0.01∗0.672\pm 0.01^{*}0.728±0.02∗0.728\pm 0.02^{*}0.745±0.02†0.745\pm 0.02^{\dagger}
STGNN0.887±0.01∗0.887\pm 0.01^{*}0.814±0.010.814\pm 0.010.675±0.010.675\pm 0.01––
SGNN0.883±0.01∗0.883\pm 0.01^{*}0.813±0.010.813\pm 0.010.683±0.010.683\pm 0.010.735±0.01∗0.735\pm 0.01^{*}0.766±0.020.766\pm 0.02
Table 2: AUROC comparison across the five prediction tasks. Results are reported as mean ±\pm standard deviation across the five outer testing folds. For the multiclass tuberculosis treatment outcome, AUROC is macro-averaged over the five classes. The best-performing model is shown in bold , and the second-best is underlined . Statistical significance was assessed using a one-sided Wilcoxon signed-rank test [ 50 ] to determine whether each baseline significantly underperformed DIGOM. ∗ and † indicate p-value<0.05p\text{-value}<0.05 and 0.05≤p-value<0.10.05\leq p\text{-value}<0.1 , respectively. From left to right, the tasks are 48-hour ICU mortality risk, 48-hour ICU length of stay, 30-day readmission risk, tuberculosis treatment outcome, and mortality risk after tuberculosis treatment.
 48M48LOS30DRTBM
Early Fusion0.0530.0530.0140.0140.0110.0110.0250.025
Joint Fusion0.0570.057−0.005-0.0050.0180.0180.0430.043
Late Fusion0.0510.0510.0350.0350.0270.0270.0070.007
STGNN0.0240.024−0.001-0.0010.0080.008–
SGNN0.0180.018−0.009-0.0090.0090.009−0.017-0.017
MMGL0.3040.3040.0040.0040.0360.0360.0190.019
Table 3: Net Reclassification Improvement (NRI) of DIGOM over each baseline. NRI measures whether DIGOM more appropriately reclassifies patients across risk categories compared with each baseline. Positive NRI values indicate improved overall reclassification, whereas negative values indicate worse reclassification. Since NRI is defined for binary classification, the multiclass tuberculosis treatment outcome is not included in this analysis. The decision threshold is selected to maximize the difference between sensitivity and the false positive rate on the training set. 48M : 48-hour ICU mortality risk, 48LOS : 48-hour ICU length of stay, 30DR : 30-day readmission risk, TBM : mortality risk after tuberculosis treatment.
Figure 4: AUROC of DIGOM trained with a single interaction type at a time, and with no interactions, across all five prediction tasks. Each bar reports the mean AUROC over five cross-validation folds, with error bars indicating the standard deviation. The dashed teal line marks the performance of the proposed DIGOM model, in which all three interaction types are learned jointly, and serves as a reference. The contribution of each interaction type varied across tasks, with no single type consistently providing the greatest benefit.
Figure 5: AUROC of the selected baselines and DIGOM when one modality is masked at a time at inference time, across all five prediction tasks. For each masked modality, continuous columns are imputed using the column mean and categorical columns using the column median. Each spoke of the radar chart corresponds to a modality, and each line represents a model evaluated after masking that modality. Graph-based methods modeling multiple interaction types generally maintained higher performance under missing modalities than conventional fusion methods or methods restricted to a single interaction type.
Figure 6: Distribution of the average number of connections per patient in the graphs learned by DIGOM, with and without the density loss, across four prediction tasks. The distribution without the density loss could not be obtained for 30-day readmission risk due to the memory cost of constructing a dense graph for its large cohort; its distribution with the density loss is shown separately in Figure B.1 . The density loss produced substantially sparser graphs across all tasks, while Table B.2 shows that predictive performance remained largely unaffected, indicating that many connections can be pruned without loss of useful information.
Figure 7: Average connectivity between source and target modality pairs in the graphs learned by DIGOM across all five prediction tasks, averaged over the five test folds. Each cell reports the fraction of patients connected from the source modality to the target modality. Connectivity patterns varied substantially across tasks, with higher connectivity concentrated on CT scan, demographic, and clinical modalities for the tuberculosis tasks, and on radiology notes for 48-hour ICU mortality, reflecting task-specific differences in how the model retrieves relational information across patients and modalities.
Figure 8: Average connection probability between source and target modality pairs in the graphs learned by DIGOM across all five prediction tasks, averaged over the five test folds. Each cell reports the mean probability assigned to the edges selected between the source and target modality prior to sampling. Modality pairs with higher connectivity tended to exhibit lower connection probabilities, suggesting that the model selects connections with greater confidence when it can identify a small targeted set of informative relationships, and resorts to a broader but less certain set when the relevant connections are harder to identify.
Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

 48M48LOS30DR
Temporal0.663±0.02\mathbf{0.663\pm 0.02}0.692±0.010.692\pm 0.010.611±0.01\mathbf{0.611\pm 0.01}
Average0.661±0.02‾\underline{0.661\pm 0.02}0.680±0.030.680\pm 0.030.574±0.010.574\pm 0.01
Weighted average0.625±0.030.625\pm 0.030.700±0.01\mathbf{0.700\pm 0.01}0.578±0.020.578\pm 0.02
First0.632±0.030.632\pm 0.030.629±0.020.629\pm 0.020.564±0.010.564\pm 0.01
Last0.639±0.030.639\pm 0.030.688±0.020.688\pm 0.020.576±0.010.576\pm 0.01
Appendix
Table A.1 : AUROC comparison across different aggregation strategies for chest X-ray imaging across the three prediction tasks in MIMIC-IV. The best strategy is shown in bold , and the second-best is underlined . Results are reported as mean ±\pm standard deviation across the five inner testing folds. Temporal aggregates the image embeddings using a GRU, Average computes their unweighted average, Weighted average assigns higher weights to embeddings of images closer to the prediction time, and First and Last use only the first and last embeddings, respectively. 48M : 48-hour ICU mortality risk, 48LOS : 48-hour ICU length of stay, 30DR : 30-day readmission risk.
 48M48LOS30DR
Temporal0.716±0.020.716\pm 0.020.747±0.020.747\pm 0.020.641±0.01\mathbf{0.641\pm 0.01}
Average0.718±0.01\mathbf{0.718\pm 0.01}0.751±0.01\mathbf{0.751\pm 0.01}0.634±0.010.634\pm 0.01
Weighted average0.697±0.030.697\pm 0.030.736±0.020.736\pm 0.020.633±0.010.633\pm 0.01
First0.634±0.030.634\pm 0.030.629±0.010.629\pm 0.010.613±0.010.613\pm 0.01
Last0.677±0.010.677\pm 0.010.702±0.020.702\pm 0.020.603±0.020.603\pm 0.02
Appendix
Table A.2 : AUROC comparison across different aggregation strategies for radiology notes across the three prediction tasks in MIMIC-IV. The best strategy is shown in bold , and the second-best is underlined . Results are reported as mean ±\pm standard deviation across the five inner testing folds. Temporal aggregates the note embeddings using a GRU, Average computes their unweighted average, Weighted average assigns higher weights to embeddings of notes closer to the prediction time, and First and Last use only the first and last embeddings, respectively. 48M : 48-hour ICU mortality risk, 48LOS : 48-hour ICU length of stay, 30DR : 30-day readmission risk.
 48h Mortality48h LOS30d ReadmissionTB OutcomeTB Mortality
DIGOM w/ density loss7.42%7.42\%4.06%4.06\%3.01%3.01\%9.65%9.65\%4.99%4.99\%
DIGOM w/o density loss50.62%50.62\%52.63%52.63\%–50.03%50.03\%39.82%39.82\%
MaxCorrMGNN top- kk5.94%5.94\%6.33%6.33\%0.99%0.99\%5.01%5.01\%4.23%4.23\%
MaxCorrMGNN–––62.68%62.68\%63.48%63.48\%
Appendix
Table B.1 : Average connectivity rate for DIGOM and MaxCorrMGNN across prediction tasks, with and without graph density constraints. For MaxCorrMGNN, connectivity is reported as the maximum between the two propagation graphs A⋅CA\cdot C and C⋅AC\cdot A , where AA captures Cross-patient same-modal similarities and CC captures cross-modal similarities. Note that the top- kk sparsity constraint is applied independently to AA and CC . Thus, the products A⋅CA\cdot C and C⋅AC\cdot A yield a different effective sparsity, which is why the reported connectivity for MaxCorrMGNN with top- kk does not directly reflect kk . “–” indicates that the condition could not be evaluated due to memory constraints from the large cohort size.
Figure B.1 : Distribution of the average number of connections per patient in the graphs learned by DIGOM for 30-day readmission risk prediction. The density loss effectively constrains graph connectivity despite the large cohort size, producing a sparse graph concentrated around the target density ρ\rho .
 w/o density losswith density loss
48M0.895±0.010.895\pm 0.010.901±0.010.901\pm 0.01
48LOS0.816±0.020.816\pm 0.020.816±0.020.816\pm 0.02
TBO0.752±0.010.752\pm 0.010.761±0.010.761\pm 0.01
TBM0.758±0.040.758\pm 0.040.771±0.020.771\pm 0.02
Appendix
Table B.2 : Effect of the density loss in DIGOM on AUROC. Results are reported as mean ±\pm standard deviation across the five outer testing folds. The 30-day readmission task could not be evaluated without the density loss due to the prohibitive memory cost of constructing a dense graph for its large cohort. 48M : 48-hour ICU mortality risk, 48LOS : 48-hour ICU length of stay, TBO : tuberculosis treatment outcome, and TBM : mortality risk after tuberculosis treatment.
Figure B.2 : Average connectivity between source and target modality pairs in the graphs learned by DIGOM, reported separately for the negative and positive classes across the four binary prediction tasks, averaged over the five test folds. For 48-hour ICU mortality and tuberculosis mortality risk, connectivity is higher in the positive class for the most connected modality pairs, suggesting that the model retrieves information from a broader set of patients when predicting the more clinically severe outcome.
Figure B.3 : Average connectivity between source and target modality pairs in the graphs learned by DIGOM, reported separately for each of the five tuberculosis treatment outcome classes, averaged over the five test folds. Connectivity is highest for the Died, Completed, and Cured classes, particularly for CT scan and demographic modalities, suggesting that the model retrieves information from a broader set of patients when predicting these outcomes.
Model48h Mortality48h LOS30d ReadmissionTB OutcomeTB Mortality
DIGOM w/ density loss13.7713.7711.5511.55105.97105.970.830.830.780.78
DIGOM w/o density loss44.3244.3245.9145.91–1.911.911.981.98
MaxCorrMGNN top- kk11.4611.4612.1312.1332.7832.780.650.650.560.56
MaxCorrMGNN–––5.895.896.016.01
Appendix
Table B.3 : Average inference time (in seconds) on the test set for DIGOM and MaxCorrMGNN across prediction tasks, with and without graph density constraints. Times are averaged across the five outer test folds. “–” indicates that the condition could not be evaluated due to memory constraints from the large cohort size.
ModelTB OutcomeTB Mortality
Joint Fusion0.702±0.010.702\pm 0.01–
Late Fusion–0.723±0.020.723\pm 0.02
SGNN0.721±0.010.721\pm 0.010.734±0.020.734\pm 0.02
MMGL0.705±0.010.705\pm 0.010.718±0.020.718\pm 0.02
MaxCorrMGNN0.723±0.010.723\pm 0.010.707±0.020.707\pm 0.02
DIGOM (ours)0.728±0.010.728\pm 0.010.727±0.020.727\pm 0.02
Appendix
Table B.4 : AUROC on the tuberculosis tasks when models are retrained without the demographic modality. Only the models included in the missing-modality analysis are reported; conventional fusion is therefore represented by its best-performing variant for each task. Results are reported as mean ±\pm standard deviation across the five outer testing folds.
HyperparameterValues
Learning rate[0.0001,0.01][0.0001,0.01]
Positive class weight[0.1,0.9][0.1,0.9]
GRU hidden size{64,128,256,512,1024}\{64,128,256,512,1024\}
ρ\rho (target density){0.01,0.02,0.03,0.04,0.05}\{0.01,0.02,0.03,0.04,0.05\} ∗
λ1\lambda_{1}0.0010.001
λ2\lambda_{2}[0.01,0.5][0.01,0.5]
Appendix
Table C.1 : Hyperparameter search ranges used by DIGOM. Ranges are shared across tasks unless otherwise noted.
ModelHyperparameterValues
Common to all modelsLearning rate[0.0001,0.01][0.0001,0.01]
 Positive class weight †[0.1,0.9][0.1,0.9]
Early FusionDropout[0,0.2][0,0.2]
GRU hidden size{64,128,256,512,1024}\{64,128,256,512,1024\}
MLP layers[1,4][1,4]
MLP hidden size{64,128,256,512,1024}\{64,128,256,512,1024\}
Appendix
Table C.2 : Hyperparameter search ranges used for each baseline model. Ranges are shared across tasks unless otherwise noted.
More in this topic

Clinical Outcome Prediction

Clinical outcome prediction uses patient records, imaging, signals, and biomarkers to estimate risks, disease trajectories, treatment response, and survival.

See all 265 papers

Explore similar work

CardsList

  1. Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction

    Jul 10, 2026Nikkie Hooman, Zhongjie Wu, Eric C. Larson +1Multimodal RobustnessClinical Prediction

    Southern Methodist University Dallas, USA

  2. Multimodal Graph Representation Learning with Dynamic Information Pathways

    Mar 10, 2026Xiaobin Hong, Mingkai Lin, Xiaoli Wang +2Multimodal Graph LearningMultimodal Embedding

    State Key Laboratory for Novel Software Technology, Nanjing University · Nanjing Forest University, College of Information Science and Technology · Zhejiang Provincial Seaport Investment & Operation Group Co. Ltd

  3. M2G-LLM: Enhancing Clinical Prediction via Multimodal Graph Reasoning and LLM Context Injection

    Sep 18, 2026Inyoung Choi, Sukwon Yun, Jiayi Xin +3Multimodal Large Language ModelsClinical Prediction

    University of Pennsylvania, Philadelphia, PA, USA · University of North Carolina at Chapel Hill, Chapel Hill, NC, USA