Operation-aware Multimodal Clinical Decision Support System for ICU Discharge Risk Assessment
Organizations: Graduate School of Informatics, Kyoto University, Kyoto, Japan
Abstract
Unplanned 30-day intensive care unit (ICU) readmission is a critical benchmark for evaluating healthcare quality. Most predictive models assume readmission risk depends solely on a patient's individual pathophysiological state, ignoring hospital operational context and resource capacity. In practice, discharge decisions are dynamic trade-offs between patient stability and bed turnover pressures. To address this gap, we develop an Operation-aware Multimodal Clinical Decision Support System (CDSS) for ICU discharge risk assessment. The system combines multi-agent collaborative extraction with self-evolving retrieval-augmented generation (RAG) to eliminate template noise from progress notes, while constructing departmental workload indicators alongside admission flow features, continuous vital signs, and sparse medical codes (ICD/DRG). Addressing severe class imbalance (about 10.5% readmissions) and alarm fatigue, we implement data normalization, class reweighting, and decision-threshold calibration. In 20 independent runs across 20,000 ICU stays from MIMIC-III, empirical findings demonstrate that: vital signs alone have limited predictive capacity under real-world class imbalance (PR-AUC of 0.1387 +/- 0.0086); directly concatenating raw text embeddings degrades decision tree performance (XGBoost and LightGBM PR-AUC drop to 0.3893 and 0.4030); whereas an end-to-end multimodal fine-tuning architecture (Transformer E2E FineTune) utilizing layer-wise parameter freezing and differential learning rates reverses text negative transfer, achieving the highest AUROC of 0.8288 +/- 0.0110, PR-AUC of 0.4373 +/- 0.0310, and F1-Score of 0.4757 +/- 0.0196. Operational indicators supply independent risk signals, laying an engineering foundation for high-utility clinical decision support tools.
Figures & tables
| Model Category | Model Name & Variant | Optimal Threshold | AUROC | PR-AUC | F1-Score |
|---|---|---|---|---|---|
| Tree Baselines & Text Ablation | XGBoost Base | ||||
| XGBoost + LLM Notes | |||||
| LightGBM Base | |||||
| LightGBM + LLM Notes | |||||
| Deep Multimodal Networks & Fusion | LSTM Base (Pure Structured) | ||||
| LSTM LateFusion (+ Notes) |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Characteristic / Feature | Overall Cohort ( ) | Readmitted ( , ) | Non-Readmitted ( , ) | -Value |
| Patient Demographics and Baseline Covariates | ||||
| Age at Admission (years) | ( ) | ( ) | ( ) | |
| Biological Sex | ||||
| — Female | ||||
| — Male | ||||
| Ethnicity Breakdown | ||||
| Clinical Domain | Risk Task | Rule Specification |
| Cardiovascular | Readmission | [Readmission Rule]: In post-CABG or post-valve surgery patients, the emergence of new-onset atrial fibrillation with rapid ventricular response represents an acute readmission hazard even if pharmacological or electrical cardioversion achieved transient sinus rhythm. |
| Cardiovascular | Mortality | [Mortality Rule]: In acute myocardial infarction complicated by cardiogenic shock, persistent requirement for two or more concurrent vasoactive infusions (e.g., Norepinephrine Epinephrine) indicates extreme in-hospital mortality regardless of baseline LVEF. |
| Respiratory | Readmission | [Readmission Rule]: If a patient was extubated within 24 hours prior to ICU discharge but presents residual tachypnea ( ) or requires escalating high-flow nasal cannula/BiPAP requirements, assign an elevated readmission risk. |
| Respiratory | Mortality | [Mortality Rule]: If a patient treated for acute respiratory distress exhibits documented bilateral radiographic clearance and stable oxygenation on minimal supplemental oxygen ( ), do not score high mortality risk solely based on initial intubation status. |
| Neurology | Mortality | [Mortality Rule]: When physical examination satisfies brain death criteria or presents profound hypoxic-ischemic encephalopathy with absent brainstem reflexes, assign maximum mortality risk regardless of hemodynamic stability. |
| Neurology | Readmission | [Readmission Rule]: Residual delirium, agitation, or confusion without acute focal neurological deficits increases nursing care friction but does not warrant elevated ICU readmission risk unless complicated by imminent airway compromise. |
| Component / Module | Hyperparameter Name | Value / Specification | Rationale / Design Justification |
| Cohort Splits | Total Cohort Size | aligned ICU stays | Filtered MIMIC-III adult stays with 48h data |
| Split Proportion | 70% Train / 10% Val / 20% Test | 14,000 / 2,000 / 4,000 sample partitioning | |
| Randomization Seeds | 20 runs (Seeds 42–61) | Eliminates split-specific bias in evaluation | |
| Loss Re-weighting | Dynamically matches negative-to-positive ratio in train split | ||
| Training Optimization | Maximum Epochs | epochs | Early stopping with patience on validation loss |
| Mini-Batch Size | (End-to-End: ) | Balances throughput with gradient signal stability |
| Model Architecture | Cutoff | AUROC | PR-AUC | Precision | Recall | F1-Score | Brier Score |
|---|---|---|---|---|---|---|---|
| Tree Baselines and Shallow Text Concatenation | |||||||
| XGBoost Base | |||||||
| XGBoost + LLM Notes | |||||||
| LightGBM Base | |||||||
| LightGBM + LLM Notes | |||||||
| Deep Multimodal Neural Networks | |||||||
| Model Architecture | AUROC | PR-AUC | Precision | Recall | F1-Score | Accuracy |
|---|---|---|---|---|---|---|
| XGBoost Base | ||||||
| XGBoost + LLM Notes | ||||||
| LightGBM Base | ||||||
| LightGBM + LLM Notes | ||||||
| LSTM Base | ||||||
| LSTM LateFusion (+ Notes) |
Clinical Decision Support
Clinical Decision Support uses data, medical knowledge, and AI to help clinicians assess patients and choose tests, diagnoses, and treatments.
See all 254 papers