cs.AIMar 23, 2026
SaveCerebra: A Multidisciplinary AI Board for Multimodal Dementia Characterization and Risk Assessment
Organizations: Department of Biomedical Data Science, Stanford University · Department of Radiation Oncology, Stanford University · Center for Data Science, New York University · J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida · Department of Biomedical Engineering and Informatics, Indiana University Indianapolis · Department of Neurology, NYU Grossman School of Medicine · Department of Neuroscience, NYU Grossman School of Medicine · UF Health Family Medicine – Haile Plantation · Department of Family Medicine, Indiana University School of Medicine · Department of Biostatistics and Health Data Science, Indiana University School of Medicine · Center for Biomedical Informatics, Regenstrief Institute · Scripps Research Translational Institute · Courant Institute, New York University · Center for Cognitive Aging and Memory, McKnight Brain Institute, University of Florida · Population Health Department, NYU Langone Health · Radiology Department, NYU Langone Health
Abstract
Modern clinical practice increasingly depends on reasoning over heterogeneous, evolving, and incomplete patient data. Although recent advances in multimodal foundation models have improved performance on various clinical tasks, most existing models remain static, opaque, and poorly aligned with real-world clinical workflows. We present Cerebra, an interactive multi-agent AI team that coordinates specialized agents for EHR, clinical notes, and medical imaging analysis. These outputs are synthesized into a clinician-facing dashboard that combines visual analytics with a conversational interface, enabling clinicians to interrogate predictions and contextualize risk at the point of care. Cerebra supports privacy-preserving deployment by operating on structured representations and remains robust when modalities are incomplete. We evaluated Cerebra using a massive multi-institutional dataset spanning 3 million patients from four independent healthcare systems. Cerebra consistently outperformed both state-of-the-art single-modality models and large multimodal language model baselines. In dementia risk prediction, it achieved AUROCs up to 0.80, compared with 0.74 for the strongest single-modality model and 0.68 for language model baselines. For dementia diagnosis, it achieved an AUROC of 0.86, and for survival prediction, a C-index of 0.81. In a reader study with experienced physicians, Cerebra significantly improved expert performance, increasing accuracy by 17.5 percentage points in prospective dementia risk estimation. These results demonstrate Cerebra's potential for interpretable, robust decision support in clinical care.
Figures & tables
Figure 1: Overview of the CEREBRA agentic AI system for clinical decision support. a. Clinicians interact with CEREBRA through an interactive dashboard that accepts queries and feedback. Modality-specific agents analyze patient data and participate in a structured multi-agent discussion coordinated by a summary agent to produce a final conclusion. b. A super agent moderates the data agent and modality agents. c–e. Modality agents operating under an independent workflow, training models, and producing modality-specific evidence and results. f. The summary agent performs a multi-agent discussion in a propose-and-critique style. g. Multi-institutional data integration and cohort construction. h. CEREBRA supports various clinical tasks, including risk prediction, survival analysis, diagnostic assistance with human-in-the-loop.
Figure 2: Dementia risk prediction performance of CEREBRA . a. Dementia risk prediction on the NYU cohort for prediction horizons of 1, 2, and 3 years. Performance is measured using AUROC (left) and AUPRC (right), comparing CEREBRA with MedGemma, GPT-4o, and GPT-5 baselines. b. External validation across independent healthcare systems (NYU Langone Hospital - Long Island, Indiana University, and University of Florida). Heatmaps show AUROC and AUPRC for 1-, 2-, and 3-year prediction tasks, with cohort statistics shown below; rows labeled 1 and 0 denote the number of positive (dementia diagnosis onset) and negative (no dementia diagnosis) samples, respectively. c. Kaplan–Meier survival analysis stratified by CEREBRA -predicted risk groups (top and bottom 25%). d. C-index from Cox proportional hazards modeling performed by CEREBRA across modalities (EHR, clinical notes, imaging) and their multimodal combination. Error bars indicate uncertainty estimated via bootstrap resampling of the test set (1,000 replicates); error bars indicate 1 standard deviation for AUROC and AUPRC.
| Category Label | Acronym | Description |
| NC | NC | Normal cognition |
| MCI | MCI | Mild cognitive impairment |
| AD/ADRD | AD | Alzheimer’s disease |
| LBD | Lewy body dementia, including dementia with Lewy bodies and Parkinson’s disease dementia | |
| VD | Vascular dementia, vascular brain injury and vascular dementia, including stroke | |
| FTD | Frontotemporal lobar degeneration and its variants, including primary progressive, aphasia, corticobasal degeneration and progressive supranuclear palsy, and with or without amyotrophic lateral sclerosis |
Table 1: Glossary related to cognitive conditions
Figure 3: Dementia diagnosis performance of CEREBRA in NYU cohort . a-b. Receiver operating characteristic (ROC) curves and Precision–recall curves for dementia diagnosis on the NYU cohort, comparing CEREBRA with MedGemma, GPT-4o, and GPT-5. c-d. Performance stratified by sex, shown using the area under the ROC curve (AUROC) and the area under the precision–recall curve (AUPRC). e-f. Performance stratified by dementia subtype (Alzheimer’s disease, Lewy body dementia, vascular dementia, frontotemporal dementia, and others), shown using AUROC and AUPRC. Each dot denotes one bootstrap estimate of model performance ( resamples); boxplots summarize the distribution of these bootstrap estimates.
Figure 4: Evaluation of multimodal fusion and missing-modality robustness in CEREBRA . a. Multimodal inputs used by CEREBRA , including structured EHR data, clinical notes, and medical images (OCT and brain MRI), with the number of samples available for each modality. b–c. Risk prediction (3 Year) performance comparing heuristic fusion methods based on minimum (lowest-risk), maximum (highest-risk), and average risk scores with agentic equal discussion and propose-and-critique style discussion. d. Correlation between modality agents’ risk scores, showing complementary (non-redundant) risk assessments across image, EHR, and note agents. e. Performance of CEREBRA on the NYU cohort with missing modalities.
Figure 5: Feedback loop and reasoning evidence analysis. a. Demonstration of summary agent pipeline with the dynamic medical notebook. Summary agent collects outputs from modality agents for a multi-agent discussion, while the aggregated risk score is properly calibrated with the prior feedback provided by clinicians in the dynamic medical notebook. b. Performance in AUROC on dementia prediction task improves with increasing number of automatically generated feedback from erroneous predictions. c. CEREBRA ’s reasoning achieves significantly better agreement with reference clinical notes than GPT-4o. d. Qualitative analysis on the most frequent positive keywords across time in CEREBRA reasoning. Different time horizons manifest differential clinical evidence characteristics, as well as diversified medical keywords.
Figure 6: Clinician reader study evaluating CEREBRA . a. Example dashboard interface summarizing predicted 3-year dementia risk, extracted risk factors, supporting evidence and suggested actions. b. Aggregate performance of clinician risk assessments with versus without dashboard assistance (accuracy, sensitivity and specificity; six clinicians; 40 cases per clinician; 20 per condition; case reads per condition). Percent-point changes are annotated. c. Self-reported confidence distribution (low, moderate, high) with versus without dashboard assistance ( per condition). d. Clinician’s performance (sensitivity) stratified by predicted diagnostic category across major dementia etiologies and an Others category. e. Clinician evaluation of dashboard content and utility for dashboard-assisted reads ( ): perceived accuracy and relevance of risk factors, accuracy and alignment of supporting evidence, and perceived helpfulness for understanding risk and easing decision-making.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| NYU | LI | UF | INPC | ||||||||||
| EHR | Note | Image | EHR | Note | Image | EHR | Note | Image | EHR | Note | Image | ||
| Initial Cohort | Patient | 2M | 521K | 241K | 306K | 7208 | 2934 | 42K | 43K | 54K | 163K | 115K | - |
| Record | 4B | 22M | 2M | 40M | 209K | 16K | 33M | 363K | 176K | 300M | 11M | - | |
| Applying filter with inclusion Criteria | |||||||||||||
| Filtered Cohort | Patient | 53K | 508 | 4369 | 37K | ||||||||
| Record | 369M | 1M | 139K | 3M | 11k | 3,349 | 6M | 96K | 29K | 158M | 5M | - | |
Supplementary Table 1 : Number of patients and records per modality across institutions, before and after applying inclusion criteria filtering.
| Institution (group) | Age (y), mean ± s.d. | Male, n(%) | Race (White, Black, Asian, Pacific, multirace, unknown), n |
| NYU | |||
| NC [n=42289] | 75.60 ± 7.73 | 17104 (40.45%) | (26707, 3960, 1804, 76, 4219, 5523) |
| AD [n=5588] | 82.03 ± 7.98 | 2486 (44.49%) | (3842, 415, 246, 16, 558, 511) |
| VD [n=1336] | 82.96 ± 8.28 | 594 (44.46%) | (834, 165, 71, 5, 149, 112) |
| LBD [n=3588] | 79.12 ± 7.96 | 1703 (47.46%) | (2551, 268, 127, 10, 337, 295) |
| FTD [n=970] | 79.75 ± 7.78 | 548 (56.49%) | (705, 54, 37, 0, 75, 99) |
Supplementary Table 2 : Study population by institution and diagnostic group with dementia subtypes. Data were drawn from three health systems, including NYU, LI, UF, and INPC (accessed through the Regenstrief Institute). Diagnostic group acronyms are defined in Table 1 .
Supplementary Figure 1 : Dataset curation details. (a) Diagnosis Dataset by Backtracking. Positive: the first positive visit date is the end date of observation window; Negative: randomly select a visit date as the end date of observation window. (b) Time-to-Event Survival Analysis, indexed by MRI Scan Visits. Positive: find the time from MRI scan visits to the first positive diagnosis; Negative: find the time from MRI scan visits to the end of record. (c)Prediction dataset by rolling window. Positive: There is meaningful new data within the observation window, and there is no data leakage; otherwise Negative .
| Condition | Criteria (ICD-10 codes and Medications) |
| AD/ADRD | F01.*: Any vascular dementia |
| F02.*: Dementia in other diseases classified elsewhere with or without behavioral disturbance | |
| F03.*: Unspecified dementia with/without behavioral disturbance | |
| F04.*: Amnestic disorder due to known physiological condition | |
| G23.1: Progressive supranuclear palsy | |
| G30.*: Any Alzheimer’s disease |
Supplementary Table 3 : Definition criteria for AD/ADRD/MCI case identification.
| Subtype | Criteria (ICD-10 codes and Medications) |
| Alzheimer’s Disease | G30.*: Any Alzheimer’s disease |
| DONEPEZIL | |
| RIVASTIGMINE | |
| GALANTAMINE | |
| MEMANTINE | |
| TACRINE |
Supplementary Table 4 : Definition criteria for clinical dementia subtype classification.
| Institution | Modality | Metric | 1-year | 2-year | 3-year |
| NYU | EHR | AUROC | 71.9 | 70.4 | 73.5 |
| AUPRC | 9.5 | 12.2 | 17.4 | ||
| Notes | AUROC | 72.4 | 71.9 | 73.4 | |
| AUPRC | 6.8 | 9.8 | 14.2 | ||
| Images - Volume | AUROC | 71.9 | 70.7 | 69.2 | |
| AUPRC | 6.4 | 11.0 | 14.5 |
Supplementary Table 5 : Modality agent’s performance of dementia prediction task with EHR, notes and images respectively across NYU, LI, INPC, and UF. The 1/2/3-year horizons indicate whether, based on the available historical data, an individual will be labeled as developing dementia in the coming 1/2/3 years (label window) after 6 months prediction window. For each dataset, positive/negative (P/N) sample ratio and positive rate are also reported. Image modality results for the INPC cohort is not reported because the dataset does not contain imaging data. For NYU, we also benchmarked image agent performance using VoCo as deep learning approach. Best AUROC and AUPRC for each time horizon at each site are highlighted.
| Institution | Modality | AUROC | AUPRC | P/N samples | Positive rate (%) |
| NYU | EHR | 82.1 | 38.9 | 239/2177 | 9.89 |
| Notes | 80.5 | 34.2 | |||
| Images | 71.4 | 24.4 | |||
| CEREBRA | 84.6 | 41.4 | |||
| LI | EHR | 77.6 | 14.3 | 59/1211 | 4.65 |
| Notes | 72.1 | 14.3 |
Supplementary Table 6 : Performance of dementia diagnosis task across four sites for each modality, compared with CEREBRA performance. For LI, evaluation is performed using the models trained with NYU data, showcasing the generalization on an external evaluation dataset without retraining. Best AUROC and AUPRC for each time horizon are highlighted.
| Modality | Model | Metric | 1-year | 2-year | 3-year |
| EHR | GPT-OSS-120B | AUROC | 56.13 | 56.77 | 56.62 |
| AUPRC | 15.35 | 24.44 | 37.52 | ||
| CEREBRA | AUROC | 62.34 | 64.53 | 66.81 | |
| AUPRC | 20.65 | 31.23 | 50.35 | ||
| Note | GPT-OSS-120B | AUROC | 62.42 | 63.17 | 64.61 |
| AUPRC | 18.31 | 32.45 | 44.93 |
Supplementary Table 7 : Performance of MCI to ADRD conversion risk prediction task across different time horizons in the INPC cohort for each modality, compared with CEREBRA (using GPT-OSS-120B as the backbone) performance. Best AUROC and AUPRC for each time horizon are highlighted.
| Modality | Model | Metric | diagnosis | 1-year | 2-year | 3-year |
| EHR | MedGemma | AUROC | 61.2 | 44.1 | 53.6 | 53.8 |
| AUPRC | 13.8 | 4.6 | 4.3 | 6.1 | ||
| GPT-4o | AUROC | 73.2 | 60.4 | 60.6 | 63.1 | |
| AUPRC | 18.3 | 3.7 | 6.0 | 8.1 | ||
| GPT-5 | AUROC | 74.6 | 60.9 | 62.9 | 62.7 | |
| AUPRC | 18.5 | 3.7 | 6.5 | 8.0 |
Supplementary Table 8 : LLM baseline results on the NYU cohort for dementia diagnosis and prediction among cognitively normal patients at 1-, 2-, and 3-year horizons. For each modality and task, the best-performing result is highlighted in bold.
| Modality | Model | C-index |
| EHR | MedGemma | 50.0 |
| GPT-4o | 64.9 | |
| GPT-5 | 63.1 | |
| CEREBRA | 78.2 | |
| Note | MedGemma | 53.6 |
| GPT-4o | 58.0 |
Supplementary Table 9 : LLM baseline results on time-to-event survival analysis on the NYU cohort. For each modality, the best-performing result is highlighted in bold.
Supplementary Figure 2 : Top features extracted by modality agents for 3-year normal cognition to dementia prediction task. These features are determined by respective modality agents to have positively contributed to dementia.
Supplementary Figure 3 : Interactive interface of CEREBRA . Cerebra provides a conversational interface in the dashboard that enables clinicians to interact with CEREBRA , exploring the reasoning behind the decision and providing feedback to guide the system.
Supplementary Figure 4 : CEREBRA dashboard, modality supporting evidence and recommendation sections.