eess.IVApr 5, 2026
SaveBAAI Cardiac Agent: An intelligent multimodal agent for automated reasoning and diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging
Organizations: Beijing Academy of Artificial Intelligence, No. 150 Chengfu Road, Haidian District, Beijing 100084, China · Department of Radiology, Beijing Anzhen Hospital, Beijing Institute of Heart, Lung & Vascular Diseases, Capital Medical University, 2 Anzhen Road, Beijing 100029, China · Department of MR, the First Affiliated Hospital, Henan Medical University, 88 Jiankang Road, Weihui 453100, China · Department of Cardiology, Clinical Center for Coronary Heart Disease, Beijing Institute of Heart, Lung and Blood Vessel Disease, Beijing Anzhen Hospital, Capital Medical University, Beijing 100029, China ·
Abstract
Cardiac magnetic resonance (CMR) is a cornerstone for diagnosing cardiovascular disease. However, it remains underutilized due to complex, time-consuming interpretation across multi-sequences, phases, quantitative measures that heavily reliant on specialized expertise. Here, we present BAAI Cardiac Agent, a multimodal intelligent system designed for end-to-end CMR interpretation. The agent integrates specialized cardiac expert models to perform automated segmentation of cardiac structures, functional quantification, tissue characterization and disease diagnosis, and generates structured clinical reports within a unified workflow. Evaluated on CMR datasets from two hospitals (2413 patients) spanning 7-types of major cardiovascular diseases, the agent achieved an area under the receiver-operating-characteristic curve exceeding 0.93 internally and 0.81 externally. In the task of estimating left ventricular function indices, the results generated by this system for core parameters such as ejection fraction, stroke volume, and left ventricular mass are highly consistent with clinical reports, with Pearson correlation coefficients all exceeding 0.90. The agent outperformed state-of-the-art models in segmentation and diagnostic tasks, and generated clinical reports showing high concordance with expert radiologists (six readers across three experience levels). By dynamically orchestrating expert models for coordinated multimodal analysis, this agent framework enables accurate, efficient CMR interpretation and highlights its potentials for complex clinical imaging workflows. Code is available at https://github.com/plantain-herb/Cardiac-Agent.
Figures & tables
Figure 1: A schematic overview of the BAAI Cardiac Agent illustrating the process of radiology report generation from multi-sequence CMR scans (upper section), in comparison with the traditional clinical workflow (lower section). Upper section: the agent-assisted workflow, in which the user submits instructions and upload all CMR sequences to the BAAI Cardiac Agent. The agent automatically interprets the request, invokes appropriate analytic modules, and performs segmentation, quantitative assessment, and diagnostic reasoning. A structured, quantitative enriched radiology report is then generated within approximately 90 seconds. Lower section: the workflow of a traditional radiologist. This workflow begins with manual or semi-automatic segmentation of cine images, followed by qualification of key cardiac function parameters. Diagnostic interpretation is then performed by integrating information from late gadolinium enhancement (LGE) and rest myocardium perfusion imaging (Rest MPI), after which the final report is manually compiled. This sequential workflow typically requires approximately 1800 seconds.
| Metric | Method | SAX cine | 2CH cine | 4CH cine | SAX LGE |
|---|---|---|---|---|---|
| ResUNet++ | 84.69±0.05 | 86.97±0.04 | 84.41±0.06 | 58.08±0.16 | |
| nnUNet | 87.42±0.06 | 88.14±0.05 | 81.61±0.14 | 73.89±0.18 | |
| DSC | DiffUNet | 77.19±0.13 | 83.05±0.08 | 77.38±0.11 | 47.28±0.20 |
| MedSAM2 | 80.86±0.20 | 77.89±0.20 | 82.07±0.08 | 69.70±0.21 | |
| Ours | 90.21±0.02 | 88.75±0.03 | 86.92±0.03 | 75.07±0.07 | |
| ResUNet++ | 16.58±8.13 | 12.28±6.62 | 13.73±10.46 | 16.11±5.57 |
Table 1: Performance comparison between the BAAI Cardiac Agent segmentation expert model and SOTA segmentation networks. Computed DSC (%), HD, and ASD values are shown as mean ± standard deviation.
Figure 2: Performance of the expert CVDs diagnostic model in internal and external testing. a–c: Receiver operating characteristic curves for CDS; d–h: Receiver operating characteristic curves for NICMS. The red curves represent the internal test set, and the blue curves represent the external validation set. i: Confusion matrix of the AI diagnostic model predictions versus ground truth labels based on the full cohort (n = 2,232), with a color gradient visually representing the proportional distribution of predictions across disease categories.
| Internal Test Set | External Validation Set | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| N | Ours | VST | ViT | ResNet | N | Ours | VST | ViT | ResNet | |
| NH | 64 | 0.856 | 0.839 | 0.855 | 0.864 | 67 | 0.753 | 0.712 | 0.720 | 0.717 |
| IHD | 161 | 0.860 | 0.812 | 0.799 | 0.835 | 49 | 0.582 | 0.601 | 0.544 | 0.631 |
| NICM | 140 | 0.857 | 0.830 | 0.817 | 0.832 | 159 | 0.762 | 0.716 | 0.703 | 0.713 |
| F-W F 1 | 0.858 | 0.824 | 0.817 | 0.839 | 0.728 | 0.694 | 0.679 | 0.699 | ||
| Accuracy | 0.858 | 0.825 | 0.816 | 0.833 | 0.724 | 0.687 | 0.673 | 0.695 | ||
Table 2: Performance comparison of cardiac disease screening (CDS) in normal heart (NH), ischemic heart disease (IHD) and non-ischemic cardiomyopathy (NICM) cohorts between the BAAI Cardiac Agent CDS expert model and three SOTA models. (N: Number of subjects, F-W F 1 : Frequency-weighted F 1 )
| Internal Test Set | External Validation Set | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| N | Ours | VST | ViT | ResNet | N | Ours | VST | ViT | ResNet | |
| HCM | 72 | 0.899 | 0.865 | 0.848 | 0.888 | 43 | 0.769 | 0.729 | 0.719 | 0.748 |
| DCM | 26 | 0.800 | 0.798 | 0.813 | 0.800 | 91 | 0.812 | 0.711 | 0.721 | 0.714 |
| RCM | 18 | 0.732 | 0.762 | 0.815 | 0.762 | 6 | 0.444 | 0.483 | 0.567 | 0.567 |
| ACM | 13 | 0.636 | 0.636 | 0.727 | 0.727 | 8 | 0.421 | 0.657 | 0.435 | 0.421 |
| Myocarditis | 26 | 0.857 | 0.779 | 0.779 | 0.797 | 11 | 0.621 | 0.671 | 0.597 | 0.653 |
Table 3: Performance comparison between the BAAI Cardiac Agent expert diagnostic model and three SOTA models in the subclassification of non-ischemic cardiomyopathy, including hypertrophic cardiomyopathy (HCM), dilated cardiomyopathy (DCM), restrictive cardiomyopathy (RCM), arrhythmogenic cardiomyopathy (ACM) and myocarditis.
Figure 3: Evaluation results of BAAI Cardiac Agent report consistency and expert model invocation success rate. (a) Bland-Altman plots comparing BAAI Cardiac Agent measurements (left ventricular end-diastolic volume, LVEDV; left ventricular end-systolic volume, LVESV; left ventricular ejection fraction, LVEF; stroke volume, SV; left ventricular mass, LVM; left ventricular end-diastolic diameter, LVEDD) with manual reports in the internal cohort; (b) Mean distribution of left ventricular end-diastolic wall thickness (LVEDWT) measured by manual reports using the 17-SM and by the BAAI Cardiac Agent in the internal cohort. The bullseye plot correlates the basal, mid, and apical segments of the left ventricle with the outer, middle, and inner layers in sequence, with the apex at the center; (c) Success rate of expert model invocations by the BAAI Cardiac Agent on the internal and external test sets; (d) Accuracy performance of the BAAI Cardiac Agent in cardiac-specific VQA tasks.
| Model | Junior radiologists | Mid radiologists | Senior radiologists | |||
|---|---|---|---|---|---|---|
| Score | Conf. | Score | Conf. | Score | Conf. | |
| LLaVa-Med | 26.10±2.83 | 24.50 | 22.07±3.22 | 21.00 | 13.48±3.50 | 13.50 |
| MedM-VL | 27.93±6.12 | 23.00 | 23.85±6.81 | 22.00 | 15.65±5.50 | 19.00 |
| MMedAgent | 24.05±3.20 | 22.50 | 20.02±3.60 | 20.00 | 12.78±3.25 | 17.50 |
| Qwen-VL-30B | 58.52±3.73 | 52.50 | 57.18±4.68 | 49.50 | 51.67±5.75 | 46.00 |
| Ours | 87.93±2.20 | 94.00 | 87.52±2.84 | 90.50 | 86.53±4.21 | 87.50 |
Table 4: Performance comparison of different LMMs across radiologist experience levels (junior, middle and senior). Score denotes the comprehensive rating by radiologists of CMR report generation by different models, and Conf. denotes the increase in confidence in writing reports reported by radiologists after reviewing the model output.
Figure 4: Schematic diagram of the BAAI Cardiac Agent workflow, in which assigned tasks are completed through adaptive invocation and coordinated use of specialized medical image segmentation tools.
Figure 5: Task-modality-tool mapping of the BAAI Cardiac Agent: the left panel presents the tasks integrated in the agent and their corresponding medical imaging modalities, while the right panel lists the tools required to complete the respective imaging tasks.
Figure 6: The architecture of the proposed expert model for cardiac segmentation and CVDs diagnosis is described as follows: (a) It is a two-stage coarse-to-fine cardiac segmentation framework. In the first stage, a U-shaped backbone locates the ROI of the heart. In the second stage, based on the ROI output from the first stage, the framework fuses the global and local features of the heart to generate a refined, high-resolution cardiac segmentation map. (b) It is a two-stage automated screening and diagnostic framework for CVDs. In the first stage, the model leverages cine images (including 2CH, 4CH, and SAX views) to implement tri-class screening, categorizing subjects into three groups: NH, IHD, or NICM. For patients identified as having NICM via screening, the second stage further integrates their retained 4CH cine images, short-axis cine images, and SAX LGE images to output more refined diagnostic results for NICM subtypes. (c) It represents the overall network structure of the U-shaped backbone.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Internal Dataset | External Dataset | |||||||
|---|---|---|---|---|---|---|---|---|
| N | Male | Female | Age | N | Male | Female | Age | |
| Total | 2134 | 1558 (73%) | 576 (27%) | 47±20 | 279 | 171 (61%) | 108 (39%) | 49±17 |
| NH | 322 | 190 (59%) | 132 (41%) | 22±17 | 67 | 26 (39%) | 41 (61%) | 43±21 |
| IHD | 805 | 676 (84%) | 129 (16%) | 57±12 | 49 | 37 (76%) | 12 (24%) | 56±12 |
| NICM | 830 | 564 (68%) | 266 (32%) | 48±18 | 159 | 107 (67%) | 52 (33%) | 49±16 |
| HCM | 402 | 269 (67%) | 133 (33%) | 51±16 | 43 | 32 (74%) | 11 (26%) | 51±14 |
Table D1: Characteristics of the internal and external validation datasets (N: Number of subjects, Age in years: mean±SD (range))
| Metric | Method | LV Myo | LV Cavity | RV Cavity |
|---|---|---|---|---|
| ResUNet++ | 91.29±0.03 | 85.14±0.04 | 77.64±0.08 | |
| nnUNet | 88.19±0.04 | 92.44±0.03 | 81.63 ± 0.05 | |
| DSC | DiffUNet | 83.37±0.16 | 80.34±0.11 | 67.88±0.12 |
| MedSAM2 | 78.60±0.28 | 82.95±0.31 | 81.02±0.07 | |
| Ours | 92.44±0.03 | 96.19±0.02 | 82.00±0.05 | |
| ResUNet++ | 9.67±5.99 | 9.01±3.47 | 31.07±24.36 |
Table D2: A comparative study between the BAAI Cardiac Agent segmentation expert model and SOTA segmentation networks on the SAX cine dataset. We report DSC (%), HD, and ASD as mean ± standard deviation.
| Metric | Method | LV Myo | LV Cavity |
|---|---|---|---|
| ResUNet++ | 83.39±0.05 | 90.56±0.03 | |
| nnUNet | 85.49±0.03 | 90.79±0.04 | |
| DSC | DiffUNet | 78.92±0.09 | 87.17±0.07 |
| MedSAM2 | 87.17±0.04 | 68.61±0.40 | |
| Ours | 84.95±0.05 | 92.54±0.03 | |
| ResUNet++ | 13.84±10.82 | 10.71±4.28 |
Table D3: A comparative analysis of the BAAI Cardiac Agent segmentation expert model against current SOTA segmentation networks using the 2CH cine dataset. Results are presented as mean ± standard deviation for DSC (%), HD, and ASD.
| Metric | Method | LV Myo | LV Cavity | RV Myo | RV Cavity | LA Cavity | RA Cavity |
|---|---|---|---|---|---|---|---|
| ResUNet++ | 83.75±0.04 | 92.94±0.03 | 62.17±0.14 | 88.43±0.04 | 91.37±0.04 | 87.82±0.06 | |
| nnUNet | 82.34±0.06 | 92.23±0.03 | 57.64±0.14 | 85.39±0.04 | 86.69±0.11 | 85.37±0.09 | |
| DSC | DiffUNet | 78.31±0.06 | 88.02±0.07 | 54.81±0.15 | 79.29±0.11 | 86.32±0.10 | 77.51±0.17 |
| MedSAM2 | 86.57±0.03 | 62.87±0.44 | 70.54±0.07 | 90.26±0.02 | 91.61±0.05 | 90.54±0.04 | |
| Ours | 86.12±0.05 | 94.25±0.02 | 68.77±0.09 | 89.93±0.03 | 92.23±0.04 | 90.22±0.05 | |
| ResUNet++ | 15.81±22.46 | 16.43±25.82 | 10.61±4.41 | 10.45±4.79 | 7.27±4.52 | 21.81±33.01 |
Table D4: A comparative study of the BAAI Cardiac Agent segmentation expert model versus SOTA segmentation networks on the 4CH cine dataset. We report DSC (%), HD, and ASD in the format of mean ± standard deviation.
| Metric | Method | LV Myo | LV Cavity | LGE |
|---|---|---|---|---|
| ResUNet++ | 66.40±0.12 | 81.72±0.09 | 26.13±0.26 | |
| nnUNet | 74.12±0.10 | 87.36±0.06 | 60.19±0.21 | |
| DSC | DiffUNet | 53.09±0.15 | 58.85 ± 0.25 | 29.88±0.19 |
| MedSAM2 | 70.07±0.27 | 72.27±0.34 | 65.37±0.23 | |
| Ours | 73.67±0.08 | 88.60±0.05 | 60.83±0.13 | |
| ResUNet++ | 13.13±5.60 | 9.88±4.35 | 26.68±14.25 |
Table D5: A comparative analysis of the BAAI Cardiac Agent segmentation expert model and leading segmentation networks evaluated on the SAX LGE dataset. Performance metrics (DSC (%), HD, and ASD) are reported as mean ± standard deviation.
Fig. D1: Qualitative comparison examples of segmentation results between the segmentation expert model we proposed and existing SOTA methods. From top to bottom, four image examples for each of 2CH cine, 4CH cine, SAX cine, and SAX LGE are presented in sequence, with the labeled values indicating DSC results. (Best viewed in color)
Fig. D2: Cumulative LGE degree in the LV 17-SM: BAAI Cardiac Agent vs physician annotations. (Best viewed in color)
Fig. D3: Consistency evaluation between the BAAI Cardiac Agent and clinical reports. (a) Bland-Altman plots compare cardiac output (CO), right ventricular end-diastolic diameter (RVEDD), and left/right atrial maximum transverse diameters (LAT4CHD, RAT4CHD) measured by the BAAI Cardiac Agent with manual clinical reports in the internal cohort; (b) Mean left ventricular end-diastolic wall thickness (LVEDWT) distributions from manual 17-segment model (17-SM) reports and the BAAI Cardiac Agent in internal normal and hypertrophic cardiomyopathy cohorts. The bullseye plot sequentially maps left ventricular basal, mid, and apical segments to the plot’s outer, middle, and inner layers, with the center representing the apex.
Fig. D4: Schematic diagram of a complete medical report generated by the BAAI Cardiac Agent, illustrating the measurement of cardiac function parameters and imaging diagnostic analysis for a patient with dilated cardiomyopathy (DCM).
Fig. D5: Qualitative comparison between BAAI Cardiac Agent and other SOTA methods across different tasks. Undesired and desired responses are highlighted in licorice yellow and pine green, respectively.
More in this topic
Medical Diagnosis
Medical diagnosis uses clinical data, tests, and images to identify diseases, assess risk, and guide care.
See all 255 papers