CineMR: Tool-Integrated Vision-Language Reasoning for Quantitative Cardiac MRI Assessment

Kunyang Li; Hai Nguyen; Joshua Lowe; Chenguang Zhao; Peace C. Madueme; Mehdi Hedjazi Moghari; Mubarak Shah; Pegah Khosravi

cs.CVOct 1, 2026

CineMR: Tool-Integrated Vision-Language Reasoning for Quantitative Cardiac MRI Assessment

Authors: Kunyang Li, Hai Nguyen, Joshua Lowe, Chenguang Zhao, Peace C. Madueme, Mehdi Hedjazi Moghari, Mubarak Shah, Pegah Khosravi, +1 more

Organizations: Institute for Artificial Intelligence, University of Central Florida · Department of Clinical Sciences, College of Medicine, University of Central Florida · Nemours Cardiac Center, Nemours Children’s Hospital, Florida · Children’s Heart Center, WVU Golisano Children’s, West Virginia

Abstract

Cardiovascular magnetic resonance (CMR), including cine imaging, is a reference standard for the noninvasive assessment of cardiac morphology and ventricular function. Cine CMR interpretation integrates qualitative visual assessment with quantitative measurements of ventricular volumes, ejection fraction, myocardial mass, wall thickness, and regional wall motion. Current medical vision-language models (VLMs) cannot reliably derive quantitative measurements from multidimensional cine images without analysis tools. We present CineMR, a tool-augmented VLM that invokes cardiac image-analysis tools and integrates their outputs into interleaved reasoning for quantitative CMR assessment. We also construct a multi-cohort visual question answering benchmark covering quantitative metric extraction, multiclass diagnosis, and differential diagnosis, together with tools for segmentation, phase selection, volumetry, morphometry, and regional wall motion analysis. CineMR is trained with supervised fine-tuning (SFT) on tool-interaction traces followed by Group Relative Policy Optimization (GRPO) with conditional tool-use rewards. On the multi-cohort cine CMR benchmark, CineMR achieves 35.9% pass@1 and 58.9% pass@4, compared with 1.5% pass@1 for the Qwen3-VL-8B backbone and 0.0% and 7.0% pass@1 for LLaVA-Med v1.5 and MedGemma-4B, respectively. Correct tool invocation reaches 99.8% after GRPO, up from 78.9% after SFT. Live tool outputs improve ventricular measurement accuracy by 20.4--23.7% over direct model predictions, and removing all tools reduces pass@1 from 35.9% to 27.9%. These results highlight the importance of reliable tool use for quantitative cine CMR reasoning and support CineMR as a promising approach for assistive cardiac image assessment. Code, benchmark resources, and model weights are available at https://github.com/AI-MIND-Lab/CineMR.

Figures & tables

Figure 1 : Comparison of model outputs for the same cine CMR case, where the benchmark reference category is dilated cardiomyopathy (DCM). In this illustrative example, LLaVA-Med fails to provide a definitive assessment, whereas MedGemma produces unsupported tool-call content and repetitive text. In contrast, CineMR invokes executable tools to obtain quantitative measurements and evaluates them according to criteria encoded in the study-specific Rule-Based Cardiac Phenotype Classifier, thereby producing a traceable output grounded in explicit measurements and decision rules.

Panel of Figure 2 — Figure 2 : Overview of the CineMR framework, designed to mirror the real-world cardiologist workflow. Just as a clinician first inspects the cine images to form a visual impression, CineMR generates an initial phenotypic hypothesis via chain-of-thought reasoning. The clinician then orders quantitative analyses (segmentation, volumetry, wall thickness measurement); analogously, CineMR invokes a cardiology-specific quantitative tool suite to obtain precise metrics (EDV, ESV, EF, mass, MVR, wall thickness, sphericity index, regional wall motion). Finally, where the clinician interprets these measurements against ESC/AHA guideline thresholds to reach a diagnosis, CineMR routes them through a guideline-aligned Rule-Based Cardiac Phenotype Classifier with threshold-gated branching, producing an auditable reasoning chain that parallels the clinical diagnostic process from visual assessment to quantitative verification to evidence-based conclusion.

Figure 3 : Representative verbatim CineMR rollout for a cine CMR case assigned to the DCM category. The figure shows the original benchmark prompt, model-generated tool calls, executable tool outputs, intermediate generated rationale, and final predicted category. CineMR invokes cardiac segmentation, ventricular volumetry, and morphometric analysis tools before applying the published and study-specific criteria encoded in the Rule-Based Cardiac Phenotype Classifier. Tool results are displayed at their native quantitative precision. The intermediate rationale and final response are reproduced verbatim and should not be interpreted as independently validated clinical conclusions.

Figure 4 : The tools produce the quantitative measurements required by the study-specific Rule-Based Cardiac Phenotype Classifier. Final classification includes the following categories: hypertrophic cardiomyopathy (HCM), athlete’s heart or physiological athletic remodeling, arrhythmogenic right ventricular cardiomyopathy (ARVC), left ventricular non-compaction (LVNC), ischemic heart disease (IHD), myocardial infarction (MINF), dilated cardiomyopathy (DCM), hypertensive heart disease (HHD), normal (NOR), and indeterminate.

Tool	Input $\rightarrow$ Output	Unlocked Metrics	Role
segment_cardiac	$\mathcal{I}\rightarrow\mathbf{s}$	LV/RV/MYO masks	prerequisite
measure_volume	$\mathbf{s}\rightarrow\mathbf{v}$	EDV, ESV, EF, EDVi	computational
compute_cardiac_metrics	$(\mathbf{s},\mathbf{v})\rightarrow\mathbf{m}$	mass, MVR, EDWT, S:L, SI	computational
wall_motion_analysis	$\mathbf{s}\rightarrow\mathbf{w}$	segment-level WMA	computational
select_cardiac_phases	$\mathbf{s}\rightarrow(t_{\text{ED}},t_{\text{ES}})$	temporal grounding	auxiliary
zoom_anatomy	$(t,\text{target})\rightarrow\text{ROI}$	local visual detail	auxiliary

Table 1 : Cardiology-specific tool suite. Each tool maps to Rule-Based Cardiac Phenotype Classifier nodes requiring quantitative thresholds inaccessible to visual inspection.

	ACDC			M&Ms			M&Ms-2			Overall
Method	P@1	P@4	Mean	P@1	P@4	Mean	P@1	P@4	Mean	P@1	P@4	Mean
LLaVA-Med v1.5	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
MedGemma-4B	5.2	16.1	5.5	7.7	19.1	7.4	6.7	17.0	6.3	7.0	18.0	6.7
Qwen3-VL-8B	1.3	4.9	1.4	1.4	4.2	1.4	1.9	4.6	1.4	1.5	4.5	1.4
SFT	15.2	28.3	15.6	26.5	41.6	26.3	24.2	42.0	24.4	23.6	38.9	23.6
GRPO	33.9	58.0	34.7	36.2	58.0	35.4	36.9	62.0	37.3	35.9	58.9	35.7

Table 2 : Held-out test performance by cohort (

n=3{,}320

; ACDC: 696, M&Ms: 1,846, M&Ms-2: 778). All methods use

k=4

rollouts with temperature 0.7. SFT and GRPO correspond to the matched pre- and post-RL models under the same live-tool protocol. GRPO consistently improves performance over SFT across all cohorts, yielding the best overall results.

	MedGemma-4B			Qwen3-VL-8B			CineMR (SFT)			CineMR (GRPO)
Slice	P@1	P@4	Mean	P@1	P@4	Mean	P@1	P@4	Mean	P@1	P@4	Mean
Layer 1	1.9	7.5	2.2	2.9	8.3	2.6	6.5	20.2	6.4	10.8	31.5	10.1
Layer 2	17.6	38.8	17.0	0.5	1.7	0.4	27.6	33.0	27.3	65.3	92.0	65.3
Layer 3	8.4	23.3	7.2	0.6	1.9	0.6	55.2	83.7	55.8	60.9	86.7	60.7
Layer 4	2.8	15.0	4.2	0.0	1.9	0.5	50.5	78.5	52.1	45.8	78.5	50.0
Layer 5	0.0	0.0	0.0	0.0	0.0	0.0	12.9	33.3	12.6	13.3	32.5	13.0

Table 3 : Layer-wise performance of SFT (pre-RL) and GRPO (post-RL) with live tool execution, compared with the off-the-shelf MedGemma-4B-it and Qwen3-VL-8B baselines. Results are reported by cognitive layer on the held-out test set (

n=3{,}320

). GRPO improves over SFT across most cognitive layers, with the largest gains observed in Layers 2–3.

Panel of Figure 5 — Figure 5 : Tool-use diagnostics and tool-withholding ablation. (a) Correct tool-call rate for SFT and GRPO on gold tool-requiring items. (b) GRPO accuracy of direct model predictions versus live measure_volume outputs on ventricular volume and ejection-fraction questions. (c) GRPO pass@1 on the held-out test set ( $n=3{,}320$ , $k=4$ ) with full tool access, without computation tools, and without any tools; error bars show 95% bootstrap confidence intervals. GRPO raises correct tool invocation from 78.9% to 99.8%; live tool outputs improve measurement accuracy by 20.4–23.7%, and removing all tools lowers pass@1 from 35.9% to 27.9%.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Acronym	Definition
LV / RV	Left / right ventricle (ventricular)
LVEF / RVEF	Left / right ventricular ejection fraction
LVEDV / LVESV	LV end-diastolic / end-systolic volume
RVEDV / RVESV	RV end-diastolic / end-systolic volume
EDWT	End-diastolic wall thickness
S:L ratio	Septal-to-lateral wall thickness ratio

Appendix

Table 4 : Acronym glossary.

Template ID	Layer	Tools	Question
L1_lvef	L1	✓	What is the left ventricular ejection fraction (LVEF) for this patient? Report as a percentage and indicate whether it is normal, mildly reduced, reduced, or severely reduced.
L1_lvedv	L1	✓	Report the left ventricular end-diastolic volume (LVEDV) and end-systolic volume (LVESV) for this cardiac MRI study, including BSA-indexed values if anthropometric data are available.
L1_rvedv	L1	✓	What are the right ventricular end-diastolic volume (RVEDV), end-systolic volume (RVESV), and RV ejection fraction (RVEF) for this patient?
L1_lv_mass	L1	✓	Calculate the left ventricular myocardial mass from this cardiac MRI segmentation. Report the absolute LV mass in grams.
L1_rvef	L1	✓	What is the right ventricular ejection fraction (RVEF) for this patient? Is it within normal limits?
L1_max_edwt	L1	✓	What is the maximum end-diastolic wall thickness (max EDWT) of the left ventricle in this cardiac MRI, and does it exceed the threshold for hypertrophy?

Appendix

Table 5 : All 19 evaluation question templates, by cognitive layer. Tools : whether tool use is required (✓) or optional (--) for that template; a template may override its layer’s default (e.g. L2_is_asymmetric requires tools because the septal:lateral ratio must be computed).

Explore similar work

Sep 30, 2026cs.CV

Learning Where to Look: Anatomical Grounding and Guided Attention for Cardiac MRI Vision-Language Models

Cardiac magnetic resonance imaging (CMR) enables assessment of cardiac anatomy, ventricular function, and myocardial tissue characteristics. Clinicians interpret these images by identifying cardiac structures and focusing on the regions relevant to each clinical question, motivating anatomically guided vision-language models (VLMs). Yet CMR-specific supervision for anatomical localisation and clinical question answering remains limited. To address this gap, we investigate fine-grained CMR visual question answering through anatomical grounding and guided attention. We construct 128,915 anatomical-grounding and 42,799 clinical QA pairs across short-axis cine, late gadolinium enhancement, and long-axis cine. These datasets support anatomical recognition, localisation, and clinical assessment without requiring paired reports for individual training images. To help the model learn where to look, we introduce Cardiac Anatomy-Routed Attention (CARA), which selects predicted anatomical priors according to the question and guides decoder attention with learned task-specific strengths. Combining anatomical grounding pretraining with CARA yields our model, CARA-VL. Experiments demonstrate CARA-VL's strengths in clinical assessment and regional localisation across CMR imaging settings, with promising generalization to an external clinical cohort. Together, our data and method provide a practical framework for studying and advancing cardiac visual understanding in VLMs. We will release the QA data derived from public datasets upon publication.

Bangwei Guo, Xiao Chen, Boris Mailhe +8

Rutgers University, NJ, USA · United Imaging Intelligence, Boston, MA, USA · University of Texas Southwestern Medical Center, TX, USA +1

Cardiac Magnetic Resonance Imaging Medical Vision-Language Models Cardiac Anatomy

Ask PDF

May 28, 2026cs.CV

CardioLens: Revealing the Clinical Reality Gap of MLLMs via Multi-Sequence Cardiac MRI Evaluations

Multimodal Large Language Models (MLLMs) have shown strong performance on public medical benchmarks, yet existing evaluations often remain weak proxies for clinical use, relying on isolated inputs and simplified recognition-style tasks. We introduce CardioLens, a leakage-resistant evaluation testbed for multi-sequence Cardiovascular Magnetic Resonance (CMR), constructed from private hospital archives through a rigorous report-to-QA construction and verification pipeline. CardioLens contains 473,896 slices and 13,494 verified QA pairs across 4D Cine, LGE, perfusion, and T2-weighted imaging, and evaluates three stages of CMR interpretation: image understanding, report generation, and disease diagnosis. Across 24 state-of-the-art MLLMs, CardioLens reveals a substantial clinical reality gap: models perform poorly overall, with performance degrading along the real CMR workflow. Confusion analysis further shows a category-collapse failure mode, where models default to frequent abnormal categories rather than distinguishing clinically distinct findings. To rule out MLLM-compatible input construction as the primary cause, we compare random, clinically motivated, and data-driven slice selection protocols under different slice budgets; performance changes only marginally, typically by about 1%. Explicit reasoning prompts also fail to rescue performance, often making models more conservative rather than improving visual evidence use. These results show that current MLLMs remain far from reliable CMR interpretation, where clinical decisions require integrating distributed evidence across sequences, views, and temporal phases. CardioLens provides a clinically grounded testbed for developing next-generation MLLMs toward real-world clinical deployment.

Zixian Su, Hongkai Zhang, Fan Gao +12

Beijing Academy of Artificial Intelligence · Beijing Anzhen Hospital · Beihang University +1

Cardiac Magnetic Resonance Imaging Medical Vision-Language Models Multimodal Clinical Data Multimodal Large Language Models

Ask PDF

Sep 1, 2026cs.CV

CMRVision: A Foundation Model for Cardiac MR Image Analysis

Cardiac magnetic resonance (CMR) imaging provides complementary information on cardiac anatomy, function, and tissue characterization across multiple sequences and views. In this work, we investigate foundation model pretraining for 2D CMR and introduce CMRVision, a CMR-specific foundation model trained using DINOv3-style self-supervised learning on a multi-center, multi-sequence cohort of 36 million CMR images. We systematically evaluate architectural and training design choices for domain-specific pretraining. CMRVision is evaluated on two downstream tasks: multi-task segmentation across cine, late gadolinium enhancement (LGE), and mapping sequences, and cine view classification. Our experiments show that CMR-specific pretraining, smaller patch sizes, and patch-level objectives consistently improve downstream performance. Across a multi-task segmentation benchmark, CMRVision achieved the strongest overall performance, outperforming prior natural-image (NI), medical-image, supervised, and CMR foundation model baselines. Improvements were modest but consistent across structures and sequences, with Dice scores ranging from 0.940-0.967 for LV and 0.855-0.905 for myocardium, and reaching 0.929 for RV, 0.920 for LA, and 0.931 for RA. The largest gains were observed for myocardium segmentation in LGE and mapping images. In a zero-shot segmentation task on unseen LGE long-axis views, the model achieved an average Dice score of 0.692, demonstrating cross-view generalization. For cine view classification, CMRVision achieved the highest average accuracy (0.906), compared to prior methods reported in the literature. These results highlight the potential of CMRVision to support robust and generalizable cardiac MRI analysis across multiple sequences and views.

Athira J. Jacob, Puneet Sharma, Daniel Rueckert

Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ, USA · Chair for Al in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany · Department of Computing, Imperial College London, UK

Cardiac Magnetic Resonance Imaging Echocardiography Cardiac Metamorphic Relation Adequacy Wireless Foundation Models Pretraining+1

Ask PDF