CineMR: Tool-Integrated Vision-Language Reasoning for Quantitative Cardiac MRI Assessment

Kunyang Li; Hai Nguyen; Joshua Lowe; Chenguang Zhao; Peace C. Madueme; Mehdi Hedjazi Moghari; Mubarak Shah; Pegah Khosravi

cs.CVOct 1, 2026

CineMR: Tool-Integrated Vision-Language Reasoning for Quantitative Cardiac MRI Assessment

Authors: Kunyang Li, Hai Nguyen, Joshua Lowe, Chenguang Zhao, Peace C. Madueme, Mehdi Hedjazi Moghari, Mubarak Shah, Pegah Khosravi, +1 more

Organizations: Institute for Artificial Intelligence, University of Central Florida · Department of Clinical Sciences, College of Medicine, University of Central Florida · Nemours Cardiac Center, Nemours Children’s Hospital, Florida · Children’s Heart Center, WVU Golisano Children’s, West Virginia

Abstract

Cardiovascular magnetic resonance (CMR), including cine imaging, is a reference standard for the noninvasive assessment of cardiac morphology and ventricular function. Cine CMR interpretation integrates qualitative visual assessment with quantitative measurements of ventricular volumes, ejection fraction, myocardial mass, wall thickness, and regional wall motion. Current medical vision-language models (VLMs) cannot reliably derive quantitative measurements from multidimensional cine images without analysis tools. We present CineMR, a tool-augmented VLM that invokes cardiac image-analysis tools and integrates their outputs into interleaved reasoning for quantitative CMR assessment. We also construct a multi-cohort visual question answering benchmark covering quantitative metric extraction, multiclass diagnosis, and differential diagnosis, together with tools for segmentation, phase selection, volumetry, morphometry, and regional wall motion analysis. CineMR is trained with supervised fine-tuning (SFT) on tool-interaction traces followed by Group Relative Policy Optimization (GRPO) with conditional tool-use rewards. On the multi-cohort cine CMR benchmark, CineMR achieves 35.9% pass@1 and 58.9% pass@4, compared with 1.5% pass@1 for the Qwen3-VL-8B backbone and 0.0% and 7.0% pass@1 for LLaVA-Med v1.5 and MedGemma-4B, respectively. Correct tool invocation reaches 99.8% after GRPO, up from 78.9% after SFT. Live tool outputs improve ventricular measurement accuracy by 20.4--23.7% over direct model predictions, and removing all tools reduces pass@1 from 35.9% to 27.9%. These results highlight the importance of reliable tool use for quantitative cine CMR reasoning and support CineMR as a promising approach for assistive cardiac image assessment. Code, benchmark resources, and model weights are available at https://github.com/AI-MIND-Lab/CineMR.

Figures & tables

Figure 1 : Comparison of model outputs for the same cine CMR case, where the benchmark reference category is dilated cardiomyopathy (DCM). In this illustrative example, LLaVA-Med fails to provide a definitive assessment, whereas MedGemma produces unsupported tool-call content and repetitive text. In contrast, CineMR invokes executable tools to obtain quantitative measurements and evaluates them according to criteria encoded in the study-specific Rule-Based Cardiac Phenotype Classifier, thereby producing a traceable output grounded in explicit measurements and decision rules.

Panel of Figure 2 — Figure 2 : Overview of the CineMR framework, designed to mirror the real-world cardiologist workflow. Just as a clinician first inspects the cine images to form a visual impression, CineMR generates an initial phenotypic hypothesis via chain-of-thought reasoning. The clinician then orders quantitative analyses (segmentation, volumetry, wall thickness measurement); analogously, CineMR invokes a cardiology-specific quantitative tool suite to obtain precise metrics (EDV, ESV, EF, mass, MVR, wall thickness, sphericity index, regional wall motion). Finally, where the clinician interprets these measurements against ESC/AHA guideline thresholds to reach a diagnosis, CineMR routes them through a guideline-aligned Rule-Based Cardiac Phenotype Classifier with threshold-gated branching, producing an auditable reasoning chain that parallels the clinical diagnostic process from visual assessment to quantitative verification to evidence-based conclusion.

Figure 3 : Representative verbatim CineMR rollout for a cine CMR case assigned to the DCM category. The figure shows the original benchmark prompt, model-generated tool calls, executable tool outputs, intermediate generated rationale, and final predicted category. CineMR invokes cardiac segmentation, ventricular volumetry, and morphometric analysis tools before applying the published and study-specific criteria encoded in the Rule-Based Cardiac Phenotype Classifier. Tool results are displayed at their native quantitative precision. The intermediate rationale and final response are reproduced verbatim and should not be interpreted as independently validated clinical conclusions.

Figure 4 : The tools produce the quantitative measurements required by the study-specific Rule-Based Cardiac Phenotype Classifier. Final classification includes the following categories: hypertrophic cardiomyopathy (HCM), athlete’s heart or physiological athletic remodeling, arrhythmogenic right ventricular cardiomyopathy (ARVC), left ventricular non-compaction (LVNC), ischemic heart disease (IHD), myocardial infarction (MINF), dilated cardiomyopathy (DCM), hypertensive heart disease (HHD), normal (NOR), and indeterminate.

Tool	Input $\rightarrow$ Output	Unlocked Metrics	Role
segment_cardiac	$\mathcal{I}\rightarrow\mathbf{s}$	LV/RV/MYO masks	prerequisite
measure_volume	$\mathbf{s}\rightarrow\mathbf{v}$	EDV, ESV, EF, EDVi	computational
compute_cardiac_metrics	$(\mathbf{s},\mathbf{v})\rightarrow\mathbf{m}$	mass, MVR, EDWT, S:L, SI	computational
wall_motion_analysis	$\mathbf{s}\rightarrow\mathbf{w}$	segment-level WMA	computational
select_cardiac_phases	$\mathbf{s}\rightarrow(t_{\text{ED}},t_{\text{ES}})$	temporal grounding	auxiliary
zoom_anatomy	$(t,\text{target})\rightarrow\text{ROI}$	local visual detail	auxiliary

Table 1 : Cardiology-specific tool suite. Each tool maps to Rule-Based Cardiac Phenotype Classifier nodes requiring quantitative thresholds inaccessible to visual inspection.

	ACDC			M&Ms			M&Ms-2			Overall
Method	P@1	P@4	Mean	P@1	P@4	Mean	P@1	P@4	Mean	P@1	P@4	Mean
LLaVA-Med v1.5	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
MedGemma-4B	5.2	16.1	5.5	7.7	19.1	7.4	6.7	17.0	6.3	7.0	18.0	6.7
Qwen3-VL-8B	1.3	4.9	1.4	1.4	4.2	1.4	1.9	4.6	1.4	1.5	4.5	1.4
SFT	15.2	28.3	15.6	26.5	41.6	26.3	24.2	42.0	24.4	23.6	38.9	23.6
GRPO	33.9	58.0	34.7	36.2	58.0	35.4	36.9	62.0	37.3	35.9	58.9	35.7

Table 2 : Held-out test performance by cohort (

n=3{,}320

; ACDC: 696, M&Ms: 1,846, M&Ms-2: 778). All methods use

k=4

rollouts with temperature 0.7. SFT and GRPO correspond to the matched pre- and post-RL models under the same live-tool protocol. GRPO consistently improves performance over SFT across all cohorts, yielding the best overall results.

	MedGemma-4B			Qwen3-VL-8B			CineMR (SFT)			CineMR (GRPO)
Slice	P@1	P@4	Mean	P@1	P@4	Mean	P@1	P@4	Mean	P@1	P@4	Mean
Layer 1	1.9	7.5	2.2	2.9	8.3	2.6	6.5	20.2	6.4	10.8	31.5	10.1
Layer 2	17.6	38.8	17.0	0.5	1.7	0.4	27.6	33.0	27.3	65.3	92.0	65.3
Layer 3	8.4	23.3	7.2	0.6	1.9	0.6	55.2	83.7	55.8	60.9	86.7	60.7
Layer 4	2.8	15.0	4.2	0.0	1.9	0.5	50.5	78.5	52.1	45.8	78.5	50.0
Layer 5	0.0	0.0	0.0	0.0	0.0	0.0	12.9	33.3	12.6	13.3	32.5	13.0

Table 3 : Layer-wise performance of SFT (pre-RL) and GRPO (post-RL) with live tool execution, compared with the off-the-shelf MedGemma-4B-it and Qwen3-VL-8B baselines. Results are reported by cognitive layer on the held-out test set (

n=3{,}320

). GRPO improves over SFT across most cognitive layers, with the largest gains observed in Layers 2–3.

Panel of Figure 5 — Figure 5 : Tool-use diagnostics and tool-withholding ablation. (a) Correct tool-call rate for SFT and GRPO on gold tool-requiring items. (b) GRPO accuracy of direct model predictions versus live measure_volume outputs on ventricular volume and ejection-fraction questions. (c) GRPO pass@1 on the held-out test set ( $n=3{,}320$ , $k=4$ ) with full tool access, without computation tools, and without any tools; error bars show 95% bootstrap confidence intervals. GRPO raises correct tool invocation from 78.9% to 99.8%; live tool outputs improve measurement accuracy by 20.4–23.7%, and removing all tools lowers pass@1 from 35.9% to 27.9%.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Acronym	Definition
LV / RV	Left / right ventricle (ventricular)
LVEF / RVEF	Left / right ventricular ejection fraction
LVEDV / LVESV	LV end-diastolic / end-systolic volume
RVEDV / RVESV	RV end-diastolic / end-systolic volume
EDWT	End-diastolic wall thickness
S:L ratio	Septal-to-lateral wall thickness ratio

Appendix

Table 4 : Acronym glossary.

Template ID	Layer	Tools	Question
L1_lvef	L1	✓	What is the left ventricular ejection fraction (LVEF) for this patient? Report as a percentage and indicate whether it is normal, mildly reduced, reduced, or severely reduced.
L1_lvedv	L1	✓	Report the left ventricular end-diastolic volume (LVEDV) and end-systolic volume (LVESV) for this cardiac MRI study, including BSA-indexed values if anthropometric data are available.
L1_rvedv	L1	✓	What are the right ventricular end-diastolic volume (RVEDV), end-systolic volume (RVESV), and RV ejection fraction (RVEF) for this patient?
L1_lv_mass	L1	✓	Calculate the left ventricular myocardial mass from this cardiac MRI segmentation. Report the absolute LV mass in grams.
L1_rvef	L1	✓	What is the right ventricular ejection fraction (RVEF) for this patient? Is it within normal limits?
L1_max_edwt	L1	✓	What is the maximum end-diastolic wall thickness (max EDWT) of the left ventricle in this cardiac MRI, and does it exceed the threshold for hypertrophy?

Appendix

Table 5 : All 19 evaluation question templates, by cognitive layer. Tools : whether tool use is required (✓) or optional (--) for that template; a template may override its layer’s default (e.g. L2_is_asymmetric requires tools because the septal:lateral ratio must be computed).

Explore similar work

Learning Where to Look: Anatomical Grounding and Guided Attention for Cardiac MRI Vision-Language Models
Sep 30, 2026Bangwei Guo, Xiao Chen, Boris Mailhe +8Cardiac Magnetic Resonance Imaging Medical Vision-Language Models
Rutgers University, NJ, USA · United Imaging Intelligence, Boston, MA, USA · University of Texas Southwestern Medical Center, TX, USA +1
CardioLens: Revealing the Clinical Reality Gap of MLLMs via Multi-Sequence Cardiac MRI Evaluations
May 28, 2026Zixian Su, Hongkai Zhang, Fan Gao +12Cardiac Magnetic Resonance Imaging Medical Vision-Language Models
Beijing Academy of Artificial Intelligence · Beijing Anzhen Hospital · Beihang University +1
CMRVision: A Foundation Model for Cardiac MR Image Analysis
Sep 1, 2026Athira J. Jacob, Puneet Sharma, Daniel RueckertCardiac Magnetic Resonance Imaging Echocardiography
Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ, USA · Chair for Al in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany · Department of Computing, Imperial College London, UK

Sep 30, 2026, cs.CVJ/K move · Enter open · S save

Learning Where to Look: Anatomical Grounding and Guided Attention for Cardiac MRI Vision-Language Models

Bangwei Guo, Xiao Chen, Boris Mailhe, Jia Yao+7

Rutgers University, NJ, USA · United Imaging Intelligence, Boston, MA, USA · University of Texas Southwestern Medical Center, TX, USA+1

Cardiac Magnetic Resonance Imaging Medical Vision-Language Models Cardiac Anatomy

Open paper AskPDF arXiv

Cardiac magnetic resonance imaging (CMR) enables assessment of cardiac anatomy, ventricular function, and myocardial tissue characteristics. Clinicians interpret these images by identifying cardiac structures and focusing on the regions relevant to each clinical question, motivating anatomically guided vision-language models (VLMs). Yet CMR-specific supervision for anatomical localisation and clinical question answering remains limited. To address this gap, we investigate fine-grained CMR visual question answering through anatomical grounding and guided attention. We construct 128,915 anatomical-grounding and 42,799 clinical QA pairs across short-axis cine, late gadolinium enhancement, and long-axis cine. These datasets support anatomical recognition, localisation, and clinical assessment without requiring paired reports for individual training images. To help the model learn where to look, we introduce Cardiac Anatomy-Routed Attention (CARA), which selects predicted anatomical priors according to the question and guides decoder attention with learned task-specific strengths. Combining anatomical grounding pretraining with CARA yields our model, CARA-VL. Experiments demonstrate CARA-VL's strengths in clinical assessment and regional localisation across CMR imaging settings, with promising generalization to an external clinical cohort. Together, our data and method provide a practical framework for studying and advancing cardiac visual understanding in VLMs. We will release the QA data derived from public datasets upon publication.

Abstract

Explore similar work

Learning Where to Look: Anatomical Grounding and Guided Attention for Cardiac MRI Vision-Language Models

CardioLens: Revealing the Clinical Reality Gap of MLLMs via Multi-Sequence Cardiac MRI Evaluations

CMRVision: A Foundation Model for Cardiac MR Image Analysis