cs.CVMar 13, 2026

Opportunistic Cardiac Health Assessment: Estimating Phenotypes from Localizer MRI through Multi-Modal Representations

Authors: Busra Nur Zeybek, Özgün Turgut, Yundi Zhang, Jiazhen Pan, Robert Graf, Sophie Starck, Daniel Rueckert, Sevgi Gokce Kafali

Organizations: Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Germany · Munich Center for Machine Learning (MCML), Germany · Department of Diagnostic and Interventional Neuroradiology, School of Medicine, TUM University Hospital, Germany. · Department of Computing, Imperial College London, United Kingdom

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Cardiovascular diseases are the leading cause of death. Cardiac phenotypes (CPs), e.g., ejection fraction, are the gold standard for assessing cardiac health, but they are derived from cine cardiac magnetic resonance imaging (CMR), which is costly and requires high spatio-temporal resolution. Every magnetic resonance (MR) examination begins with rapid and coarse localizers for scan planning, which are discarded thereafter. Despite non-diagnostic image quality and lack of temporal information, localizers can provide valuable structural information rapidly. In addition to imaging, patient-level information, including demographics and lifestyle, influence the cardiac health assessment. Electrocardiograms (ECGs) are inexpensive, routinely ordered in clinical practice, and capture the temporal activity of the heart. Here, we introduce C-TRIP (Cardiac Tri-modal Representations for Imaging Phenotypes), a multi-modal framework that aligns localizer MRI, ECG signals, and tabular metadata to learn a robust latent space and predict CPs using localizer images as an opportunistic alternative to CMR. By combining these three modalities, we leverage cheap spatial and temporal information from localizers, and ECG, respectively while benefiting from patient-specific context provided by tabular data. Our pipeline consists of three stages. First, encoders are trained independently to learn uni-modal representations. The second stage fuses the pre-trained encoders to unify the latent space. The final stage uses the enriched representation space for CP prediction, with inference performed exclusively on localizer MRI. Proposed C-TRIP yields accurate functional CPs, and high correlations for structural CPs. Since localizers are inherently rapid and low-cost, our C-TRIP framework could enable better accessibility for CP estimation.

Figures & tables

AspectLocalizer MRICardiac CINE MRI
PurposeScan planning; not for diagnosisCP quantification and diagnosis
Scan timeVery short (seconds per stack)Longer
Temporal infoNone (static images)Multi-frame across the cardiac cycle
MotionNot ECG-gated; susceptible to motionECG-gated; captures cardiac motion
Image qualityLower resolution; variable contrastHigh spatial resolution and contrast
StandardizationLess standardized across sitesEstablished clinical protocols
Table 1: Comparison of routinely acquired localizer MRI and CMR.
Figure 1: Our multi-modal contrastive learning framework. First stage includes pre-training encoders for each modality. Second stage involves unification of the latent spaces from three modalities, in which the data from the same subjects were pulled, and different subjects were pushed away. The last stage uses a regression head to predict all 18 cardiac phenotypes solely from localizer MRIs.
 LET pTLVEF [%]LVM [g]RVEF [%]RVEDV [mL]
     MDLoAMDLoAMDLoAMDLoA
CMR sup{}_{\text{sup}}✗✗✗✗0.3[−8.0,8.5]\boldsymbol{[-8.0,8.5]}-0.3[−24.5,24.0]\boldsymbol{[-24.5,24.0]}-0.2[−8.7,8.4]\boldsymbol{[-8.7,8.4]}2.8[−39.2,44.8]\boldsymbol{[-39.2,44.8]}
L sup{}_{\text{sup}}✓✗✗✗0.1[−10.8,11.1][-10.8,11.1]0.70.7[−23.4,24.8][-23.4,24.8]0.10.1[−10.3,10.4][-10.3,10.4]1.81.8[−39.8,43.4][-39.8,43.4]
E sup{}_{\text{sup}}✗✓✗✗0.40.4[−10.9,11.6][-10.9,11.6]−0.8-0.8[−35.9,34.3][-35.9,34.3]0.20.2[−10.8,11.2][-10.8,11.2]−0.6-0.6[−60.6,59.4][-60.6,59.4]
T sup{}_{\text{sup}}✗✗✗✓0.1[−11.3,11.5][-11.3,11.5]2.32.3[−22.5,27.0][-22.5,27.0]−0.5-0.5[−11.5,10.4][-11.5,10.4]5.05.0[−38.9,48.9][-38.9,48.9]
L+T✓✗✗✓0.1[−10.6,10.8][-10.6,10.8]0.30.3[−23.1,23.8][-23.1,23.8]0.0[−10.2,10.2][-10.2,10.2]1.11.1[−39.8,41.9][-39.8,41.9]
Table 2: Performance comparison across modalities. T/T p : tabular data without/with CPs; E: electrocardiogram; L: localizer MRI. CMR sup : provided as reference. Bold text and gray shading denote best MD and LoA.
Figure 2: A) Attention maps . Maps represent the [CLS] token’s self-attention weights from the final ViT block, head-averaged and up-sampled. While the supervised localizer baseline, LsupL_{sup} , exhibits leaked attention to non-cardiac structures (e.g., lungs and chest wall), C-TRIP was able to focus on biologically relevant structures. Images are cropped to the heart region for visibility. B) UMAPs showing representation space before and after alignment. Our localizer-centric design reflects itself in UMAPs. Due to strong anatomical correlation, LL and TT merge easily. The relationship between LL and EE is more complex, so EE remains more distinct, but is pulled toward the same phenotypic gradient/trend from low to high LVM, or male/female distinction.
Figure 3: Performance and scaling comparison of our proposed multimodal approach against unimodal supervised baselines across four CPs. (Top Row) Scatter plots comparing model predictions to ground truth values using 100% of the available fine-tuning data for LVEF, LVM, RVEF, RVEDV. Pearson RR is reported in the legends. The plots show 500 subjects for better visibility. (Bottom Row) Fine-tuning data scaling behavior. The plots demonstrate model performance (Pearson RR ) as a function of the fine-tuning subset size (1%, 10%, and 100%, displayed on a logarithmic scale). The shaded regions represent the 95% confidence intervals. Our proposed method C-TRIP (in green) is compared against single modality baselines.
More in this topic

Left Ventricular Ejection Fraction Estimation

Left ventricular ejection fraction estimation uses cardiac imaging and physiological signals to quantify how much blood the heart pumps per beat.

See all 16 papers

Explore similar work

CardsList
Mar 9, 2026

Echo2ECG: Enhancing ECG Representations with Cardiac Morphology from Multi-View Echos

Electrocardiography (ECG) is a low-cost, widely used modality for diagnosing electrical abnormalities like atrial fibrillation by capturing the heart's electrical activity. However, it cannot directly measure cardiac morphological phenotypes, such as left ventricular ejection fraction (LVEF), which typically require echocardiography (Echo). Predicting these phenotypes from ECG would enable early, accessible health screening. Existing self-supervised methods suffer from a representational mismatch by aligning ECGs to single-view Echos, which only capture local, spatially restricted anatomical snapshots. To address this, we propose Echo2ECG, a multimodal self-supervised learning framework that enriches ECG representations with the heart's morphological structure captured in multi-view Echos. We evaluate Echo2ECG as an ECG feature extractor on two clinically relevant tasks that fundamentally require morphological information: (1) classification of structural cardiac phenotypes across three datasets, and (2) retrieval of Echo studies with similar morphological characteristics using ECG queries. Our extracted ECG representations consistently outperform those of state-of-the-art unimodal and multimodal baselines across both tasks, despite being 18x smaller than the largest baseline. These results demonstrate that Echo2ECG is a robust, powerful ECG feature extractor. Our code is accessible at https://github.com/michelleespranita/Echo2ECG.

Michelle Espranita Liman, Özgün Turgut, Alexander Müller +3

Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Germany · Department of Cardiology, TUM University Hospital, Germany · Department of Computing, Imperial College London, UK +1

Electrocardiogram Representation LearningMultimodal LearningElectrocardiogram ClassificationSelf-Supervised Time Series Representation LearningCross-Modal Retrieval
AskPDF
Jan 28, 2026

ECGFlowCMR: Pretraining with ECG-Generated Cine CMR Helps Cardiac Disease Classification and Phenotype Prediction

Cardiac Magnetic Resonance (CMR) imaging provides a comprehensive assessment of cardiac structure and function but remains constrained by high acquisition costs and reliance on expert annotations, limiting the availability of large-scale labeled datasets. In contrast, electrocardiograms (ECGs) are inexpensive, widely accessible, and offer a promising modality for conditioning the generative synthesis of cine CMR. To this end, we propose ECGFlowCMR, a novel ECG-to-CMR generative framework that integrates a Phase-Aware Masked Autoencoder (PA-MAE) and an Anatomy-Motion Disentangled Flow (AMDF) to address two fundamental challenges: (1) the cross-modal temporal mismatch between multi-beat ECG recordings and single-cycle CMR sequences, and (2) the anatomical observability gap due to the limited structural information inherent in ECGs. Extensive experiments on the UK Biobank and a proprietary clinical dataset demonstrate that ECGFlowCMR can generate realistic cine CMR sequences from ECG inputs, enabling scalable pretraining and improving performance on downstream cardiac disease classification and phenotype prediction tasks.

Xiaocheng Fang, Zhengyao Ding, Guangkun Nie +9

State Key Laboratory of General Artificial Intelligence, Peking University · School of Intelligence Science and Technology, Peking University, Beijing, China · Polytechnic Institute of Zhejiang University, Zhejiang University, Hangzhou, China +4

Synthetic Data PretrainingMedical DiagnosisClinical PredictionMedical Image SynthesisConditional Generative ModelingCardiac MRI
AskPDF
Sep 8, 2026

Leveraging Cardiac Imaging to Improve ECG-Based Detection of Chagas Disease in Resource-Constrained Settings

Chagas disease is a major cause of cardiomyopathy in Latin America. Cardiac magnetic resonance (CMR) imaging can characterize its structural abnormalities, but scanners and expert readers remain scarce in endemic regions. Electrocardiography (ECG) is inexpensive and widely available, yet structural disease must be inferred indirectly from electrical signals. We propose to transfer CMR-derived structural knowledge to ECG through contrastive pre-training. Using 63,193 paired ECG-CMR examinations from the UK Biobank, we align an ECG encoder with a clinically grounded CMR embedding space using an asymmetric InfoNCE objective. Despite seeing no Chagas cases during pre-training, the resulting representation improves ECG-based Chagas detection. Across CODE-15% and SaMi-Trop, a frozen linear probe achieves an AUROC of 0.851 and sensitivity at the top 5% of predicted risk (Top5%-TPR) of 0.427 in five-fold cross-validation, compared with 0.827 and 0.377 for an unaligned ECG-FM baseline. On the PhysioNet/CinC 2025 Challenge test set, our model obtains the highest AUROC on SaMi-Trop-3 and the best ELSA-Brasil challenge score among the three top-performing methods, indicating that imaging-supervised ECG representations can generalize to populations and settings beyond the pre-training distribution.

Laura Alvarez-Florez, Daniel Uyterlinde, Samuel Ruipérez-Campillo +3

Department of Biomedical Engineering and Physics, Amsterdam University Medical Center, The Netherlands · Quantitative Healthcare Analysis Group, University of Amsterdam, The2026 Netherlands · Department of Clinical and Experimental Cardiology, Amsterdam University Medical Center, The Netherlands +2

Cross-Modal Representation LearningMedical DiagnosisElectrocardiogram ClassificationElectrocardiogram Interpretation
AskPDF