Opportunistic Cardiac Health Assessment: Estimating Phenotypes from Localizer MRI through Multi-Modal Representations

Busra Nur Zeybek; Özgün Turgut; Yundi Zhang; Jiazhen Pan; Robert Graf; Sophie Starck; Daniel Rueckert; Sevgi Gokce Kafali

cs.CVMar 13, 2026

Opportunistic Cardiac Health Assessment: Estimating Phenotypes from Localizer MRI through Multi-Modal Representations

Authors: Busra Nur Zeybek, Özgün Turgut, Yundi Zhang, Jiazhen Pan, Robert Graf, Sophie Starck, Daniel Rueckert, Sevgi Gokce Kafali

Organizations: Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Germany · Munich Center for Machine Learning (MCML), Germany · Department of Diagnostic and Interventional Neuroradiology, School of Medicine, TUM University Hospital, Germany. · Department of Computing, Imperial College London, United Kingdom

Abstract

Cardiovascular diseases are the leading cause of death. Cardiac phenotypes (CPs), e.g., ejection fraction, are the gold standard for assessing cardiac health, but they are derived from cine cardiac magnetic resonance imaging (CMR), which is costly and requires high spatio-temporal resolution. Every magnetic resonance (MR) examination begins with rapid and coarse localizers for scan planning, which are discarded thereafter. Despite non-diagnostic image quality and lack of temporal information, localizers can provide valuable structural information rapidly. In addition to imaging, patient-level information, including demographics and lifestyle, influence the cardiac health assessment. Electrocardiograms (ECGs) are inexpensive, routinely ordered in clinical practice, and capture the temporal activity of the heart. Here, we introduce C-TRIP (Cardiac Tri-modal Representations for Imaging Phenotypes), a multi-modal framework that aligns localizer MRI, ECG signals, and tabular metadata to learn a robust latent space and predict CPs using localizer images as an opportunistic alternative to CMR. By combining these three modalities, we leverage cheap spatial and temporal information from localizers, and ECG, respectively while benefiting from patient-specific context provided by tabular data. Our pipeline consists of three stages. First, encoders are trained independently to learn uni-modal representations. The second stage fuses the pre-trained encoders to unify the latent space. The final stage uses the enriched representation space for CP prediction, with inference performed exclusively on localizer MRI. Proposed C-TRIP yields accurate functional CPs, and high correlations for structural CPs. Since localizers are inherently rapid and low-cost, our C-TRIP framework could enable better accessibility for CP estimation.

Figures & tables

Aspect	Localizer MRI	Cardiac CINE MRI
Purpose	Scan planning; not for diagnosis	CP quantification and diagnosis
Scan time	Very short (seconds per stack)	Longer
Temporal info	None (static images)	Multi-frame across the cardiac cycle
Motion	Not ECG-gated; susceptible to motion	ECG-gated; captures cardiac motion
Image quality	Lower resolution; variable contrast	High spatial resolution and contrast
Standardization	Less standardized across sites	Established clinical protocols

Table 1: Comparison of routinely acquired localizer MRI and CMR.

Figure 1: Our multi-modal contrastive learning framework. First stage includes pre-training encoders for each modality. Second stage involves unification of the latent spaces from three modalities, in which the data from the same subjects were pulled, and different subjects were pushed away. The last stage uses a regression head to predict all 18 cardiac phenotypes solely from localizer MRIs.

T p

LVEF [%]

LVM [g]

RVEF [%]

RVEDV [mL]

LoA

CMR

{}_{\text{sup}}

✗

0.3

\boldsymbol{[-8.0,8.5]}

-0.3

\boldsymbol{[-24.5,24.0]}

-0.2

\boldsymbol{[-8.7,8.4]}

2.8

\boldsymbol{[-39.2,44.8]}

{}_{\text{sup}}

✓

✗

0.1

[-10.8,11.1]

0.7

[-23.4,24.8]

0.1

[-10.3,10.4]

1.8

[-39.8,43.4]

{}_{\text{sup}}

✗

✓

✗

0.4

[-10.9,11.6]

-0.8

[-35.9,34.3]

0.2

[-10.8,11.2]

-0.6

[-60.6,59.4]

{}_{\text{sup}}

✗

✓

0.1

[-11.3,11.5]

2.3

[-22.5,27.0]

-0.5

[-11.5,10.4]

5.0

[-38.9,48.9]

L+T

✓

✗

✓

0.1

[-10.6,10.8]

0.3

[-23.1,23.8]

0.0

[-10.2,10.2]

1.1

[-39.8,41.9]

Table 2: Performance comparison across modalities. T/T p : tabular data without/with CPs; E: electrocardiogram; L: localizer MRI. CMR sup : provided as reference. Bold text and gray shading denote best MD and LoA.

Figure 2: A) Attention maps . Maps represent the [CLS] token’s self-attention weights from the final ViT block, head-averaged and up-sampled. While the supervised localizer baseline, $L_{sup}$ , exhibits leaked attention to non-cardiac structures (e.g., lungs and chest wall), C-TRIP was able to focus on biologically relevant structures. Images are cropped to the heart region for visibility. B) UMAPs showing representation space before and after alignment. Our localizer-centric design reflects itself in UMAPs. Due to strong anatomical correlation, $L$ and $T$ merge easily. The relationship between $L$ and $E$ is more complex, so $E$ remains more distinct, but is pulled toward the same phenotypic gradient/trend from low to high LVM, or male/female distinction.

Figure 3: Performance and scaling comparison of our proposed multimodal approach against unimodal supervised baselines across four CPs. (Top Row) Scatter plots comparing model predictions to ground truth values using 100% of the available fine-tuning data for LVEF, LVM, RVEF, RVEDV. Pearson $R$ is reported in the legends. The plots show 500 subjects for better visibility. (Bottom Row) Fine-tuning data scaling behavior. The plots demonstrate model performance (Pearson $R$ ) as a function of the fine-tuning subset size (1%, 10%, and 100%, displayed on a logarithmic scale). The shaded regions represent the 95% confidence intervals. Our proposed method C-TRIP (in green) is compared against single modality baselines.

Left Ventricular Ejection Fraction Estimation

Left ventricular ejection fraction estimation uses cardiac imaging and physiological signals to quantify how much blood the heart pumps per beat.

See all 16 papers

Echo2ECG: Enhancing ECG Representations with Cardiac Morphology from Multi-View Echos
Mar 9, 2026Michelle Espranita Liman, Özgün Turgut, Alexander Müller +3Electrocardiogram Representation Learning Multimodal Learning
Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Germany · Department of Cardiology, TUM University Hospital, Germany · Department of Computing, Imperial College London, UK +1
ECGFlowCMR: Pretraining with ECG-Generated Cine CMR Helps Cardiac Disease Classification and Phenotype Prediction
Jan 28, 2026Xiaocheng Fang, Zhengyao Ding, Guangkun Nie +9Synthetic Data Pretraining Medical Diagnosis
State Key Laboratory of General Artificial Intelligence, Peking University · School of Intelligence Science and Technology, Peking University, Beijing, China · Polytechnic Institute of Zhejiang University, Zhejiang University, Hangzhou, China +4
Leveraging Cardiac Imaging to Improve ECG-Based Detection of Chagas Disease in Resource-Constrained Settings
Sep 8, 2026Laura Alvarez-Florez, Daniel Uyterlinde, Samuel Ruipérez-Campillo +3Cross-Modal Representation Learning Medical Diagnosis
Department of Biomedical Engineering and Physics, Amsterdam University Medical Center, The Netherlands · Quantitative Healthcare Analysis Group, University of Amsterdam, The2026 Netherlands · Department of Clinical and Experimental Cardiology, Amsterdam University Medical Center, The Netherlands +2

Mar 9, 2026J/K move · Enter open · S save

Echo2ECG: Enhancing ECG Representations with Cardiac Morphology from Multi-View Echos

Michelle Espranita Liman, Özgün Turgut, Alexander Müller, Eimo Martens+2

Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Germany · Department of Cardiology, TUM University Hospital, Germany · Department of Computing, Imperial College London, UK+1

Electrocardiogram Representation Learning Multimodal Learning Electrocardiogram Classification Self-Supervised Time Series Representation Learning Cross-Modal Retrieval

Open paper AskPDF arXiv

Electrocardiography (ECG) is a low-cost, widely used modality for diagnosing electrical abnormalities like atrial fibrillation by capturing the heart's electrical activity. However, it cannot directly measure cardiac morphological phenotypes, such as left ventricular ejection fraction (LVEF), which typically require echocardiography (Echo). Predicting these phenotypes from ECG would enable early, accessible health screening. Existing self-supervised methods suffer from a representational mismatch by aligning ECGs to single-view Echos, which only capture local, spatially restricted anatomical snapshots. To address this, we propose Echo2ECG, a multimodal self-supervised learning framework that enriches ECG representations with the heart's morphological structure captured in multi-view Echos. We evaluate Echo2ECG as an ECG feature extractor on two clinically relevant tasks that fundamentally require morphological information: (1) classification of structural cardiac phenotypes across three datasets, and (2) retrieval of Echo studies with similar morphological characteristics using ECG queries. Our extracted ECG representations consistently outperform those of state-of-the-art unimodal and multimodal baselines across both tasks, despite being 18x smaller than the largest baseline. These results demonstrate that Echo2ECG is a robust, powerful ECG feature extractor. Our code is accessible at https://github.com/michelleespranita/Echo2ECG.

Abstract

Left Ventricular Ejection Fraction Estimation

Explore similar work

Echo2ECG: Enhancing ECG Representations with Cardiac Morphology from Multi-View Echos

ECGFlowCMR: Pretraining with ECG-Generated Cine CMR Helps Cardiac Disease Classification and Phenotype Prediction

Leveraging Cardiac Imaging to Improve ECG-Based Detection of Chagas Disease in Resource-Constrained Settings