Echo2ECG: Enhancing ECG Representations with Cardiac Morphology from Multi-View Echos
Authors: Michelle Espranita Liman, Özgün Turgut, Alexander Müller, Eimo Martens, Daniel Rueckert, Philip Müller
Organizations: Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Germany · Department of Cardiology, TUM University Hospital, Germany · Department of Computing, Imperial College London, UK · Munich Center for Machine Learning (MCML), Germany
Electrocardiography (ECG) is a low-cost, widely used modality for diagnosing electrical abnormalities like atrial fibrillation by capturing the heart's electrical activity. However, it cannot directly measure cardiac morphological phenotypes, such as left ventricular ejection fraction (LVEF), which typically require echocardiography (Echo). Predicting these phenotypes from ECG would enable early, accessible health screening. Existing self-supervised methods suffer from a representational mismatch by aligning ECGs to single-view Echos, which only capture local, spatially restricted anatomical snapshots. To address this, we propose Echo2ECG, a multimodal self-supervised learning framework that enriches ECG representations with the heart's morphological structure captured in multi-view Echos. We evaluate Echo2ECG as an ECG feature extractor on two clinically relevant tasks that fundamentally require morphological information: (1) classification of structural cardiac phenotypes across three datasets, and (2) retrieval of Echo studies with similar morphological characteristics using ECG queries. Our extracted ECG representations consistently outperform those of state-of-the-art unimodal and multimodal baselines across both tasks, despite being 18x smaller than the largest baseline. These results demonstrate that Echo2ECG is a robust, powerful ECG feature extractor. Our code is accessible at https://github.com/michelleespranita/Echo2ECG.
Figures & tables
Figure 1: Overview of Echo2ECG . To address the representational mismatch in prior methods aligning ECGs with single-view Echos, our framework uses contrastive learning to align ECGs with multi-view Echo studies. This multi-view alignment distills comprehensive cardiac morphological information from Echos into ECG representations. We aggregate view-level Echo embeddings from a frozen, powerful Echo encoder into a study-level Echo embedding. During pre-training, we optimize only the ECG encoder, the Echo view aggregator, and modality-specific projection layers. The ECG encoder is then used to extract ECG representations for subsequent downstream tasks.
Split
#Pairs
#Patients
#ECGs
#Echo Studies
#Echo Views
Age
Sex (%Male)
Train
11,859
6,836
11,516
8,712
179,571
65.6 ± 16.1
63.1%
Val
1,248
565
1,213
934
20,633
65.3 ± 16.7
64.7%
Test
1,358
566
1,290
924
20,903
64.8 ± 16.6
64.1%
Table 1: Internal pre-training dataset details from TUM Klinikum Rechts der Isar. Age is presented as Mean ± SD.
Task
Dataset
Train ( N )
Val ( N )
Test ( N )
LVEF clf (0/1/2)
Internal
294 (33/33/33%)
564 (11/16/73%)
562 (14/15/71%)
EchoNext
300 (33/33/33%)
4,112 (17/7/76%)
4,827 (16/7/77%)
UKB
255 (33/33/33%)
2,834 (1/4/95%)
2,827 (1/4/95%)
SHD clf (0/1)
EchoNext
72,475 (48/52%)
4,626 (57/43%)
5,442 (57/43%)
Table 2: Downstream datasets and label distributions. LVEF: 0/1/2 = reduced/mildly reduced/normal; SHD: 0/1 = absence/presence. These tasks assess how well the ECG representations capture morphological information.
Paired Modality
Method
Internal
External
Internal
EchoNext
UKB
-
OTIS
(7.1M)
0.688 [0.650, 0.727]
0.697 [0.680, 0.713]
0.637 [0.574, 0.694]
-
xECG
(57.0M)
0.732 [0.696, 0.769]
0.695 [0.677, 0.711]
0.531 [0.474, 0.591]
-
ECGFounder
(30.7M)
0.746 [0.710, 0.781]
0.686 [0.670, 0.702]
0.653 [0.592, 0.708]
Text+Echo
EchoingECG
(126.6M)
0.740 [0.703, 0.776]
0.676 [0.661, 0.692]
0.604 [0.541, 0.662]
CMR
PTACL
(19.3M)
0.723 [0.686, 0.759]
0.704 [0.689, 0.719]
0.729 [0.677, 0.770]
Table 3: Best and second-best AUROC [95% CI] ( ↑ ) for ECG-based LVEF classification. Echo2ECG provides high-quality, out-of-the box ECG features, achieving the highest AUROC on our internal dataset and on EchoNext. On UKB, it ranks second behind PTACL, which was pre-trained on UKB by pairing ECG with CMR—a superior modality to Echo for LVEF quantification.
Figure 2: AUROC ( ↑ ) for ECG-based SHD classification. (a) A kNN classifier trained on just 1% of data using ECG representations extracted by Echo2ECG outperforms most models trained on 100% of the data, meaning that Echo2ECG provides robust ECG features that work well under low-data regimes. (b) Echo2ECG outperforms EchoingECG at 0.1% training data, despite being 18 × smaller.
Method
LVEDV
LVESV
LVSV
LVEF
P@1 ↑
MnR ↓
P@1 ↑
MnR ↓
P@1 ↑
MnR ↓
P@1 ↑
MnR ↓
Random retrieval
0.296
5.915
0.366
6.081
0.323
4.570
0.295
7.230
EchoingECG
0.241
10.111
0.337
10.998
0.325
5.320
0.232
13.035
Echo2ECG ( single -view Echo)
0.332
5.376
0.517
4.366
0.334
5.125
0.415
5.346
Echo2ECG ( multi -view Echo)
0.387
4.358
0.517
3.841
0.379
4.193
0.404
5.101
Table 4: Cross-modal retrieval performance for structural cardiac phenotypes. Multi-view alignment allows Echo2ECG to accurately retrieve Echo studies from ECG queries.
Alignment
View Agg.
LVEF clf (Internal)
SHD clf (EchoNext 1%)
ECG ↔ single -view Echo
-
0.747
0.743
ECG ↔ multi -view Echo
mean pool
0.775
0.752
ECG ↔ multi -view Echo
CLS pool
0.756
0.754
ECG ↔ multi -view Echo
attention pool
0.785
0.757
Table 5: Ablation of Echo2ECG components. Mean AUROC [95% CI] ( ↑ ) is reported. Multi-view alignment and attention pooling improve the quality of ECG features.
Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · College of information science and engineering, Ritsumeikan university, ibaraki Osaka, Japan +1