Learning ECG Image Representations via Dual Physiological-Aware Alignments

Hung Manh Pham; Jialu Tang; Aaqib Saeed; Dong Ma; Bin Zhu; Pan Zhou

cs.LGApr 2, 2026

Learning ECG Image Representations via Dual Physiological-Aware Alignments

Authors: Hung Manh Pham, Jialu Tang, Aaqib Saeed, Dong Ma, Bin Zhu, Pan Zhou

Organizations: University of Cambridge · Singapore Management University · Eindhoven University of Technology, Netherlands

Abstract

Electrocardiograms (ECGs) are among the most widely used diagnostic tools for cardiovascular diseases, and a large amount of ECG data worldwide appears only in image form. However, most existing automated ECG analysis methods rely on access to raw signal recordings, limiting their applicability in real-world and resource-constrained settings. In this paper, we present ECG-Scan, a self-supervised framework for learning clinically generalized representations from ECG images through dual physiological-aware alignments: 1) Our approach optimizes image representation learning using multimodal contrastive alignment between image and gold-standard signal-text modalities. 2) We further integrate domain knowledge via soft-lead constraints, regularizing the reconstruction process and improving signal lead inter-consistency. Extensive benchmarking across multiple datasets and downstream tasks demonstrates that our image-based model achieves superior performance compared to existing image baselines and notably narrows the gap between ECG image and signal analysis. These results highlight the potential of self-supervised image modeling to unlock large-scale legacy ECG data and broaden access to automated cardiovascular diagnostics.

Figures & tables

Figure 1. (a) Common ECG acquisition in a clinical environment, where an ECG machine is connected to a printer to get a paper-based document for patients. Optionally, a local scanner might be connected for digital archive (images, in PDF), depending on the ECG machine types and hardware systems. (b) Patient capturing of ECG printouts using a smartphone camera (or a scanner), resulting in an image-based ECG for their own long-term archive. (c) Possible remote review of ECG images during consultation service, where expert clinicians interpret in-depth cardiac patterns from shared ECG images.

Figure 2. Illustration of ECG-Scan. We present a multimodal framework that aligns ECG images, signals, and clinical texts through dual physiological-aware alignment strategy.

Figure 3. Illustration of Einthoven’s Law and Goldberger’s equations for limb leads of ECG signals.

Lead	PTB-XL	CPSC	CSN
Lead III	50.70	89.62	46.41
Lead aVR	48.65	45.61	36.54
Lead aVL	47.38	41.89	33.55
Lead aVF	47.13	43.95	35.70

Table 1. Mean SNR values across downstream datasets. Here, SNR was computed by comparing limb leads (III, aVR, aVL, and aVF) calculated from Leads I and II using Einthoven’s and Goldberger’s formulas with the actual recorded leads.

	Methods	PTBXL-Super			PTBXL-Sub			PTBXL-Form			PTBXL-Rhythm			CPSC2018			CSN			Average
	Methods	1%	10%	100%	1%	10%	100%	1%	10%	100%	1%	10%	100%	1%	10%	100%	1%	10%	100%	1%	10%	100%
10s Signal Input	SimCLR ( Chen et al., 2020 )	$63.41$	$69.77$	$73.53$	$60.84$	$68.27$	$73.39$	$54.98$	$56.97$	$62.52$	$51.41$	$69.44$	$77.73$	$59.78$	$68.52$	$76.54$	$59.02$	$67.26$	$73.20$	$58.24$	$66.70$	$72.82$
	BYOL ( Grill et al., 2020 )	$71.70$	$73.83$	$76.45$	$57.16$	$67.44$	$71.64$	$48.73$	$61.63$	$70.82$	$41.99$	$74.40$	$77.17$	$60.88$	$74.42$	$78.75$	$54.20$	$71.92$	$74.69$	$55.78$	$70.61$	$74.92$
	BarlowTwins ( Zbontar et al., 2021 )	$72.87$	$75.96$	$78.41$	$62.57$	$70.84$	$74.34$	$52.12$	$60.39$	$66.14$	$50.12$	$73.54$	$77.62$	$55.12$	$72.75$	$78.39$	$60.72$	$71.64$	$77.43$	$58.92$	$70.85$	$75.39$
	MoCo-v3 ( Chen et al., 2021 )	$73.19$	$76.65$	$78.26$	$55.88$	$69.21$	$76.69$	$50.32$	$63.71$	$71.31$	$51.38$	$71.66$	$74.33$	$62.13$	$76.74$	$75.29$	$54.61$	$74.26$	$77.68$	$57.92$	$72.04$	$75.59$
	SimSiam ( Chen and He, 2021 )	$73.15$	$72.70$	$75.63$	$62.52$	$69.31$	$76.38$	$55.16$	$62.91$	$71.31$	$49.30$	$69.47$	$75.92$	$58.35$	$72.89$	$75.31$	$58.25$	$68.61$	$77.41$	$59.46$	$69.32$	$75.33$

Table 2. Linear probing performance (AUC in %) across multiple models, datasets, and training sizes. Here, we compare the common signal foundation models as upper bounds, with different image baselines.

Methods	ECG Input	PTBXL-Super	PTBXL-Sub	PTBXL-Form	PTBXL-Rhythm	CPSC2018	CSN	Average
MERL ( Liu et al., 2024a )	10s Signal	74.2	75.7	65.9	78.5	82.8	74.4	75.3
D-BETA ( Hung et al., 2025 )	10s Signal	76.2	75.9	66.1	88.6	80.1	76.3	77.1
MELP ( Wang et al., 2025 )	10s Signal	76.2	81.2	69.1	85.4	84.2	77.6	79.0
DIP + D-BETA	Image	61.2	63.4	53.1	76.5	58.4	66.3	63.2
nnU-Net + D-BETA ( Krones et al., 2024 )	Image	61.1	65.2	58.3	75.9	72.0	66.5	66.5
Open-Digitizer + D-BETA ( Stenhede et al., 2026 )	Image	67.3	64.8	58.6	82.7	73.6	64.4	68.6

Table 3. Zero-shot performance (AUC in %) across multiple models and datasets. Here, we reported three signal foundation models: MERL, D-BETA, and MELP as upper bounds (signal-text), comparing with different image baselines (image-text).

Human Expert’s Interpretation			Signal-based Models		Image-based Models
Cardio Resident	Emergency Resident	Medical Student	MERL	D-BETA	DIP + D-BETA	nnUnet + D-BETA	Open-Digitizer + D-BETA	ECG-Scan
92.07	90.52	93.61	85.14	96.79	64.97	85.20	94.04	94.78

Table 4. ECG interpretation comparisons (AUC in %) on the same source CODE-test dataset: Human experts ( Ribeiro et al., 2020 ) vs. different 10s signal foundation models (zero-shot signal-text) vs. different image baselines (zero-shot image-text).

Source Domain	Zero-shot	Training Ratio	PTBXL-Super		CPSC2018		CSN		Average
Target Domain	Zero-shot	Training Ratio	CPSC2018	CSN	PTBXL-Super	CSN	PTBXL-Super	CPSC2018	Average
SimCLR ( Chen et al., 2020 )	✗	100%	69.62	73.05	56.65	66.36	59.74	62.11	65.22
BYOL ( Grill et al., 2020 )	✗	100%	70.27	74.01	57.32	67.56	60.39	63.24	65.63
BarlowTwins ( Zbontar et al., 2021 )	✗	100%	68.98	72.85	55.97	65.89	58.76	61.35	64.13
MoCo-v3 ( Chen et al., 2021 )	✗	100%	69.41	73.29	56.54	66.12	59.82	62.07	64.21
SimSiam ( Chen and He, 2021 )	✗	100%	70.06	73.92	57.21	67.48	60.23	63.09	65.33

Table 5. Performance under data distribution shift. Here, ECG-Scan deals with image input, while the others use 10s signals.

$\mathcal{L}_{\text{ctr}}$	$\mathcal{L}_{\text{gram}}$	$\mathcal{L}_{\text{rule}}$	Zero-shot	Linear probing
			–	$62.71\pm 9.62$
✓			$73.62\pm 7.25$	$68.35\pm 8.19$
✓	✓		$75.11\pm 6.70$	$69.76\pm 8.94$
✓	✓	✓	$75.83\pm 6.79$	$71.92\pm 8.72$

Table 6. Effects of dual physiological-aware alignment.

Modality	Replaced Encoder	Zero-shot	Linear probing
Text	Bio-ClinicalBERT ( Alsentzer et al., 2019 )	$74.38\pm 8.15$	$70.95\pm 8.82$
Image	MedSigLIP ( Sellergren et al., 2025 )	$73.52\pm 7.86$	$70.35\pm 8.81$
Signal	MELP ( Wang et al., 2025 )	$73.58\pm 8.77$	$69.47\pm 8.99$
ECG-Scan		$75.83\pm 6.79$	$71.92\pm 8.72$

Table 7. Effects of different encoders across modalities.

Panel of Figure 4 — Figure 4. T-SNE visualizations of representations learned by different ECG encoders on the CSN testing set. Here, ECG-Scan deals with image input, while the others use 10s ECG signals. Each color represents a cardiac diagnosis category.

Panel of Figure 5 — Figure 5. Examples of image augmentations during pretraining.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6. Linear probing performance comparing 10 s and 2.5 s ECG signal inputs across six datasets, averaged over seven ECG foundation models.

Dataset	# Categories	Train	Valid	Test	Optimizer	# Epoch	Batch size	Learning rate
MIMIC-IV-ECG ( Gow et al., 2023 )	–	710,560	78,951	–	AdamW	–	80	0.0005
PTBXL-Super ( Wagner et al., 2020 )	5	17,084	2,146	2,158	AdamW	100	16	0.001
PTBXL-Sub ( Wagner et al., 2020 )	23	17,084	2,146	2,158	AdamW	100	16	0.001
PTBXL-Form ( Wagner et al., 2020 )	19	7,197	901	880	AdamW	100	16	0.001
PTBXL-Rhythm ( Wagner et al., 2020 )	12	16,832	2,100	2,098	AdamW	100	16	0.001
CPSC2018 ( Liu et al., 2018 )	9	4,950	551	1,376	AdamW	100	16	0.001

Appendix

Table 8. Details on data and training configurations.

Explore similar work

CG-DMER: Hybrid Contrastive-Generative Framework for Disentangled Multimodal ECG Representation Learning
Feb 24, 2026Ziwei Niu, Hao Sun, Shujun Bian +4Multimodal Disentangled Representation Learning Representation Learning
Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · College of information science and engineering, Ritsumeikan university, ibaraki Osaka, Japan +1
Information-theoretic Multimodal Representation Learning for Electrocardiogram Signals
May 26, 2026Phu X. Nguyen, Konstantinos Kontras, Wei Dai +5Representation Learning Cross-Modal Representation Learning
1KU Leuven · 3MIT · 4DFKI +1
Echo2ECG: Enhancing ECG Representations with Cardiac Morphology from Multi-View Echos
Mar 9, 2026Michelle Espranita Liman, Özgün Turgut, Alexander Müller +3Multimodal Learning Electrocardiogram Classification
Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Germany · Department of Cardiology, TUM University Hospital, Germany · Department of Computing, Imperial College London, UK +1

Feb 24, 2026J/K move · Enter open · S save

CG-DMER: Hybrid Contrastive-Generative Framework for Disentangled Multimodal ECG Representation Learning

Ziwei Niu, Hao Sun, Shujun Bian, Xihong Yang+3

Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · College of information science and engineering, Ritsumeikan university, ibaraki Osaka, Japan+1

Multimodal Disentangled Representation Learning Representation Learning Electrocardiography Multimodal Contrastive Learning

Open paper AskPDF arXiv

Accurate interpretation of electrocardiogram (ECG) signals is crucial for diagnosing cardiovascular diseases. Recent multimodal approaches that integrate ECGs with accompanying clinical reports show strong potential, but they still face two main concerns from a modality perspective: (1) intra-modality: existing models process ECGs in a lead-agnostic manner, overlooking spatial-temporal dependencies across leads, which restricts their effectiveness in modeling fine-grained diagnostic patterns; (2) inter-modality: existing methods directly align ECG signals with clinical reports, introducing modality-specific biases due to the free-text nature of the reports. In light of these two issues, we propose CG-DMER, a contrastive-generative framework for disentangled multimodal ECG representation learning, powered by two key designs: (1) Spatial-temporal masked modeling is designed to better capture fine-grained temporal dynamics and inter-lead spatial dependencies by applying masking across both spatial and temporal dimensions and reconstructing the missing information. (2) A representation disentanglement and alignment strategy is designed to mitigate unnecessary noise and modality-specific biases by introducing modality-specific and modality-shared encoders, ensuring a clearer separation between modality-invariant and modality-specific representations. Experiments on three public datasets demonstrate that CG-DMER achieves state-of-the-art performance across diverse downstream tasks.

Abstract

Explore similar work

CG-DMER: Hybrid Contrastive-Generative Framework for Disentangled Multimodal ECG Representation Learning

Information-theoretic Multimodal Representation Learning for Electrocardiogram Signals

Echo2ECG: Enhancing ECG Representations with Cardiac Morphology from Multi-View Echos