CG-DMER: Hybrid Contrastive-Generative Framework for Disentangled Multimodal ECG Representation Learning

Ziwei Niu; Hao Sun; Shujun Bian; Xihong Yang; Lanfen Lin; Yuxin Liu; Yueming Jin

cs.AIFeb 24, 2026

CG-DMER: Hybrid Contrastive-Generative Framework for Disentangled Multimodal ECG Representation Learning

Authors: Ziwei Niu, Hao Sun, Shujun Bian, Xihong Yang, Lanfen Lin, Yuxin Liu, Yueming Jin

Organizations: Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · College of information science and engineering, Ritsumeikan university, ibaraki Osaka, Japan · Department of Electrical and Computer Engineering, National University of Singapore, Singapore, Singapore

Abstract

Accurate interpretation of electrocardiogram (ECG) signals is crucial for diagnosing cardiovascular diseases. Recent multimodal approaches that integrate ECGs with accompanying clinical reports show strong potential, but they still face two main concerns from a modality perspective: (1) intra-modality: existing models process ECGs in a lead-agnostic manner, overlooking spatial-temporal dependencies across leads, which restricts their effectiveness in modeling fine-grained diagnostic patterns; (2) inter-modality: existing methods directly align ECG signals with clinical reports, introducing modality-specific biases due to the free-text nature of the reports. In light of these two issues, we propose CG-DMER, a contrastive-generative framework for disentangled multimodal ECG representation learning, powered by two key designs: (1) Spatial-temporal masked modeling is designed to better capture fine-grained temporal dynamics and inter-lead spatial dependencies by applying masking across both spatial and temporal dimensions and reconstructing the missing information. (2) A representation disentanglement and alignment strategy is designed to mitigate unnecessary noise and modality-specific biases by introducing modality-specific and modality-shared encoders, ensuring a clearer separation between modality-invariant and modality-specific representations. Experiments on three public datasets demonstrate that CG-DMER achieves state-of-the-art performance across diverse downstream tasks.

Figures & tables

Figure 1: A schematic view of the proposed method.

	PTBXL-Super			PTBXL-Sub			PTBXL-Form			PTBXL-Rhythm			CPSC2018			CSN
Method	1%	10%	100%	1%	10%	100%	1%	10%	100%	1%	10%	100%	1%	10%	100%	1%	10%	100%
Random Init (CNN)	70.45	77.09	81.61	55.82	67.60	77.91	55.82	62.54	73.00	46.26	62.36	79.29	54.96	71.47	78.33	47.22	63.17	73.13
Random Init (Trans)	70.31	75.27	77.54	53.36	67.56	77.43	53.47	61.84	72.08	45.36	60.33	77.26	52.93	68.00	77.44	45.55	60.23	71.37
SimCLR [ 18 ]	63.41	69.77	73.53	60.84	68.27	73.39	54.98	56.97	62.52	51.41	69.44	77.73	59.78	68.52	76.54	59.02	67.26	73.20
BYOL [ 19 ]	71.70	73.83	76.45	57.16	67.44	71.64	48.73	61.63	70.82	41.99	74.40	77.17	60.88	74.42	78.75	54.20	71.92	74.69
BarlowTwins [ 20 ]	72.87	75.96	78.41	62.57	70.84	74.34	52.12	60.39	66.14	50.12	73.54	77.62	55.12	72.75	78.39	60.72	71.64	77.43

Table 1: Linear probing performance comparison between CG-DMER and existing ECG representation learning methods. The best and second-best results are highlighted in red and blue , respectively.

\dagger

denotes results reproduced by our implementation.

Methods	PTBXL-Super	PTBXL-Sub	PTBXL-Form	PTBXL-Rhythm	CPSC2018	CSN	Average
$\text{ETP}^{\dagger}$ [ 10 ]	49.63	62.25	57.44	66.56	67.13	60.22	60.54
MERL (ResNet) [ 11 ]	74.20	75.70	65.90	78.50	82.80	74.40	75.25
MERL (Vit) [ 11 ]	73.43	72.62	62.75	75.90	79.21	72.37	72.71
C-MELT [ 12 ]	75.12	74.38	65.55	79.87	81.61	74.43	75.16
CG-DMER (Ours)	75.65	74.62	66.44	80.77	82.93	75.59	76.00

Table 2: Zero-shot performance comparison across multiple datasets. The best and second-best results are highlighted in red and blue , respectively.

\dagger

denotes results reproduced by our implementation.

Methods	$\mathcal{L}_{e\_rec}$	$\mathcal{L}_{t\_rec}$	$\mathcal{L}_{orth}$	$\mathcal{L}_{SigLIP}$	Linear (1%)	Zero-shot
Variant 1	-	-	-	-	64.43	73.36
Variant 2	✓	-	-	-	67.39	74.87
Variant 3	✓	✓	-	-	67.82	75.00
Variant 4	✓	✓	✓	-	69.31	75.77
CG-DMER	✓	✓	✓	✓	70.15	76.00

Table 3: Effects of model components.

Figure 2: Results of CG-DMER with different patch numbers and masking ratio on linear probing and zero-shot.

Figure 3: T-SNE visualization on the CSN test set.

Explore similar work

Information-theoretic Multimodal Representation Learning for Electrocardiogram Signals
May 26, 2026Phu X. Nguyen, Konstantinos Kontras, Wei Dai +5Representation Learning Cross-Modal Representation Learning
1KU Leuven · 3MIT · 4DFKI +1
Learning ECG Image Representations via Dual Physiological-Aware Alignments
Apr 2, 2026Hung Manh Pham, Jialu Tang, Aaqib Saeed +3Medical Image Representation Learning Electrocardiography
University of Cambridge · Singapore Management University · Eindhoven University of Technology, Netherlands
Graph-Based Pseudo-multimodal Contrastive Learning for 12-Lead ECG Representations
Aug 27, 2026Mengyu Wang, Kozo Okada, Takafumi Goto +4Electrocardiogram Classification Time Series Representation Learning
Graduate School of Engineering Science Yokohama National University Yokohama, Japan · Division of Cardiology Yokohama City University Medical Center Yokohama, Japan · Technology and Innovation Department Fukuda Denshi Co.,Ltd Tokyo, Japan

May 26, 2026J/K move · Enter open · S save

Information-theoretic Multimodal Representation Learning for Electrocardiogram Signals

Phu X. Nguyen, Konstantinos Kontras, Wei Dai, Huy Phan+4

1KU Leuven · 3MIT · 4DFKI+1

Representation Learning Cross-Modal Representation Learning Self-Supervised Time Series Representation Learning Multimodal Pretraining Electrocardiogram Classification Multimodal Contrastive Learning

Open paper AskPDF arXiv

Electrocardiograms (ECGs) are widely used non-invasive measurements of cardiac activity and play a central role in clinical diagnosis. Recent multimodal approaches align ECG signals with clinical reports to incorporate diagnostic semantics, but clinical reports often fail to preserve the rich physiological structure of ECG waveforms, particularly across multiple levels of abstraction ranging from coarse diagnostic categories to fine-grained morphology. To address this limitation, we formulate ECG representation learning from an information-theoretic perspective and derive a tractable objective that jointly preserves signal structure and integrates clinical semantics. Based on this principle, we propose MERIT (Multimodal ECG Representation via Information Theory), a dual-branch pretraining framework combining masked ECG modeling with ECG--text contrastive alignment. Extensive experiments on PTB-XL and additional benchmarks demonstrate consistent improvements over prior methods, including gains exceeding

3%

F1 on PTB-XL All and

5%

F1 on SubClass classification. In zero-shot evaluation, MERIT further improves performance by up to

+2.66\%

AUC and

+2.11\%

F1 on PTB-XL SubClass, while also demonstrating robustness under multiple distribution-shift settings. Moreover, leveraging the learned ECG representations for ECG-conditioned clinical text generation with large language models improves text quality across several metrics, including ROUGE and METEOR. Together, these results demonstrate that MERIT learns more informative and clinically meaningful ECG representations, particularly for fine-grained clinical applications.

Abstract

Explore similar work

Information-theoretic Multimodal Representation Learning for Electrocardiogram Signals

Learning ECG Image Representations via Dual Physiological-Aware Alignments

Graph-Based Pseudo-multimodal Contrastive Learning for 12-Lead ECG Representations