TransECG-SSL: Hybrid Self-Supervised Representation Learning for ECG Arrhythmia Classification

Ahtisham Ayyub; Fasih Haider; Muhammad Rehan Usman; Saturnino Luz; Christos Politis; Muhammad Arslan Usman

cs.LGOct 26, 2026

TransECG-SSL: Hybrid Self-Supervised Representation Learning for ECG Arrhythmia Classification

Authors: Ahtisham Ayyub, Fasih Haider, Muhammad Rehan Usman, Saturnino Luz, Christos Politis, Muhammad Arslan Usman

Organizations: School of Computer Science and Mathematics, Kingston University, London, United Kingdom · Usher Institute, University of Edinburgh, Edinburgh, United Kingdom

Abstract

Cross-dataset generalisation remains a major challenge for automated electrocardiogram (ECG) arrhythmia classification because of differences in patient populations, acquisition protocols, annotation practices, and class distributions. Learning representations that remain informative across independent datasets could reduce reliance on dataset-specific labelled data. We propose TransECG-SSL, a self-supervised framework for transferable heartbeat representation learning from unlabelled ECG data. A Transformer encoder is pretrained using a hybrid objective combining masked signal reconstruction and contrastive learning, then frozen and evaluated by linear probing under patient-disjoint evaluation. Experiments across three public ECG databases and seven source--target settings show that Hybrid SSL consistently outperforms randomly initialised Transformers and achieves higher balanced accuracy than architecture-matched source-supervised pretraining. Reconstruction-only SSL remains competitive with Hybrid SSL, whereas contrastive-only pretraining performs substantially worse, highlighting the importance of masked reconstruction for transferability. Cross-dataset transfer nevertheless reduces balanced accuracy relative to target-specific self-supervised pretraining, while minority arrhythmia classes remain challenging under severe class imbalance. These findings demonstrate the potential of label-efficient self-supervised learning for cross-dataset ECG representation transfer while highlighting persistent limitations from domain shift and class imbalance.

Figures & tables

Component	Setting
Input source	WFDB ECG records and reference beat annotations.
ECG lead	First available ECG lead was used consistently across datasets.
Bandpass filtering	Fourth-order Butterworth bandpass filter from 0.5–40 Hz using zero-phase second-order-section filtering.
Resampling	Continuous ECG signals were resampled to 256 Hz before beat extraction. R-peak locations were mapped to the resampled signal index space.
Beat alignment	Beats were centred on the annotated R-peak.
Window length	256 samples per beat.

TABLE I: ECG preprocessing and beat-segmentation parameters used in TransECG-SSL.

Dataset	Split	Pts.	Recs.	N	S	V	F	Q	Total
MIT	Train	31	31	56,914	1,861	4,802	435	6,038	70,050
	Test	16	17	33,668	920	2,433	367	2,132	39,520
INC	Train	21	51	104,015	1,333	14,351	147	4	119,850
	Test	11	24	49,581	625	5,655	72	2	55,935
SVDB	Train	52	52	108,676	8,167	6,656	–	–	123,499
	Test	26	26	53,575	4,028	3,282	–	–	60,885

TABLE II: Patient-disjoint class-aware train/test splits used in the final experiments.

Experiment	SSL pretraining source	Frozen embedding extraction	Evaluated classes	Purpose
MIT-BIH $\rightarrow$ MIT-BIH	MIT-BIH training partition	MIT-BIH patient-disjoint training and test partitions	N/S/V/F/Q	Target-matched reference for MIT-BIH.
INCART $\rightarrow$ INCART	INCART training partition	INCART patient-disjoint training and test partitions	N/S/V/F/Q	Target-matched reference for INCART.
SVDB $\rightarrow$ SVDB	SVDB training partition	SVDB patient-disjoint training and test partitions	N/S/V	Target-matched reference for SVDB.
MIT-BIH $\rightarrow$ INCART	MIT-BIH training partition	INCART patient-disjoint training and test partitions	N/S/V/F/Q	Cross-dataset transfer from MIT-BIH to INCART.
INCART $\rightarrow$ MIT-BIH	INCART training partition	MIT-BIH patient-disjoint training and test partitions	N/S/V/F/Q	Cross-dataset transfer from INCART to MIT-BIH.
MIT-BIH $\rightarrow$ SVDB	MIT-BIH training partition	SVDB patient-disjoint training and test partitions	N/S/V	Cross-dataset transfer from MIT-BIH to SVDB.

TABLE III: Frozen representation transfer and linear-probe evaluation matrix used in the final TransECG-SSL experiments.

Experiment	Acc.	BA	A–BA gap	F1	AUROC	AUPRC	MCC	$\kappa$
MIT $\rightarrow$ MIT	$.848\pm.008$	$.582\pm.012$	$.266$	$.497\pm.013$	$.902\pm.011$	$.592\pm.014$	$.562\pm.014$	$.543\pm.015$
INC $\rightarrow$ INC	$.845\pm.033$	$.383\pm.014$	$.461$	$.324\pm.017$	$.833\pm.024$	$.367\pm.022$	$.553\pm.050$	$.503\pm.062$
SVDB $\rightarrow$ SVDB	$.830\pm.014$	$.646\pm.022$	$.184$	$.565\pm.023$	$.846\pm.017$	$.637\pm.017$	$.405\pm.029$	$.391\pm.031$
MIT $\rightarrow$ INC	$.874\pm.022$	$.367\pm.007$	$.507$	$.325\pm.012$	$.815\pm.029$	$.349\pm.020$	$.559\pm.051$	$.532\pm.058$
INC $\rightarrow$ MIT	$.867\pm.006$	$.551\pm.012$	$.316$	$.485\pm.013$	$.886\pm.010$	$.584\pm.013$	$.576\pm.006$	$.566\pm.009$
MIT $\rightarrow$ SVDB	$.876\pm.015$	$.556\pm.037$	$.320$	$.562\pm.035$	$.866\pm.026$	$.561\pm.054$	$.392\pm.052$	$.388\pm.050$

TABLE IV: Overall transfer performance of the proposed Hybrid SSL encoder.

Train $\rightarrow$ Test	Hyb.	Rec.	Con.	Src.	Rand.
Balanced accuracy
MIT $\rightarrow$ MIT	$.582\!\pm\!.012$	$.586\!\pm\!.018$	$.242\!\pm\!.029$	$.560\!\pm\!.029$	$.482\!\pm\!.031$
INC $\rightarrow$ INC	$.383\!\pm\!.014$	$.387\!\pm\!.008$	$.221\!\pm\!.020$	$.369\!\pm\!.007$	$.281\!\pm\!.014$
SVDB $\rightarrow$ SVDB	$.646\!\pm\!.022$	$.614\!\pm\!.024$	$.336\!\pm\!.005$	$.567\!\pm\!.060$	$.388\!\pm\!.018$
MIT $\rightarrow$ INC	$.367\!\pm\!.007$	$.371\!\pm\!.015$	$.202\!\pm\!.003$	$.332\!\pm\!.025$	$.281\!\pm\!.014$
INC $\rightarrow$ MIT	$.551\!\pm\!.012$	$.518\!\pm\!.023$	$.352\!\pm\!.077$	$.428\!\pm\!.071$	$.482\!\pm\!.031$

TABLE V: Architecture-matched Transformer comparison under frozen target linear probing.

Metric	MIT-BIH	INCART	SVDB
Accuracy	$0.816\pm 0.076$	$0.840\pm 0.028$	$0.850\pm 0.029$
Balanced Accuracy	$0.550\pm 0.030$	$0.361\pm 0.011$	$0.551\pm 0.031$
Macro-F1	$0.459\pm 0.030$	$0.305\pm 0.006$	$0.538\pm 0.044$
Macro-AUROC	$0.861\pm 0.034$	$0.774\pm 0.043$	$0.834\pm 0.008$
Macro-AUPRC	$0.504\pm 0.040$	$0.314\pm 0.026$	$0.548\pm 0.045$
MCC	$0.519\pm 0.092$	$0.519\pm 0.045$	$0.339\pm 0.061$

TABLE VI: Target-supervised performance of the end-to-end Transformer model. Values are reported as mean

\pm

standard deviation.

Condition	Trainable parameters
Hybrid SSL	3,482,176
Reconstruction-only SSL	3,457,408
Contrastive-only SSL	3,383,360
Source-supervised (5-class)	3,359,237
Source-supervised (3-class)	3,358,979
Target-supervised (5-class)	3,359,237

TABLE VII: Trainable parameter counts across architecture-matched conditions. All conditions use the same Transformer encoder with 3,358,592 parameters.

Transfer	Hyb. BA ( $\Delta$ BA)	Rec. BA ( $\Delta$ BA)	Src. BA ( $\Delta$ BA)
MIT $\rightarrow$ INC	$.367\,(-.016)$	$\textbf{.371}\,(-.016)$	$.332\,(-.038)$
INC $\rightarrow$ MIT	$\textbf{.551}\,(-.031)$	$.518\,(-.068)$	$.428\,(-.132)$
MIT $\rightarrow$ SVDB	$\textbf{.556}\,(-.091)$	$.518\,(-.096)$	$.486\,(-.080)$
INC $\rightarrow$ SVDB	$\textbf{.589}\,(-.057)$	$.587\,(-.027)$	$.429\,(-.138)$

TABLE VIII: Cross-dataset performance and target-normalised transfer gaps. Panel A compares balanced accuracy (BA) and Panel B compares macro-F1 across the three principal representation-learning strategies.

Fig. 2: Aggregated row-normalised confusion matrices for Hybrid SSL across the seven source–target routes.

Experiment	Classes	BA	Macro-F1
INCART $\rightarrow$ INCART	5-class	$0.3834\pm 0.0142$	$0.3238\pm 0.0169$
	Excl. Q	$\mathbf{0.4792\pm 0.0177}$	$\mathbf{0.4048\pm 0.0211}$
MIT-BIH $\rightarrow$ INCART	5-class	$0.3669\pm 0.0073$	$0.3248\pm 0.0124$
	Excl. Q	$\mathbf{0.4586\pm 0.0091}$	$\mathbf{0.4060\pm 0.0155}$

TABLE IX: Effect of excluding the sparsely represented Q class from INCART-target evaluation. Values are mean

\pm

SD across five runs.

Exp.	C	Sup.	Rec.	PPV	F1	AUROC	AUPRC
INC $\rightarrow$ INC	N	49,581	$.841\pm.036$	$.986\pm.001$	$.907\pm.022$	$.939\pm.009$	$.990\pm.001$
	S	625	$.104\pm.049$	$.041\pm.040$	$.055\pm.047$	$.602\pm.028$	$.031\pm.024$
	V	5,655	$.973\pm.005$	$.497\pm.044$	$.657\pm.037$	$.974\pm.012$	$.807\pm.103$
	F	72	$.000\pm.000$	$.000\pm.000$	$.000\pm.000$	$.731\pm.031$	$.004\pm.002$
	Q	2	$.000\pm.000$	$.000\pm.000$	$.000\pm.000$	$.917\pm.085$	$.005\pm.004$
MIT $\rightarrow$ INC	N	49,581	$.885\pm.024$	$.975\pm.003$	$.928\pm.014$	$.924\pm.019$	$.987\pm.003$

TABLE X: Classwise Hybrid SSL performance for the two INCART-target experiments.

Comparison	Outcome	Evidence
Hybrid SSL vs. Random Transformer	Hybrid SSL	Higher balanced accuracy and macro-F1 in all seven source–target evaluation routes.
Hybrid SSL vs. Source-supervised Transformer	Hybrid SSL for BA	Higher balanced accuracy in all seven routes. Macro-F1 was higher for Hybrid SSL in 4/7 routes and for the source-supervised Transformer in 3/7 routes.
Hybrid SSL vs. Reconstruction-only SSL	Mixed	Hybrid SSL achieved higher balanced accuracy in 4/7 routes and higher macro-F1 in 5/7 routes, confirming that the incremental effect of the contrastive objective was route- and metric-dependent.
Hybrid SSL vs. Contrastive-only SSL	Hybrid SSL	Contrastive-only SSL produced substantially lower balanced accuracy and macro-F1 across all seven evaluation routes.
Cross-dataset generalisation	Hybrid SSL in 3/4 routes	Hybrid SSL achieved the highest cross-dataset balanced accuracy and macro-F1 in three of four transfer routes; reconstruction-only SSL was highest for MIT-BIH $\rightarrow$ INCART. Target-normalised degradation remained method and route-dependent.
Minority-class recognition	Unresolved	Near-zero fusion-beat (F) recall and consistently low supraventricular ectopic-beat (S) recall in several five-class evaluations.

TABLE XI: Summary of the principal experimental findings.

Electrocardiogram Representation Learning

See all 35 papers

ECG-NAT: A Self-supervised Neighborhood Attention Transformer for Multi-lead Electrocardiogram Classification
May 13, 2026Mahsa Gazeran, Sayvan Soleymanbaigi, Fatemeh Daneshfar +2Self-Supervised Time Series Representation Learning Transformer Attention
Department of Computer Engineering, University of Kurdistan, Sanandaj, Iran · Department of Mathematics and Operational Research, University of Mons, Mons, Belgium
CoRe-ECG: Advancing Self-Supervised Representation Learning for 12-Lead ECG via Contrastive and Reconstructive Synergy
Apr 13, 2026Zehao Qin, Xiaojian Lin, Ping Zhang +6Electrocardiogram Representation Learning Representation Learning
Department of Electronic Engineering, Shenzhen International Graduate School, Tsinghua University, Shenzhen 518071, PR China · Department of Cardiology, Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua Medicine, Tsinghua University, Beijing, 102218, PR China · Department of Anesthesiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, PR China +3
A Hybrid CNN--State-Space--Attention Backbone with Joint-Embedding Predictive Pretraining for 12-Lead ECG Classification
Sep 24, 2026Yakoub Bazi, Sarah Aljuhani, Mohamad M. Al Rahhal +2Electrocardiography Self-Supervised Time Series Representation Learning
Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh, 11543, Saudi Arabia · Applied Computer Science Department, College of Applied Computer Science, King Saud University, Riyadh, 11543, Saudi Arabia

May 13, 2026J/K move · Enter open · S save

ECG-NAT: A Self-supervised Neighborhood Attention Transformer for Multi-lead Electrocardiogram Classification

Mahsa Gazeran, Sayvan Soleymanbaigi, Fatemeh Daneshfar, Amjad Seyedi+1

Department of Computer Engineering, University of Kurdistan, Sanandaj, Iran · Department of Mathematics and Operational Research, University of Mons, Mons, Belgium

Self-Supervised Time Series Representation Learning Transformer Attention Electrocardiogram Classification Data-Efficient Learning

Open paper AskPDF arXiv

Electrocardiogram (ECG) arrhythmia classification remains challenging due to signal variability, noise, limited labeled data, and the difficulty in achieving both accuracy and efficiency in models. While self-supervised learning reduces label dependency, most methods target either global contextual features or local morphological patterns, but rarely implement hierarchical multi-scale feature extraction. ECG signals require architectures that simultaneously capture fine-grained beat-level morphology and broader rhythm-level dependencies with computational efficiency. To overcome this limitation, this paper proposes the Electrocardiogram Neighborhood Attention Transformer (ECG-NAT), a novel self-supervised learning approach tailored for multi-lead ECG classification. Our two-stage approach begins with generative pretraining, using a masked autoencoder to reconstruct partially masked ECG signals across multiple diverse datasets, enabling the model to learn robust, domain-invariant representations from unlabeled data. This is followed by discriminative fine-tuning with a dual-loss function that combines supervised contrastive and cross-entropy losses, aligning representation learning with label prediction. The hierarchical attention mechanism efficiently captures multi-scale temporal features from localized beat morphology to broader rhythm patterns at low computational cost. ECG-NAT achieves robust performance on benchmark datasets, with 88.1% accuracy using only 1% labeled data, demonstrating strong efficacy in low-resource settings. The framework combines superior classification performance with computational efficiency, making it practical for real-time ECG diagnosis. The code will be made available upon acceptance at: https://github.com/Mahsagazeran/ECG-NAT.

Abstract

Electrocardiogram Representation Learning

Explore similar work

ECG-NAT: A Self-supervised Neighborhood Attention Transformer for Multi-lead Electrocardiogram Classification

CoRe-ECG: Advancing Self-Supervised Representation Learning for 12-Lead ECG via Contrastive and Reconstructive Synergy

A Hybrid CNN--State-Space--Attention Backbone with Joint-Embedding Predictive Pretraining for 12-Lead ECG Classification