ENACT-Heart -- ENsemble-based Assessment Using CNN and Transformer on Heart Sounds

Jiho Han; Adnan Shaout

cs.SDFeb 24, 2025

ENACT-Heart -- ENsemble-based Assessment Using CNN and Transformer on Heart Sounds

Authors: Jiho Han, Adnan Shaout

Organizations: Industrial AI Lab, SimPlatform Co. Ltd. Affiliate Research Institute, Geumcheon-gu, Seoul, Republic of Korea · Department of Computer Science and Engineering, The University of Michigan - Dearborn, Dearborn, MI, USA · Department of Electrical and Computer Engineering, The University of Michigan – Dearborn, Dearborn, MI, USA

Abstract

ENACT-Heart is a two-branch ensemble for heart-sound classification. A Vision Transformer (ViT) processes spectrogram images, while a convolutional neural network (CNN) processes visualizations of waveforms and spectral centroids derived from the same recordings. The branch predictions are combined by a weighted average with fixed, input-independent weights. On the PASCAL heart-sound dataset, the ensemble achieved an accuracy of 97.52%, compared with 93.88% for the ViT and 95.45% for the CNN. The ensemble achieved a macro-F1 of 0.94 and recall of 0.74 for extra heart sounds. On this dataset, late fusion improved accuracy and macro-F1 relative to both individual branches.

Figures & tables

Study	Model and fusion strategy	Data and task
Malik et al. [ 1 ]	MFCC features; LSTM classifier	PASCAL; PhysioNet 2017.
Yang et al. [ 2 ]	Transformer classifier	Five categories: normal and four valve diseases.
Wang et al. [ 3 ]	Parallel CNN–Transformer; feature-level fusion	Yaseen; PhysioNet 2016, evaluated separately.
Cheng and Sun [ 4 ]	1D convolution followed by a Transformer	Three datasets; binary and multiclass tasks.
Liu et al. [ 5 ]	Bispectrum features; Vision Transformer	PhysioNet 2022; normal/abnormal classification.
Jumphoo et al. [ 6 ]	Convolution–DeiT with transfer learning	Heart Sound Murmur database; valve-disease classification.

TABLE I: Related methods and evaluation scope.

Panel of Figure 2 — Fig. 2: Data preparation for the two model branches: clip-length standardization, Gaussian-noise augmentation, and generation of spectrogram and waveform–spectral-centroid images.

		Macro average			Weighted average
Model	Accuracy (%)	Precision	Recall	F1	Precision	Recall	F1
ViT	93.88	0.89	0.88	0.88	0.94	0.94	0.94
CNN	95.45	0.94	0.89	0.91	0.95	0.95	0.95
ENACT-Heart	97.52	0.96	0.92	0.94	0.97	0.98	0.97

TABLE II: Overall classification performance on the pooled PASCAL dataset. Boldface denotes the highest value for each metric.

	ViT			CNN			ENACT-Heart
Class	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
Artifact	0.80	0.96	0.87	0.92	0.93	0.92	0.93	0.97	0.95
Extra heart sound	0.73	0.65	0.69	0.86	0.67	0.76	0.89	0.74	0.81
Extrasystole	0.98	0.84	0.91	0.98	0.87	0.92	1.00	0.91	0.96
Murmur	0.94	0.97	0.96	0.95	0.97	0.96	0.98	0.99	0.99
Normal	0.99	0.95	0.97	0.97	0.98	0.98	0.98	0.99	0.99

TABLE III: Class-level classification performance on the pooled PASCAL dataset. Boldface denotes the highest value across models for each class and metric.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Label	Subset	Dataset description
Normal	A, B	Recognizable S1–S2 pattern.
Murmur	A, B	Murmur-like sound between principal sounds.
Extra heart sound	A	Additional sound beyond S1–S2.
Extrasystole	B	Rhythm disturbance; extra or skipped beat.
Artifact	A	Non-cardiac sound or interference.

Appendix

TABLE A.I: Label definitions and source subsets in PASCAL CHSC2011 [ 7 ] .

Panel of Figure 6 — Fig. B.1: Confusion-matrix panels for four heart-sound categories. Each panel uses its own color scale. The figure does not include an artifact panel, numerical cell annotations or prediction-axis labels.

Heart Sound Classification

See all 8 papers

Scaling to Multimodal and Multichannel Heart Sound Classification with Synthetic and Augmented Biosignals
Sep 15, 2025Milan Marocchi, Matthew Fynn, Kayapanda Mandana +1Synthetic Data Augmentation Heart Sound Classification
School of Electrical Engineering, Computing, and Mathematical Sciences (EECMS), Faculty of Science and Engineering, Curtin University, Bentley, WA 6102, Australia · Department of Cardiology, Fortis Healthcare, Kolkata, 7007107, West Bengal, India
Selection of Heart Sound Segments for Synchronous Classification of Multi-channel Heart Sounds
Aug 21, 2026Marcelo Nogueira, Jorge H. Oliveira, Carlos F. Ferreira +2Heart Sound Classification Audio Classification
INESC TEC & Faculdade de Ciências, Universidade do Porto · LIACC, Universidade da Maia · INESC TEC & Instituto Superior de Engenharia do Porto
Stress Classification from ECG Signals Using Vision Transformer
Mar 19, 2026Zeeshan Ahmad, Naimul KhanElectrocardiogram Classification Electrocardiography

Sep 15, 2025J/K move · Enter open · S save

Scaling to Multimodal and Multichannel Heart Sound Classification with Synthetic and Augmented Biosignals

Milan Marocchi, Matthew Fynn, Kayapanda Mandana, Yue Rong

School of Electrical Engineering, Computing, and Mathematical Sciences (EECMS), Faculty of Science and Engineering, Curtin University, Bentley, WA 6102, Australia · Department of Cardiology, Fortis Healthcare, Kolkata, 7007107, West Bengal, India

Synthetic Data Augmentation Heart Sound Classification Multimodal Classification Audio Classification

Open paper AskPDF arXiv

Cardiovascular diseases (CVDs) are the leading cause of death worldwide, accounting for approximately 17.9 million deaths each year. Early detection is critical, creating a demand for accurate and inexpensive pre-screening methods. Deep learning has recently been applied to classify abnormal heart sounds indicative of CVDs using synchronised phonocardiogram (PCG) and electrocardiogram (ECG) signals, as well as multichannel PCG (mPCG). However, state-of-the-art architectures remain underutilised due to the limited availability of synchronised and multichannel datasets. Augmented datasets and pre-trained models provide a pathway to overcome these limitations, enabling transformer-based architectures to be trained effectively. This work combines traditional signal processing with denoising diffusion models, WaveGrad and DiffWave, to create an augmented dataset to fine-tune a Wav2Vec 2.0-based classifier on multimodal and multichannel heart sound datasets. The approach achieves state-of-the-art performance. On the Computing in Cardiology (CinC) 2016 dataset of single channel PCG, accuracy, unweighted average recall (UAR), sensitivity, specificity and Matthew's correlation coefficient (MCC) reach 92.48%, 93.05%, 93.63%, 92.48%, 94.93% and 0.8283, respectively. Using the synchronised PCG and ECG signals of the training-a dataset from CinC, 93.14%, 92.21%, 94.35%, 90.10%, 95.12% and 0.8380 are achieved for accuracy, UAR, sensitivity, specificity and MCC, respectively. Using a wearable vest dataset consisting of mPCG data, the model achieves 77.13% accuracy, 74.25% UAR, 86.47% sensitivity, 62.04% specificity, and 0.5082 MCC. These results demonstrate the effectiveness of transformer-based models for CVD detection when supported by augmented datasets, highlighting their potential to advance multimodal and multichannel heart sound classification.

Abstract

Heart Sound Classification

Explore similar work

Scaling to Multimodal and Multichannel Heart Sound Classification with Synthetic and Augmented Biosignals

Selection of Heart Sound Segments for Synchronous Classification of Multi-channel Heart Sounds

Stress Classification from ECG Signals Using Vision Transformer