cs.SDFeb 23, 2026

Continuous Telemonitoring of Heart Failure using Personalised Speech Dynamics

Authors: Yue Pan, Xingyao Wang, Hanyue Zhang, Liwei Liu, Changxin Li, Gang Yang, Rong Sheng, Yili Xia, 

Organizations: School of Information Science and Technology, Southeast University, Nanjing, China · Department of Cardiology, Chongqing Hospital of Jiangsu Province Hospital, Chongqing, China · Department of Cardiology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, China · Department of Cardiology, The Affiliated Taizhou People’s Hospital of Nanjing Medical University, Taizhou School of Clinical Medicine, Nanjing Medical University, Taizhou, China · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Remote monitoring of heart failure (HF) via speech signals provides a non-invasive and cost-effective solution for long-term patient management. However, substantial inter-individual heterogeneity in vocal characteristics often limits the accuracy of traditional cross-sectional classification models. To address this, we propose a Longitudinal Intra-Patient Tracking (LIPT) scheme designed to capture the trajectory of relative symptomatic changes within individuals. Central to this framework is a Personalised Sequential Encoder (PSE), which transforms longitudinal speech recordings into context-aware latent representations. By incorporating historical data at each timestamp, the PSE facilitates a holistic assessment of the clinical trajectory rather than modelling discrete visits independently. Experimental results from a cohort of 225 patients demonstrate that the LIPT paradigm significantly outperforms the classic cross-sectional approaches, achieving a recognition accuracy of 99.7% for clinical status transitions. The model's high sensitivity was further corroborated by additional follow-up data, confirming its efficacy in predicting HF deterioration and its potential to secure patient safety in remote, home-based settings. Furthermore, this work addresses the gap in existing literature by providing a comprehensive analysis of different speech task designs and acoustic features. Taken together, the superior performance of the LIPT framework and PSE architecture validates their readiness for integration into long-term telemonitoring systems, offering a scalable solution for remote heart failure management.

Figures & tables

Figure 1: Conceptual framework of the Longitudinal Intra-Patient Tracking (LIPT) paradigm for personalised HF monitoring. Unlike standard cross-sectional classification, which suffers from low accuracy due to inter-subject heterogeneity, the LIPT pipeline directly models intra-individual temporal sequences. The workflow comprises three primary stages: (a) Feature Extraction, in which speech signals are characterised by spectral (time-frequency), rhythmic, and glottal quality attributes at both global and frame-wise levels; (b) HF-Voice Feature Screening, which employs statistical significance tests to identify features with pathophysiological associations to HF, while correlations between various feature categories and the disease state are evaluated; and (c) Longitudinal Intra-Patient Tracking (LIPT), where sequential voice samples from each individual are processed by a Personalised Sequential Encoder (PSE) to capture longitudinal dependencies. Finally, a longitudinal classifier generates individualised tracking results, which are validated against clinical gold standards to monitor disease trajectories over time.
TaskAbbr.Chinese PinyinData Quantity
(Phonetic symbols)adm./dis.f-up (stable)f-up (re-ad.)
Vowel 1a-2187017
Vowel 2u-2137018
Vowel 3i-2207218
Short Sent. 1pgshan dong de ping guo you da you tian2177019
Short Sent. 2mmni you yi ge mei li de mei mei1976716
Table 1: Speech tasks and their respective data quantity. A total of 7 tasks were attempted, including 3 vowels, 3 short sentences, and 1 long sentence.
Global FeaturesFeature QuantityCross-SectionalLongitudinal Modeling
xgboostFNNxgboostFNN
all637351.155.756.868.2
HF-voice A177569.362.575.077.3
HF-voice B98165.969.370.581.8
RASTA270048.954.561.465.9
MFCC140058.052.361.468.2
Table 2: Performance benchmarking (Accuracy) for the cross-sectional and LIPT scheme under varying feature selection configurations
Figure 2: Performance of different speech tasks with the PSE model. The long sentence task ’count’ had the best macro-F1, followed by short sentences (’pg’, ’mm’, and ’mlh’) and vowels (’a’, ’i’, and ’u’). The long sentence task’s specificity was also the highest, albeit the sensitivity was slightly lower than that of the short sentences ’mm’ and ’mlh’. These two short sentences, focusing on voiced consonants, are also relatively balanced in sensitivity and specificity.
MethodFeaturesAcc.Prec.Sens.Spec.F1
XGBoostHF-voice A75.072.081.868.274.9
 HF-voice B70.571.468.272.770.4
FNNHF-voice A77.375.081.872.777.2
 HF-voice B81.885.077.386.481.8
PSE(ours)HF-voice A64.667.170.165.664.3
 HF-voice B65.261.957.164.864.2
Table 3: Results of XGBoost, FNN, and PSE under the LIPT scheme.
FeaturesAcc.Prec.Sens.Spec.F1
all24.721.694.46.022.9
MFCC25.916.461.116.425.9
RASTA70.641.594.464.267.6
Rhythm18.816.066.76.018.1
Quality48.23.65.559.734.4
RASTA+HF-voice A40.026.910023.940.0
Table 4: Results of the follow-up test set distinguishing rehospitalised patients.
Figure 3: Confusion matrix and Receiver Operating Characteristic (ROC) curve of the PSE on the follow-up data set, with the best settings overall (RASTA features with HF-voice feature sets A and B (see Section 3.4.3 )), with PSE models reported in section 2.2.3 . The most outstanding category of error is the false positive, indicating high confusion regarding the stable (non-hospitalised) class, where similar data is lacking in training. The relatively high area under the ROC curve (AUROC) suggests these errors stem from output misalignment rather than a lack of discriminative power.
Figure 4: Flowchart of the data collection process.
Figure 5: Architecture of the Personalised Sequential Encoder (PSE) and Longitudinal Classifier for speech-based monitoring. The framework consists of three operational stages: (1) Pretraining: The encoder acquires preliminary knowledge of speech patterns by reconstructing frame-level feature maps via a decoder. (2) Longitudinal Classifier Training: Frame-level features ( X0…XT−1X_{0}\dots X_{T-1} ) are processed through a convolutional network and aggregated with the global features to form the temporally-interpolated latent representations ( Z0…ZT−1Z_{0}\dots Z_{T-1} ). The system performs reference/target allocation to compare any pair of time points ( Zt1,Zt2Z_{t1},Z_{t2} ), using a linear layer to produce local comparison results ( y^∈[0,1]\hat{y}\in[0,1] ) optimised against clinical true labels. (3) Evaluation: The model integrates medical records and time stamps to produce final personal tracking results.
Feature Group Prefix in ComPare2016Description
NumberName
1SpectralaudspecSpectral features
2RASTAaudspecRastaRelative-Spectral filtered [ 12 ] spectral features
3ZCRpcm_zcrZero-crossing rate of time signal
4RMS Energypcm_RMSenergyRoot-mean-square signal frame energy
5FFTpcm_fftMagFast Fourier transform
Table 5: Description of the global feature set.
Feature GroupDescriptionID of features
No. Name (see Appendix B )
1 Rhythm (energy)Rhythm features correlated with energy6, 8, 36, 37, 42, 48, 50, 68, 70
2 Rhythm (zcr)Rhythm features correlated with zcr9, 38-41, 43-47, 49, 69
3 MFCCMel-Frequency cepstral coefficients 0-1351-64
4 QualityJitter, Shimmer [ 11 ] , logHNR [ 4 ] , cpp [ 9 ]2-5, 65-67
5 RASTARelative Spectral Transform (RASTA) [ 12 ]7, 10-35
Table 6: Description of the frame-level feature set.
Figure 6: Feature selection quantity and percentage with respect to the original global feature set. The spectral features (G1) are the highest in percentage in both paired and independent t-test selections, which are followed by the global RASTA features (G2). The RASTA features by frequency bands (G6) contribute the most in terms of absolute quantity in both cases.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Feature IDFeature NameSource
0F0final_smaCompare_2016 (openSMILE)
1voicingFinalUnclipped_sma
2jitterLocal_sma
3jitterDDP_sma
4shimmerLocal_sma
5logHNR_sma
Appendix
Table 7: List of frame-level features. Extracted with frame window of 200 msms and step length of 100 msms .
Figure 7: Frame-level features’ Pearson correlation matrix. Brighter colour indicate higher relevance between the two features. The numbers on each axis are the feature ID (see Appendix A ).
More in this topic

Remote Patient Monitoring

See all 9 papers

Explore similar work

CardsList

  1. Vocal Prognostic Digital Biomarkers in Monitoring Chronic Heart Failure: A Longitudinal Observational Study

    Mar 31, 2026Fan Wu, Matthias P. Nägele, Daryush D. Mehta +4Clinical Outcome PredictionPassive Acoustic Monitoring

    Centre for Digital Health Interventions, ETH Zurich, Zurich, Switzerland · Agentic System Lab, ETH Zurich, Zurich, Switzerland · Cardiology, University Heart Center Zurich, University Hospital Zurich, Zurich, Switzerland +5

  2. AI-based detection of worsening heart failure from low-resolution telemonitoring data

    Sep 24, 2026Erik Aerts, Yinan Yu, Annika Rosengren +5Remote Patient MonitoringTime Series Transformers

    Department of Computer Science and Engineering, Chalmers Universiy of Technology, Gothenburg, Sweden · Department of Computer Science and Engineering, University of Gothenburg, Gothenburg, Sweden · Center for Digital Health, Sahlgrenska University Hospital, Gothenburg, Sweden +3

  3. Is Clinical Text Enough? A Multimodal Study on Mortality Prediction in Heart Failure Patients

    Apr 2, 2026Oumaima El Khettari, Virgile Barthet, Guillaume Hocquet +3Mortality PredictionMultimodal Fusion

    Nantes Univ., École Centrale Nantes, CNRS, LS2N, UMR 6004, 44000 Nantes, France · Université Paris-Saclay, CNRS, LISN, Orsay, France · Hôpital Saint Joseph, DIMID, et Service de Cardiologie, Paris, France