Vocal Prognostic Digital Biomarkers in Monitoring Chronic Heart Failure: A Longitudinal Observational Study
Authors: Fan Wu, Matthias P. Nägele, Daryush D. Mehta, Elgar Fleisch, Frank Ruschitzka, Andreas J. Flammer, Filipe Barata
Organizations: Centre for Digital Health Interventions, ETH Zurich, Zurich, Switzerland · Agentic System Lab, ETH Zurich, Zurich, Switzerland · Cardiology, University Heart Center Zurich, University Hospital Zurich, Zurich, Switzerland · Center for Laryngeal Surgery and Voice Rehabilitation, Massachusetts General Hospital, Boston, MA · Department of Surgery, Harvard Medical School, Boston, MA · Speech and Hearing Bioscience and Technology, Division of Medical Sciences, Harvard Medical School, Boston, MA · MGH Institute of Health Professions, Boston, MA · Centre for Digital Health Interventions, University of St. Gallen, St. Gallen, Switzerland
Objective: This study aimed to evaluate which voice features can predict health deterioration in patients with chronic HF. Background: Heart failure (HF) is a chronic condition with progressive deterioration and acute decompensations, often requiring hospitalization and imposing substantial healthcare and economic burdens. Current standard-of-care (SoC) home monitoring, such as weight tracking, lacks predictive accuracy and requires high patient engagement. Voice is a promising non-invasive biomarker, though prior studies have mainly focused on acute HF stages. Methods: In a 2-month longitudinal study, 32 patients with HF collected daily voice recordings and SoC measures of weight and blood pressure at home, with biweekly questionnaires for health status. Acoustic analysis generated detailed vowel and speech features. Time-series features were extracted from aggregated lookback windows (e.g., 7 days) to predict next-day health status. Explainable machine learning with nested cross-validation identified top vocal biomarkers, and a case study illustrated model application. Results: A total of 21,863 recordings were analyzed. Acoustic vowel features showed strong correlations with health status. Time-series voice features within the lookback window outperformed corresponding standard care measures, achieving peak sensitivity and specificity of 0.826 and 0.782 versus 0.783 and 0.567 for SoC metrics. Key prognostic voice features identifying deterioration included delayed energy shift, low energy variability, and higher shimmer variability in vowels, along with reduced speaking and articulation rate, lower phonation ratio, decreased voice quality, and increased formant variability in speech. Conclusion: Voice-based monitoring offers a non-invasive approach to detect early health changes in chronic HF, supporting proactive and personalized care.
Figures & tables
Figure 1: Voice-based monitoring system for chronic HF . We conducted a home-based longitudinal study in which patients recorded daily speech and vowels. To develop and validate vocal biomarkers, we designed a pipeline including acoustic feature extraction, time-series analysis, and explainable machine-learning prediction of health deterioration, resulting in the identification of novel predictive biomarkers. Voice features outperformed SoC measures, highlighting their potential for earlier detection and proactive management to reduce hospitalizations.
Sum of auditory spectrum (loudness); F0 (pitch); RMS energy; zero-crossing rate
Speech features
Feature Type
Features
Table 1: Baseline Demographics and Clinical Characteristics (N = 32)
Variable
V1
V2
S
Age (years)
55.69±11.28
Sex
Male (90.62%), Female (9.38%)
Smoking
Stopped (78.12%), Yes (12.5%), Never (9.38%)
Height (cm)
173.78±9.23
Native German speaker
Yes (81.25%), No (18.75%)
Last LVEF (%)
27.89±9.07
Table 2: Baseline Demographics and Clinical Characteristics (N = 32)
Figure 2: Repeated measures correlation between health status and dynamic features . Health status is represented by the KCCQ overall summary score. Shown are the standard deviations of selected features over a 7-day lookback window. Each color represents one patient. Panels include: (a) max MFCC of vowel /a/; (b) mean pause duration (command task); (c–d) blood pressure; (e) weight; (f) HFaST score. r indicates the correlation coefficient, and p the significance level.
Figure 3: Classification performance comparison across different feature sets. (a) Sensitivity, (b) Specificity, (c) F1 score, and (d) MCC are reported. Each dot represents the mean performance of a feature set across multiple measurements for a given lookback window (K = 2–14 days). The shaded area shows the SoC performance, used as the baseline for comparison.
Figure 4: ROC and Precision–Recall curves for different feature sets. (a) AUC and (b) AUPRC are shown to indicate performance in distinguishing health deterioration. Each line represents a specific feature set using a lookback window of K = 9 days.
Figure 5: SHAP global and summary plots for the voice feature set. The top vowel (a) and speech (b) features are shown, with positive SHAP values indicating contributions toward worse health outcomes. Feature labels are simplified codes; see Supplementary Material for more descriptions.
Feature
Best states
Worst states
Effect size
Phonation ratio (Text)
0.914 (0.048)
0.826 (0.158)
0.846
Speaking rate (Text)
4.061 (0.231)
3.347 (0.958)
0.805
F1 Std (Command)
16.771 (6.895)
29.279 (15.859)
0.702
Articulation rate (Text)
4.446 (0.325)
4.078 (0.644)
0.662
Shimmer Q3 (/O/)
8.309e − 07 (7.032e − 07)
2.000e − 06 (2.771e − 06)
0.638
F1 FFT1 (Command)
70.583 (39.714)
118.773 (63.202)
0.626
Table 3: Statistical comparison of top features between best and worst health states.
Figure 6: Feature illustration and deterioration prediction for the case study patient. (a) Key feature fluctuations (weight, diastolic pressure, text speaking rate) with mean values per lookback window K = 9 days (orange) and KCCQ scores (dark red). (b) Predicted deterioration probabilities (blue) versus KCCQ trajectory (black), with deteriorating (red) and healthy (green) regions; KCCQ axis reversed so upward indicates deteriorating.
Remote monitoring of heart failure (HF) via speech signals provides a non-invasive and cost-effective solution for long-term patient management. However, substantial inter-individual heterogeneity in vocal characteristics often limits the accuracy of traditional cross-sectional classification models. To address this, we propose a Longitudinal Intra-Patient Tracking (LIPT) scheme designed to capture the trajectory of relative symptomatic changes within individuals. Central to this framework is a Personalised Sequential Encoder (PSE), which transforms longitudinal speech recordings into context-aware latent representations. By incorporating historical data at each timestamp, the PSE facilitates a holistic assessment of the clinical trajectory rather than modelling discrete visits independently. Experimental results from a cohort of 225 patients demonstrate that the LIPT paradigm significantly outperforms the classic cross-sectional approaches, achieving a recognition accuracy of 99.7% for clinical status transitions. The model's high sensitivity was further corroborated by additional follow-up data, confirming its efficacy in predicting HF deterioration and its potential to secure patient safety in remote, home-based settings. Furthermore, this work addresses the gap in existing literature by providing a comprehensive analysis of different speech task designs and acoustic features. Taken together, the superior performance of the LIPT framework and PSE architecture validates their readiness for integration into long-term telemonitoring systems, offering a scalable solution for remote heart failure management.
Yue Pan, Xingyao Wang, Hanyue Zhang +6
School of Information Science and Technology, Southeast University, Nanjing, China · Department of Cardiology, Chongqing Hospital of Jiangsu Province Hospital, Chongqing, China · Department of Cardiology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, China +3
Objective: Heart failure (HF) presents a healthcare challenge due to its high comorbidity burden, aging patient population and frequent hospitalizations. Remote monitoring offers a promising approach to managing HF patients by early detection of health deterioration. Developing autonomous systems to detect signs of worsening in telemonitoring data is of interest to reduce the workload of healthcare personnel. Methods: We propose the TRACER model, a Transformer with Contrastive Event Representation, designed to predict timelines leading to rare hospitalization events in low-resolution and irregularly sampled telemonitoring data. TRACER incorporates time-aware embeddings for each biomarker, contrastive pre-training to enhance anomaly detection via representation learning, and independent binary classifiers for detection. We used measurement data containing remotely recorded biomarker sequences from 276 HF patients segmented into overlapping windows based on temporal rules, and labeled the windows based on the occurrence of HF relevant hospitalizations at the latter edge of the window. Results: TRACER was able to correctly predict 66.7% timelines leading up to HF hospitalizations in the highly imbalanced real-world dataset with an overestimation of 7.9%. Reformulating the training of TRACER as an event detection problem improved the predictive performance compared with training directly on forecasting windows, enabling more effective use of the limited hospitalization events. Conclusion: TRACER demonstrated superior performance in detecting signs of worsening status in real-world telemonitoring data compared to the other tested models. Significance: TRACER shows promise in identifying signs of clinical deterioration that allow for alerts to be generated to provide counteractive treatment in patients with HF.
Erik Aerts, Yinan Yu, Annika Rosengren +5
Department of Computer Science and Engineering, Chalmers Universiy of Technology, Gothenburg, Sweden · Department of Computer Science and Engineering, University of Gothenburg, Gothenburg, Sweden · Center for Digital Health, Sahlgrenska University Hospital, Gothenburg, Sweden +3
Hospital readmission within 30 days of discharge is a leading driver of morbidity, mortality, and avoidable healthcare expenditure in congestive heart failure (CHF). Current clinical risk stratification tools rely primarily on non-imaging data and exhibit limited predictive performance. Point-of-care lung ultrasound (LUS) offers a sensitive, noninvasive window into the pulmonary congestion that characterizes CHF decompensation, yet its prognostic utility for readmission prediction remains largely unexplored. We present a pilot feasibility study, the first systematic machine learning study using B-mode LUS acquired during hospitalization to predict 30-day CHF readmission. Quantitative spatiotemporal embeddings are extracted from a pretrained Temporal Shift Module (TSM) ResNet-18 encoder, and interpretable biomarker features are separately evaluated. Through structured ablations over lung view, temporal representation, multi-view fusion, and cross-lung augmentation, we identify the key imaging factors driving readmission risk. Our findings reveal that (1) dependent lower-lung regions (Left-3, Right-3) carry the strongest prognostic signal, consistent with their greater susceptibility to hydrostatic congestion; (2) temporal difference features between sequential examinations substantially outperform single-timepoint representations, highlighting the importance of capturing disease trajectory; and (3) multi-view feature concatenation yields the best overall performance, with our top MLP model achieving an F1 score of 0.80 (95% CI: 0.62-0.96). Biomarker analysis further reveals that pleural-line abnormalities, including breaks and indentations, are as informative as the canonical A-line and B-line markers. These results support POCUS-derived biomarkers as practical, interpretable tools for noninvasive CHF risk stratification.
Jana Armouti, Laura Hutchins, Jacob Duplantis +12
Carnegie Mellon University, Pittsburgh, USA · LSUHSC Internal Medicine, New Orleans, USA · Cosmetic Surgery Facility LLC, Baltimore, MD 21093, USA