Impact of automatic speech recognition quality on Alzheimer's disease detection from spontaneous speech: a reproducible benchmark study with lexical modeling and statistical validation
Organizations: Independent Researcher, Austin, Texas, USA
Abstract
Early detection of Alzheimer's disease from spontaneous speech has emerged as a promising non-invasive screening approach. However, the influence of automatic speech recognition (ASR) quality on downstream clinical language modeling remains insufficiently understood. In this study, we investigate Alzheimer's disease detection using lexical features derived from Whisper ASR transcripts on the ADReSSo 2021 diagnosis dataset. We evaluate interpretable machine-learning models, including Logistic Regression and Linear Support Vector Machines, using TF-IDF text representations under repeated 5x5 stratified cross-validation. Our results demonstrate that transcript quality has a statistically significant impact on classification performance. Models trained on Whisper-small transcripts consistently outperform those using Whisper-base transcripts, achieving balanced accuracy above 0.7850 with Linear SVM. Paired statistical testing confirms that the observed improvements are significant. Importantly, classifier complexity contributes less to performance variation than ASR transcription quality. Feature analysis reveals that cognitively normal speakers produce more semantically precise object- and scene-descriptive language, whereas Alzheimer's speech is characterized by vagueness, discourse markers, and increased hesitation patterns. These findings suggest that high-quality ASR can enable simple, interpretable lexical models to achieve competitive Alzheimer's detection performance without explicit acoustic modeling. The study provides a reproducible benchmark pipeline and highlights ASR selection as a critical modeling decision in clinical speech-based artificial intelligence systems.
Figures & tables
| Split | AD | CN |
|---|---|---|
| Training | 87 | 79 |
| Official test-dist | 35 | 36 |
| Classifier | ASR model | Mean balanced accuracy | Standard deviation |
|---|---|---|---|
| Logistic Regression | Whisper base | 0.7491 | 0.0814 |
| Logistic Regression | Whisper small | 0.7757 | 0.0703 |
| Linear SVM | Whisper base | 0.7354 | 0.0865 |
| Linear SVM | Whisper small | 0.7850 | 0.0745 |
| Comparison | Mean difference | statistic | Two-sided | Interpretation |
|---|---|---|---|---|
| Whisper small vs base (LR) | +0.0266 | 2.4490 | 0.0220 | significant |
| Whisper small vs base (SVM) | +0.0497 | 3.8832 | 0.0007 | highly significant |
| Linear SVM vs LR (Whisper small) | +0.0093 | 1.0551 | 0.3019 | not significant |
| AD-indicative term | Coefficient | CN-indicative term | Coefficient |
|---|---|---|---|
| going | +0.1718 | open | -0.1295 |
| okay | +0.1540 | window | -0.1275 |
| happening | +0.1444 | reaching | -0.1196 |
| tell | +0.1212 | overflowing | -0.0998 |
| picture | +0.1132 | sink | -0.0978 |
| going picture | +0.1114 | cookie | -0.0912 |
Speech-Based Dementia Detection
Speech-based dementia detection uses spoken language and vocal patterns to identify cognitive impairment and support accessible screening.
See all 44 papers