Cross-Linguistic Acoustic Biomarkers of Cognitive Impairment Across Interactive and Single-Speaker Speech Tasks
Authors: Kenyu Ikeuchi, Tomoaki Ohtsuki, Michelle Cohn, Alyssa Weakley, Yui Ishihara, Joseph Hoang, Taishiro Kishimoto, Sarah Tomaszewski Farias, Jet M. J. Vonk, Chen-Nee Chuah
Organizations: Keio University, Yokohama, Japan · University of California - Davis, Davis, CA, USA · University of California - Davis Health, Sacramento, CA, USA · Keio University School of Medicine, Tokyo, Japan · University of California - San Francisco, San Francisco, CA, USA
Multilingual dementia detection is essential for scalable and equitable screening. In recent years, automated speech analysis has emerged as a promising approach for supporting dementia detection by enabling non-invasive and scalable assessment of cognitive function. However, existing studies that use automated speech analysis often rely on one type of task, such as picture description, and use black-box models. These studies often neglect the role of task and language. To address these gaps, we performed a cross-linguistic and cross-task acoustic analysis that integrates a conversational dataset with structured speech tasks across four languages (Spanish, Mandarin, English, Japanese). We applied statistical, linguist-/clinician-guided, and machine learning model-based feature selection to derive compact acoustic feature sets while maintaining classification performance. This enabled systematic comparison of cognitive impairment-relevant features across datasets and tasks. Our results revealed that acoustic features did not generalize uniformly across tasks and languages. Temporal features remained robust across all datasets, consistently capturing timing disruption regardless of language or task. In contrast, speech rate showed strong task dependence -- emerging as informative mainly in single-speaker tasks with externally anchored content and pacing, but becoming less reliable when speech content was highly individualized. Pitch-related features further exhibited language-specific patterns, favoring chroma-related features in tonal languages (e.g., Mandarin) and F0-related features in non-tonal languages. These findings caution against indiscriminate pooling of heterogeneous speech data and underscore the need for interpretable, context-aware approaches to multilingual dementia detection using automated speech analysis.
Figures & tables
Dataset (Lang, Task)
Normal
Impaired
ADReSSo (English, PD)
114 (HC)
116 (AD)
Ivanova (Spanish, Reading)
150 (HC)
127 (AD, MCI)
Chou (Mandarin, PD)
43 (HC)
44 (MCI)
PROMPT (Japanese, Conv)
74 (HC)
52 (CI)
ADRC (English, AM)
98 (HC)
110 (MCI, Dementia)
TABLE I: Dataset composition based on clinical diagnosis (after excluding MMSE < 10).
Fig. 1: Overview of the proposed method
Feature group
Single
Conversation
Temporal features (Utterance, Speech Rate)
20
27
pyAudioAnalysis
136
136
openSMILE eGeMAPS
88
88
Praat (Pitch, Intensity, Rhythm)
11
11
Total
255
262
TABLE II: Feature counts for single-speaker and conversational tasks without any feature filtering. Response pause features are not applicable to single-speaker tasks.
Dataset
(A) All
(B) KS-only
(C) KS + Expert
(D) KS + RFECV
(E) KS + Expert + RFECV
ADReSSo
255 / 0.74 (0.70–0.78)
186 / 0.76 (0.72–0.80)
171 / 0.76 (0.73–0.79)
156 / 0.75 (0.72–0.78)
142 / 0.76 (0.73–0.79)
Ivanova
255 / 0.77 (0.74–0.80)
199 / 0.77 (0.75–0.79)
178 / 0.77 (0.75–0.79)
121 / 0.77 (0.74–0.80)
114 / 0.77 (0.75–0.79)
Chou
255 / 0.66 (0.62–0.70)
253 / 0.66 (0.62–0.70)
227 / 0.67 (0.63–0.71)
131 / 0.66 (0.62–0.70)
106 / 0.67 (0.63–0.71)
PROMPT
262 / 0.79 (0.76–0.82)
251 / 0.79 (0.76–0.82)
227 / 0.79 (0.76–0.82)
156 / 0.79 (0.76–0.82)
101 / 0.79 (0.76–0.82)
ADRC
262 / 0.65 (0.63–0.67)
202 / 0.66 (0.64–0.68)
180 / 0.67 (0.65–0.69)
102 / 0.65 (0.63–0.67)
86 / 0.67 (0.65–0.69)
TABLE III: Effect of feature filtering and RFECV on feature dimensionality and classification performance. Each cell reports the number of retained features and the corresponding ROC-AUC (shown as #Features / AUC mean (95% confidence interval)).
Dataset
Ivanova
Chou
ADReSSo
ADRC
PROMPT
Language
Spanish
Mandarin
English
English
Japanese
Task
Reading
Picture Description
Picture Description
Autobiographical Memory
Conversation
The number of stable important features
53
23
127
12
29
Temporal features (Duration, Speech Rate)
High
High
Moderate
High
Moderate
Pitch (F0 variability and slope)
Moderate
Low
Moderate
High
Moderate
Pitch (chroma-based mean and std)
Moderate
High
Moderate
Low
High
TABLE IV: Cross-dataset comparison of dominant acoustic features. Top rows summarize dataset attributes, while bottom rows indicate the relative dominance ( High , Moderate , Low ) of each acoustic feature family within stable important features.
Feature
# Datasets
Family
timing_medial_pause_mean
4
Temporal
pause_ratio
3
Temporal
speech_rate
3
Temporal
timing_medial_pause_std
3
Temporal
timing_utterance_median
3
Temporal
utterance_ratio
3
Temporal
TABLE V: Most consistently selected task- and language-agnostic individual temporal features across datasets.