Organizations: Nanjing University of Posts and Telecommunications, Nanjing, China · University of Science and Technology of China, Hefei, China · Shouyi Technology, Hefei, China · University of Cambridge, Cambridge, United Kingdom
Speech-based screening is a promising, non-invasive approach for detecting Alzheimer's disease and related cognitive risks. However, models trained on a single domain often generalize poorly to unseen languages, tasks, or recording protocols. This paper investigates this deployment gap using a leave-one-corpus-out evaluation across four distinct datasets. Among 70 interpretable speech and language features, 59 exhibit direction conflicts between healthy control and cognitive risk groups across corpora, with pause, silence, and speech rate showing high protocol sensitivity. Furthermore, while the XLM-R text baseline achieves strong average performance, its Area Under the ROC Curve (AUC) drops to 0.520 on the weakest held-out domain. A standard GroupDRO baseline reaches a 0.766 mean speaker AUC and a 0.504 worst-domain AUC under the same protocol. To address this, we propose a fusion method that integrates XLM-R text baseline scores with evidence anchors selected during training. Balanced fusion achieves a 0.785 mean speaker AUC, while anchor-heavy fusion raises the worst-case speaker AUC to 0.615. This work highlights the need to audit feature transferability and report worst-case domain robustness in cognitive speech screening.
Figures & tables
Corpus
Language
Task
Samples
Speakers
HC
Risk
NCMMSC
Mandarin
Mix
246
116
90
156
Pitt Cookie
English
Picture
547
291
241
306
TAUKADIAL Mandarin
Mandarin
Connected
507
169
222
285
Mandarin Chou
Mandarin
Picture
261
87
120
141
Total
1561
663
673
888
Table 1: Corpora used for leave-one-corpus-out evaluation. HC and risk are sample-level counts.
Figure 1: Direction conflict prevalence across marker scopes.
Figure 2: Grouped radar view of standardized healthy control versus risk contrasts for all 70 markers.
Figure 3: Single-indicator external transfer of the 24 core indicators.
Representation
NCMMSC
Pitt
TAUKADIAL
Chou
Mean
Worst
Range
XLM-R text baseline
0.520
0.739
0.850
0.968
0.769
0.520
0.447
wav2vec 2.0 L12
0.462
0.502
0.835
0.955
0.688
0.462
0.493
wav2vec 2.0 L24
0.454
0.574
0.800
0.969
0.699
0.454
0.515
Table 2: Deep-only baselines under leave one corpus out evaluation.
P-value top four
Random top four
Evidence-ranked top four
All 24 core
All 70 indicators
Source-AUC anchors
k
4
4
4
24
70
4
Mean AUC
0.506
0.542
0.555
0.521
0.538
0.629
Worst AUC
0.363
0.195
0.520
0.418
0.395
0.588
Table 3: Traditional-indicator selection under leave one corpus out evaluation. All values are AUC.
Figure 4: Source-corpus anchor choices for each held-out corpus.
Figure 5: Fusion weight sensitivity across score normalizations.
Figure 6: Model comparison across held-out corpora, including the source-corpus GroupDRO domain-generalization baseline.
Control
Mean speaker-level AUC
95% CI
Label shuffle
0.490
0.465 to 0.514
Metadata only
0.469
0.416 to 0.500
Corpus only
0.500
0.500 to 0.500
Length only
0.580
0.458 to 0.694
Corpus identity probe
0.899
0.864 to 0.934
Table 4: Negative controls and corpus separability checks under leave one corpus out evaluation. Values are mean AUC with 95 percent intervals.
Center for Language and Speech Processing (CLSP), Johns Hopkins University · Research Center for Information Technology Innovation, Academia Sinica · University of Michigan, Ann Arbor +1