Cross-Lingual Parkinson's Disease Severity Assessment Using Pre-trained Speech Embeddings: A Multi-Class Evaluation

Simon Pals; Cristian Tejedor-Garcia

eess.ASSep 16, 2026

Cross-Lingual Parkinson's Disease Severity Assessment Using Pre-trained Speech Embeddings: A Multi-Class Evaluation

Authors: Simon Pals, Cristian Tejedor-Garcia

Organizations: Centre for Language Studies Radboud University Nijmegen, the Netherlands

Abstract

Parkinson's disease (PD) often manifests through speech impairments, facilitating accessible, non-invasive, and cost-effective severity assessment for early diagnosis and progression tracking. Despite advances in speech foundation models (SFMs), their cross-lingual generalization for PD severity multi-class classification remains underexplored due to limited labeled data, a lack of explainable methods and variability across languages and datasets. In this work, we evaluate pre-trained embeddings from four state-of-the-art open-source SFMs across three datasets in zero-shot and k-shot cross-lingual settings for multi-class PD severity assessment. Our results show that pre-trained speech embeddings enable meaningful cross-lingual transfer, although performance is sensitive to dataset properties, preprocessing, and adaptation strategy. Misclassifications under these conditions related to inter-speaker variability and atypical speech patterns highlight the need for more robust feature extraction and modeling for PD severity assessment while emphasizing the importance of explainability for reliable clinical insights.

Figures & tables

Fig. 1: Flowchart of the preprocessing steps for each speech file of the source datasets.

Fig. 2: Distribution of original and segmented speech files.

Fig. 3: Overview of the experimental steps.

Dataset	Wav2vec 2.0	WavLM	Whisper
PC-GITA	9	12	32
ItalianPVS	7	8	12
MDVR-KCL	3	6	7

TABLE I: Selected encoder layers for each model and dataset

Target	Source	Pipeline	Condition	Language	Accuracy (%)	Precision (%)	Recall (%)	F1 (%)	ROC-AUC (%)
		WavLM + XGB	Zero-shot	Target	58.14 $\pm$ 8.50	58.18 $\pm$ 7.54	57.73 $\pm$ 7.64	53.69 $\pm$ 6.38	75.03 $\pm$ 6.63
		WavLM + SVM	Adapted (k-shot)	Target	81.73 $\pm$ 6.36	75.02 $\pm$ 7.99	72.38 $\pm$ 2.52	71.02 $\pm$ 4.74	86.25 $\pm$ 8.74
		WavLM + SVM	Zero-shot	Source	79.19 $\pm$ 6.09	64.64 $\pm$ 15.81	60.62 $\pm$ 8.81	60.93 $\pm$ 11.95	88.30 $\pm$ 2.07
	MDVR-KCL	WavLM + SVM	Adapted (k-shot)	Source	78.07 $\pm$ 7.42	66.44 $\pm$ 13.57	60.05 $\pm$ 7.35	61.01 $\pm$ 10.09	89.00 $\pm$ 2.59
		WavLM + ERT	Zero-shot	Target	43.85 $\pm$ 2.88	42.26 $\pm$ 4.66	43.86 $\pm$ 8.08	37.75 $\pm$ 4.65	63.35 $\pm$ 7.05
		WavLM + ERT	Adapted (k-shot)	Target	77.38 $\pm$ 7.10	70.76 $\pm$ 4.81	72.96 $\pm$ 3.99	67.66 $\pm$ 6.75	87.17 $\pm$ 4.05

TABLE II: Classification performance for each source

\rightarrow

target dataset pair with the highest F1-score of the target language

Target	Source	F1 ${}^{Target}_{0-shot}$	F1 ${}^{Target}_{k-shot}$	$\Delta$ F1 Target	$\Delta$ F1 Source
ItalianPVS	MDVR-KCL	53.69	71.02	+17.33	+0.08
ItalianPVS	PC-GITA	37.75	67.66	+29.91	+1.06
MDVR-KCL	ItalianPVS	47.08	63.98	+16.90	+0.78
MDVR-KCL	PC-GITA	52.71	66.91	+14.20	+0.90
PC-GITA	ItalianPVS	35.21	56.46	+21.25	+0.17
PC-GITA	MDVR-KCL	42.39	52.88	+10.49	-1.68

TABLE III: Target- and source-language F1-score changes for each source

\rightarrow

target dataset pair

Fig. 4: Confusion matrix of the ItalianPVS dataset.

Fig. 5: Confusion matrix of the MDVR-KCL dataset.

Fig. 6: Confusion matrix for the PC-GITA dataset.

Fig. 7: Speaker hardness grid of the ItalianPVS dataset.

Explore similar work

Exploiting Speech LLM Representations for Multilingual and Cross-Lingual Parkinson's Disease Detection
Sep 13, 2026Sarthak Giri, Zi Haur Pang, Tatsuya KawaharaParkinson's Disease Clinical Speech Processing
Graduate School of Informatics, Kyoto University, Japan
Motor, Cognitive, or Corpus? What Survives Cross-Lingual Transfer in Speech-Based Parkinsons Disease Detection
Aug 13, 2026Serli Kopar, Sam Gijsen, Abner Hernandez +2Speech Processing Parkinson's Disease
Hertie Institute for AI in Brain Health, University of Tübingen, Tübingen, Germany · Tübingen AI Center, University of Tübingen, Tübingen, Germany · Charité–Universitätsmedizin, Department of Psychiatry and Psychotherapy, Berlin, Germany +1
Adapting Self-Supervised Speech Representations for Cross-lingual Dysarthria Detection in Parkinson's Disease
Mar 23, 2026Abner Hernandez, Eunjung Yeo, Kwanghee Choi +12Domain Adaptation Medical Diagnosis
1FAU Erlangen-Nürnberg, Germany · 2UT Austin, USA · 3CMU, USA +5

Sep 13, 2026J/K move · Enter open · S save

Exploiting Speech LLM Representations for Multilingual and Cross-Lingual Parkinson's Disease Detection

Sarthak Giri, Zi Haur Pang, Tatsuya Kawahara

Graduate School of Informatics, Kyoto University, Japan

Parkinson's Disease Clinical Speech Processing Speech Language Models Speech Processing

Open paper AskPDF arXiv

Speech Large Language Models (Speech LLMs) have shown strong performance across diverse tasks, yet their utility for pathological speech analysis remains underexplored. In this work, we investigate the effectiveness of internal representations from encoder and decoder components of Speech LLMs for Parkinson's Disease (PD) detection across multilingual and cross-lingual settings. Our findings reveal that encoder representations consistently outperform their decoder counterparts in most models and settings and that pathological cues may be progressively attenuated as audio representations are projected into the language model space. We further show that generative outputs are less reliable for clinical tasks compared to internal representations. To leverage information spread across multiple layers, we propose a Squeeze-and-Excitation (SE)-based dynamic layer aggregation framework, which surpasses best-layer selection in multiple experiments, suggesting that PD-relevant acoustic cues are distributed across transformer layers rather than concentrated in one.

Abstract

Explore similar work

Exploiting Speech LLM Representations for Multilingual and Cross-Lingual Parkinson's Disease Detection

Motor, Cognitive, or Corpus? What Survives Cross-Lingual Transfer in Speech-Based Parkinsons Disease Detection

Adapting Self-Supervised Speech Representations for Cross-lingual Dysarthria Detection in Parkinson's Disease