cs.SDJun 11, 2026

Towards Personalized Federated Learning for Dysarthric Speech Recognition

Authors: Tao ZhongMengzhe GengJiajun DengShujie HuXunying Liu

Organizations: The Chinese University of Hong Kong, China · National Research Council Canada, Canada

Abstract

Speech recognition is challenging for dysarthric speakers. While federated learning (FL)-based ASR can be an effective tool for protecting privacy, it suffers from heterogeneity issues caused by speaker variability. Forcing all speakers to share the same model components can be suboptimal under such heterogeneity, making personalization a promising direction; however, related research on dysarthric speech remains limited. To this end, this paper explores two aggregation strategies to achieve personalization, including the parameter-based averaging strategy and the embedding-based averaging strategy. Experiments on UASpeech and TORGO show that the proposed methods outperform the baseline regularized FedAvg by statistically significant WER reductions of up to 0.99% absolute (3.15% relative) on UASpeech and 0.56% absolute (4.73% relative) on TORGO, respectively.

Explore similar work

CardsList