DARS: Dysarthria-Aware Rhythm-Style Synthesis for ASR Enhancement

Minghui Wu; Xueling Liu; Jiahuan Fan; Haitao Tang; Yanyong Zhang; Yue Zhang

cs.SDMar 2, 2026

DARS: Dysarthria-Aware Rhythm-Style Synthesis for ASR Enhancement

Authors: Minghui Wu, Xueling Liu, Jiahuan Fan, Haitao Tang, Yanyong Zhang, Yue Zhang

Organizations: University of Science and Technology of China, China · iFlytek Co., Ltd., China · Huawei Technology, China

Abstract

Dysarthric speech exhibits abnormal prosody and significant speaker variability, presenting persistent challenges for automatic speech recognition (ASR). While text-to-speech (TTS)-based data augmentation has shown potential, existing methods often fail to accurately model the pathological rhythm and acoustic style of dysarthric speech. To address this, we propose DARS, a dysarthria-aware rhythm-style synthesis framework based on the Matcha-TTS architecture. DARS incorporates a multi-stage rhythm predictor optimized by contrastive preferences between normal and dysarthric speech, along with a dysarthric-style conditional flow matching mechanism, jointly enhancing temporal rhythm reconstruction and pathological acoustic style simulation. Experiments on the TORGO dataset demonstrate that DARS achieves a Mean Cepstral Distortion (MCD) of 4.29, closely approximating real dysarthric speech. Adapting a Whisper-based ASR system with synthetic dysarthric speech from DARS achieves a 54.22% relative reduction in word error rate (WER) compared to state-of-the-art methods, demonstrating the framework's effectiveness in enhancing recognition performance.

Figures & tables

Figure 1: DARS framework based on Matcha-TTS model

	Severe				Mod.-Sev.	Moderate	Mild
Participant	F01	M01	M02	M04	M05	F03	F04	M03
Number of Utterances	228	739	772	659	610	1097	675	806

Table 1: Dysarthria severity for the TORGO database.

ID	Whisper-Large	WER (%)
ID	Whisper-Large	Severe	Mod.-Sev.	Moderate	Mild	Overall
E1	10%	125.67	182.53	32.78	13.91	83.21
E2	20%	123.36	184.14	34.69	12.71	83.19
E3	30%	121.91	186.25	35.47	11.98	83.24

Table 2: WER Comparison on Torgo Evaluation Sets by Sentence Count.

ID	Models	$\alpha$	$\beta$	ASp	DSpG	SSp
E4	Grad-TTS [ 12 ]	–	–	6.61	6.71	6.81
E5	MATCHA-TTS (baseline)	–	–	6.25	6.43	6.64
E6	DARS (E5 w/ rhythm)	–	–	6.09	6.32	6.57
E7	DARS w/ CPO	0.5	0.5	5.83	6.07	6.21
E8	DARS w/ CPO	0.7	0.3	5.72	5.93	6.08
E9	DARS w/ CPO + style	0.7	0.3	4.29	4.46	4.61

Table 3: MCD for TTS Models Trained on TORGO Data.

ID	Data Type	training strategy		WER (%)
ID	Data Type	FT	LORA	Severe	Mod.-Sev.	Moderate	Mild	Overall
E2	-			123.36	184.14	34.69	12.71	83.19
E10	RealData	$\checkmark$		18.63	11.88	2.93	2.43	8.85
E11	RealData		$\checkmark$	22.35	13.66	3.24	2.54	10.09
E12	ASp	$\checkmark$		18.64	11.88	2.94	2.44	8.87
E13	ASp		$\checkmark$	22.37	13.67	3.24	2.56	10.12

Table 4: WER Performance Comparison Across Data Types and Training Strategies Using the Whisper-Large Model.

ID	DataSet	WER (%)
ID	DataSet	Severe	Mod.-Sev.	Moderate	Mild	Overall
E18 [ 26 ]	TORGO	55.88	49.6	36.8	12.6	39.2
E19 [ 12 ]	TORGO	23.3	13.98	3.27	2.57	16.93
E12	TORGO	18.64	11.88	2.94	2.44	8.87
E20	TORGO+ LibriSpeech Text	13.98	9.79	2.62	2.31	7.75

Table 5: WER Comparison with SOTA Systems.

Synthetic Data Augmentation for Speech Recognition

Synthetic data augmentation for automatic speech recognition uses generated speech, text, or supervision to improve recognition when real data is scarce or variable.

See all 30 papers

Improving End-to-End Speech Recognition for Dysarthric Speech through In-Domain Data Augmentation
Jun 18, 2026Paban Sapkota, Hemant Kumar Kathania, Sudarsana Reddy Kadiri +1Data Augmentation Automatic Speech Recognition
Department of Electronics and Communication Engineering, National Institute of Technology Sikkim, India · Signal Analysis and Interpretation Laboratory (SAIL), University of Southern California, Los Angeles, USA
End-to-End Simultaneous Dysarthric Speech Reconstruction with Frame-Level Adaptor and Multiple Wait-k Knowledge Distillation
Mar 2, 2026Minghui Wu, Haitao Tang, Jiahuan Fan +2Clinical Speech Processing TTS Synthesis
University of Science and Technology of China, China · iFlytek Co., Ltd., China
Cross-Dataset, Age, and Gender Generalization: A Comprehensive Analysis of Fine-Tuning Strategies for Low-Resource Children's ASR
Jun 18, 2026Abhijit Sinha, Hemant Kumar Kathania, Sudarsana Reddy Kadiri +1Speech Processing Automatic Speech Recognition
Department of Electronics and Communication Engineering, National Institute of Technology Sikkim, India · Signal Analysis and Interpretation Laboratory, University of Southern California, Los Angeles, USA

Jun 18, 2026J/K move · Enter open · S save

Improving End-to-End Speech Recognition for Dysarthric Speech through In-Domain Data Augmentation

Paban Sapkota, Hemant Kumar Kathania, Sudarsana Reddy Kadiri, Shrikanth Narayanan

Department of Electronics and Communication Engineering, National Institute of Technology Sikkim, India · Signal Analysis and Interpretation Laboratory (SAIL), University of Southern California, Los Angeles, USA

Data Augmentation Automatic Speech Recognition Synthetic Data Augmentation for Speech Recognition Low-Resource Speech Recognition Dysarthric Speech Recognition

Open paper AskPDF arXiv

Dysarthric speech recognition is crucial for facilitating effective communication among individuals with dysarthria. However, accurately recognizing dysarthric speech poses significant challenges due to varying severity levels and limited data availability. In this paper, we explore data augmentation techniques for dysarthric automatic speech recognition (ASR) systems by fine-tuning the End-to-End pre-trained Wav2Vec2 model, with a specific focus on severity levels. To address the challenges of data scarcity and the need for extensive data in fine-tuning pre-trained ASR systems for dysarthric speech, we investigate four prominent data augmentation methods: Speaking-Rate Modification (SRM), Pitch Modification (PM), Formant Modification (FM), and vocal tract Length Perturbation (VTLP), tailored to different aspects of dysarthria. The study uses individually fine-tuned Wav2Vec2 models for each severity class as baseline systems. Additionally, we conducted severity-specific fine-tuning of the ASR model using augmented data. Results demonstrate distinct efficacy patterns for each augmentation technique across severity levels. The best WERs were achieved with SRM (

s

=0.8) for low (9.02%) and medium (38.11%) severities, and with PM (

τ

=0.8) for high severity (55.15%), reflecting relative improvements of 30.02%, 16.64%, and 15.47%, respectively. These results confirm the effectiveness of the augmentation methods in improving dysarthric ASR performance.

Abstract

Synthetic Data Augmentation for Speech Recognition

Explore similar work

Improving End-to-End Speech Recognition for Dysarthric Speech through In-Domain Data Augmentation

End-to-End Simultaneous Dysarthric Speech Reconstruction with Frame-Level Adaptor and Multiple Wait-k Knowledge Distillation

Cross-Dataset, Age, and Gender Generalization: A Comprehensive Analysis of Fine-Tuning Strategies for Low-Resource Children's ASR