Organizations: University of North Carolina at Chapel Hill, Chapel Hill, NC, USA · University of Auckland, Auckland, NZ · Duke University, Durham, NC, USA · Columbia University, New York, NY, USA
Automatic speech recognition (ASR) has been extensively studied on neutral and stationary speech, yet its robustness under post-exercise physiological shift remains underexplored. Compared with resting speech, post-exercise speech often contains micro-breaths, non-semantic pauses, unstable phonation, and repetitions caused by reduced breath support, making transcription more difficult. In this work, we benchmark acoustic foundation models on post-exercise speech under a unified evaluation protocol. We compare sequence-to-sequence models (Whisper and FunASR/Paraformer) and self-supervised encoders with CTC decoding (Wav2Vec2, HuBERT, and WavLM), under both off-the-shelf inference and post-exercise in-domain fine-tuning. Across the Static/Post-All benchmark, most models degrade on post-exercise speech, while FunASR shows the strongest baseline robustness at 14.57% WER and 8.21% CER on Post-All. Fine-tuning substantially improves several CTC-based models, whereas Whisper shows unstable adaptation. As an exploratory case study, we further stratify results by fluent and non-fluent speakers; although the non-fluent subset is small, it is consistently more challenging than the fluent subset. Overall, our findings show that post-exercise ASR robustness is strongly model-dependent, that in-domain adaptation can be highly effective but not uniformly stable, and that future post-exercise ASR studies should explicitly separate fluency-related effects from exercise-induced speech variation.
Figures & tables
Fig. 1 : Framework for evaluating and enhancing ASR robustness on post-exercise speech.
Fig. 2 : Dataset distribution.
Size
Model
# Enc. layers
# Dec. layers
SSL encoder + CTC head
95M
Wav2Vec2-Base
12
–
95M
HuBERT-Base
12
–
95M
WavLM-Base+
12
–
317M
Wav2Vec2-Large
24
–
317M
HuBERT-Large
24
–
TABLE I : ASR models used in this study. Encoder and decoder layers are reported for seq2seq models, while SSL-based CTC models include only encoder layers.
Size
Model
Static Baseline
Post-All Baseline
Post-All Fine-tune (5-fold)
WER(%) ↓
CER(%) ↓
WER(%) ↓
CER(%) ↓
WER(%) (mean ± std) ↓
CER(%) (mean ± std) ↓
74M
Whisper-Base (S2S)
26.86
21.48
27.48
21.87
58.11±8.82
47.62±5.72
95M
Wav2Vec2-Base (CTC)
21.53
10.11
32.30
15.70
22.93±1.61
9.31±0.84
95M
HuBERT-Base (CTC)
31.74
13.68
45.98
21.04
24.75±0.38
9.33±0.27
95M
WavLM-Base+ (CTC)
35.32
14.78
45.45
20.46
22.94±0.43
8.67±0.29
220M
FunASR/Paraformer (L preset)
12.29
6.70
14.57
8.21
13.71±0.96
8.04±0.55
TABLE II : Results for all 9 evaluated model settings.Fine-tune uses 5-fold CV on Post-All (mean ± std). Lower WER/CER is better. Best results in each block are bold ; second-best results are underlined .
Baseline ( Post-All )
Fine-tune ( Post-All , 5-fold)
Size
Model
Fluent
Non-Fluent
Fluent
Non-Fluent
WER(%) ↓
CER(%) ↓
WER(%) ↓
CER(%) ↓
WER(%) ↓
CER(%) ↓
WER(%) ↓
CER(%) ↓
74M
Whisper-Base
27.63
20.96
49.97
48.00
65.98
54.58
95.37
82.03
95M
Wav2Vec2-Base
32.83
15.72
49.41
24.42
17.33
8.71
27.99
14.38
95M
HuBERT-Base
47.26
22.12
70.52
35.86
9.96
5.15
16.69
8.64
95M
WavLM-Base+
44.88
19.22
73.33
35.49
19.17
8.44
30.13
12.56
TABLE III : Fluency split results on Post-All ( n=509 : Fluent=475, Non-Fluent=34). Baseline = inference only; Fine-tune = evaluation of the corresponding 5-fold-trained model on each subgroup split. Lower WER/CER is better. Best results in each block are bold ; second-best results are underlined .
Static Fluent
Static Non-Fluent
Size
Model
WER (%) ↓
CER (%) ↓
WER (%) ↓
CER (%) ↓
74M
Whisper-Base
23.83
19.02
33.89
27.17
95M
Wav2Vec2-Base
19.47
9.25
26.28
12.11
95M
HuBERT-Base
29.20
12.46
37.85
16.64
95M
WavLM-Base+
31.97
13.44
43.09
17.90
220M
FunASR/Paraformer (L preset)
11.12
6.13
15.00
8.01
TABLE IV : Baseline fluency split results on Static . This reference set contains 198 structured paragraph-reading recordings (Fluent=161,Non-Fluent=37).
Automatic speech recognition (ASR) systems often perform poorly in dysarthric speech, limiting their usefulness to affected speakers in everyday communication. This paper presents a personalized ASR system for a dysarthric speaker, built by adapting a foundation ASR model to speaker-specific data. Using the TEQST tool, we collected 92 hours of read speech and later added 8.8 hours of user corrections gathered through a deployed mobile application. Starting from Whisper, fine-tuning reduced word error rate to 15.8% with only 1.4 hours of adaptation data, reached 10.7% with 22.5 hours, and achieved the best result of 9.7% when using all available data including the corrections. Using LoRA adaptation and/or Qwen3-ASR as foundation model performed worse in this setting. The results show that personalized fine-tuning can make foundation ASR models substantially more effective for dysarthric speech and suitable for practical deployment.
Christian Huber, Laura Kernahan, Alexander Waibel
Interactive Systems Lab Karlsruhe Institute of Technology Karlsruhe, Germany · Interactive Systems Lab Carnegie Mellon University Pittsburgh PA, USA
Automatic speech recognition (ASR) performance degrades sharply on dysarthric speech, yet how disordered articulation reshapes a model's internal representations is underexplored. We present a layer-wise probing analysis of a transformer ASR encoder on Mandarin dysarthric speech under three transcript-matched conditions: original dysarthric speech, speaker conditioned zero-shot TTS resynthesis, and unconditioned TTS. The probes reveal a task-dependent hierarchy: phoneme boundary information stays weak for dysarthric speech at every layer, phoneme identity becomes recoverable toward the upper layers, and recognition difficulty is encoded in the deepest layers. Tone-sensitive evaluation shows Mandarin lexical tone is a persistent error source. Cross-condition similarity divergence grows with depth, indicating that disordered speech affects high-level representations more than low-level acoustic features. Guided by these findings, single-layer LoRA at layer 7 and adaptation on subset layers 5-8 achieve performance within 3.5% and 2.48% relative margins of full encoder adaptation, respectively, while upper-layer adaptation is less effective for dysarthric speech. These findings link representation analysis to parameter-efficient fine-tuning and motivate layer-aware adaptation for low-resource Mandarin dysarthric ASR.
Darwin Jelestin Muthu, Navya Gupta, Wei Lin Tay +3
Automatic speech recognition (ASR) systems remain brittle on dysarthric and other atypical speech. Recent audio-language models raise the possibility of improving performance by conditioning on additional clinical context at inference time, but it is unclear whether these models can make use of such information. We introduce a benchmark built on the Speech Accessibility Project (SAP) dataset that tests whether diagnosis labels, clinician-derived speech ratings, and progressively richer clinical descriptions improve transcription accuracy for dysarthric speech. Across matched comparisons on nine models, we find that current models do not meaningfully use this context: diagnosis-informed and clinically detailed prompts yield negligible improvements and often degrade word error rate. We complement the prompting analysis with context-dependent fine-tuning, showing that LoRA adaptation with a mixture of clinical prompt formats achieves a WER of 0.066, a 52% relative reduction over the frozen baseline, while preserving performance when context is unavailable. Subgroup analyses reveal significant gains for Down syndrome and mild-severity speakers. These results clarify where current models fall short and provide a testbed for measuring progress toward more inclusive ASR.
Pehuén Moure, Niclas Pokel, Bilal Bounajma +4
Institute of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland · Department of Mathematics, ETH Zurich, Switzerland · Department of Computational Linguistics, University of Zurich, Switzerland