Automatic Speech recognition (ASR) struggles with Pathological speech due to persistent speaker-specific mismatch. In clinical and accessibility scenarios, only limited target-speaker data is typically available, making full model retraining impractical. We present a component-level analysis of LoRA adaptation in Whisper, investigating how adapter placement interacts with the model size, training efficiency and utterance-duration in single-speaker settings. Our experimental comparative study shows that while scaling reduces errors, it does not eliminate mismatch. Furthermore, the full attention LoRa provides the most reliable accuracy, and the best time--accuracy trade-off. Improvements vary across utterance length, with medium-length recordings benefiting the most, while short and long utterances remain challenging. These findings highlight the importance of adaptation placement and duration-dependent evaluation, under challenging, data-scarce personalization scenarios.
Figures & tables
Tier
Threshold
N
Avg. duration
Short
<3.5 s
21
2.50 s
Medium
3.5–6.0 s
139
4.81 s
Long
>6.0 s
55
10.81 s
Table 1: Duration-tier distribution of the dataset.
Metric
Target median
CV median
Reference
CPPS (dB)
4.69
7.94
>6.49 dB [ 5 ]
HNR (dB)
1.32
12.20
6.58 – 12.22 dB [ 25 ]
Jitter local (%)
4.48
2.12
0.72 – 2.12% [ 25 ]
Shimmer local (%)
18.04
9.85
8.33 – 16.81% [ 25 ]
Table 2: Acoustic characterisation of the target speaker and comparison with a female Common Voice (CV) reference subset processed with the same extraction pipeline.
Model
Params
Enc.
Dec.
Dim
Heads
CNN params
Whisper-tiny
39M
4
4
384
6
535,296
Whisper-base
74M
6
6
512
8
910,336
Whisper-small
244M
12
12
768
12
1,955,328
Whisper-medium
769M
24
24
1024
16
3,393,536
Table 3: Whisper variants used in this work. CNN params denotes the parameters in the two Conv1d layers of the encoder front-end.
Model
WER
CER
chrF
Cosine
tiny
68.8
24.9
61.2
0.600
base
46.2
15.7
73.5
0.738
small
23.2
8.6
87.0
0.885
medium
12.2
5.0
92.0
0.941
Table 4: Vanilla Whisper baselines (215-fold LOO average). WER and CER are percentages.
Model
Strategy
Error rate
Change (pp)
Change (%)
WER
CER
Δ WER
Δ CER
Δ WER
Δ CER
tiny
Vanilla
68.8
24.9
–
–
–
–
LoRA decoder
68.1
25.0
−0.7
+0.1
−1.0%
+0.4%
LoRA encoder
60.3
20.1
−8.5
−4.8
−12.4%
−19.3%
LoRA full
53.8
17.2
−15.0
−7.7
−21.8%
−30.9%
base
Vanilla
46.2
15.7
–
–
–
–
Table 5: Overall fixed-rank LoRA comparison. WER and CER are percentages; Δ (pp) values are absolute changes from the vanilla baseline; Δ (%) values are the corresponding relative changes.
Model
Strategy
chrF
Cosine
tiny
LoRA decoder
61.1
0.611
LoRA encoder
66.8
0.654
LoRA full
71.2
0.701
base
LoRA decoder
73.8
0.747
LoRA encoder
77.4
0.773
LoRA full
81.3
0.814
Table 6: Post-adaptation chrF and cosine for the three LoRA placements.
Figure 1: Training time versus WER improvement for fixed-rank LoRA configurations. Each point corresponds to one model size and one LoRA placement.
Model
Tier
WER
CER
Before
After
Δ pp
Δ %
Before
After
Δ pp
Δ %
tiny
Short
80.1
71.8
−8.3
−10.4%
46.0
39.4
−6.6
−14.3%
Medium
71.6
51.0
−20.6
−28.8%
24.4
14.5
−9.9
−40.6%
Long
57.2
54.1
−3.1
−5.4%
18.2
15.4
−2.8
−15.4%
base
Short
74.8
52.2
−22.6
−30.2%
41.1
33.1
−8.0
−19.5%
Medium
43.8
34.8
−9.0
−20.5%
13.6
8.6
−5.0
−36.8%
Table 7: LoRA full performance and deltas by duration tier. WER and CER are percentages; Δ pp values are after minus before in percentage points; Δ % values are the corresponding relative changes. Negative deltas indicate error reduction; bold deltas mark the largest reduction within each tier metric.