Pseudo-Labels as Rewards, Not Targets: Group-Relative Policy Optimization for Semi-Supervised ASR
Authors: Shashi Kumar, Yanis Labrak, Sergio Burdisso, Esaú Villatoro-Tello, Petr Motlicek, Kadri Hacioglu, Andreas Stolcke
Organizations: Idiap Research Institute, Martigny, Switzerland · EPFL, Lausanne, Switzerland · Brno University of Technology, Czech Republic · Uniphore, U.S.A.
Adapting ASR to a domain with only untranscribed audio typically relies on pseudo-labeling, a strong teacher transcribes the audio and a student is fine-tuned on it. Because supervised fine-tuning (SFT) treats every pseudo-label token as a target, the student inherits the teacher's errors. We show that using the same pseudo-labels as a reward reference for Group Relative Policy Optimization (GRPO) is more robust to label noise. In the banking domain, GRPO yields 29.3% to 56.2% relative WER improvements over SFT on Whisper-generated labels, and the advantage holds across four Whisper sizes, four teacher families, three domains and four languages. While SFT reproduces almost 100% of teacher errors verbatim, GRPO passes through only 27%. Rollout statistics explain why, erroneous pseudo-label words rarely appear in on-policy rollouts, so group-relative advantages stay aligned with the clean transcript, whereas SFT forces them into the policy's support and collapses the reward variance GRPO relies on.
Figures & tables
PL Source
Method
WER
CER
INS/DEL/SUB
LS960 (student)
SFT
66.44
46.73
42.18/5.68/18.59
Tiny
SFT
32.11
23.71
15.17/4.91/12.03
GRPO
14.06
10.68
1.87/5.69/6.49
SFT → GRPO
22.77
16.71
6.20/5.77/10.80
Small
SFT
20.23
15.64
7.95/4.00/8.28
GRPO
12.71
9.64
1.95/4.84/5.91
Table 1: Performance on the banking test set of DefinedAI. Students are trained on pseudo-labels from four Whisper teachers, from weakest to strongest, or on gold transcripts as a topline.
Teacher
SFT
GRPO
SFT → GRPO
Qwen3 ASR 1.7B
13.83
11.35
13.41
Phi-4 Multimodal IT
14.34
11.81
14.76
Audio Flamingo 3
14.96
11.83
14.32
Table 2: Performance (%WER) on the banking test set of DefinedAI with pseudo-labels from three additional teacher families.
Gold labels
Pseudo-labels
Domain
Method
WER
CER
WER
CER
Healthcare
SFT
12.28
8.40
18.72
14.32
GRPO
12.60
8.39
14.64
10.91
Insurance
SFT
14.02
10.27
19.59
15.53
GRPO
14.35
10.24
15.70
12.12
Table 3: Performance on the DefinedAI healthcare and insurance test sets. Pseudo-labels come from Whisper Large-V3, and gold-label results serve as a topline.
Gold labels
Pseudo-labels
Language
SFT
SFT → GRPO
SFT
SFT → GRPO
Hindi
27.01
28.26
24.27
22.94
Hebrew
46.40
37.33
40.03
36.89
Vietnamese
57.64
51.02
58.78
49.92
Serbian
34.31
30.68
35.01
30.17
Table 4: CER (%) on the Common Voice 26 test sets for four low-resource languages, each with under 6 h of training audio. Pseudo-labels come from Whisper Large-V3. CER is used instead of WER because these scripts are non-Latin or heavily diacritized. Direct GRPO is omitted because the English-trained student model cannot produce the target scripts (Section 4 ).
Figure 1: Fraction of erroneous pseudo-label (PL) words that the student reproduces verbatim in its final output, as a function of the number of word errors in the utterance’s pseudo-label.
Figure 2: Rollout statistics as a function of the number of PL word errors per utterance. Left: fraction of the G=4 rollouts per utterance that contain the erroneous PL word. Right: within-group standard deviation of the reward across the G=4 rollouts.