Group Relative Policy Optimisation Improves Multilingual Speech Recognition in Low-Resource Languages

Tim Wenjie Hu; Evelyn Nafula Ouma; Benjamin Akera; John A. Quinn

cs.AIOct 22, 2026

Group Relative Policy Optimisation Improves Multilingual Speech Recognition in Low-Resource Languages

Authors: Tim Wenjie Hu, Evelyn Nafula Ouma, Benjamin Akera, John A. Quinn

Organizations: Sunbird AI, Uganda

Abstract

Autoregressive encoder-decoder speech recognition models trained with teacher forcing can suffer from exposure bias, limiting their ability to recover from their own prediction errors at inference time. Recent work has shown that reinforcement learning with Group Relative Policy Optimisation (GRPO) improves speech recognition in major languages with LLM-based models. We extend this line of work to smaller, non-LLM encoder-decoder ASR models, using Whisper-large-v3 as a case study across 51 African languages. The benefits of this approach are most pronounced when training data is scarce or when input audio is noisy or degraded. Our resulting model, SunflowerASR, achieves the best published macro-averaged WER on the AfriVox-v2 (0.227) and SimbaBench (0.372) benchmarks, and is publicly available.

Figures & tables

Figure 1: CER before (supervised fine-tuning only) and after GRPO for Whisper-large-v3 on 51 African languages. The improvement becomes more pronounced at increasing levels of background noise.

Figure 2: WER by language on the Sunbird Speech Benchmark, comparing SunflowerASR against external baselines. Languages are sorted by the WER of SunflowerASR.

	Sunflower	Sahara	Gemini	Omni-CTC			Simba
Language	ASR (ours)	v2	3 Flash	7B	1B	300M	S
Swahili	0.108	0.071	0.076	0.077	0.097	0.152	0.150
Tswana	0.116	0.140	0.275	0.142	0.210	0.277	0.604
Kinyarwanda	0.133	0.066	0.165	0.104	0.138	0.215	0.591
Sotho	0.152	0.195	0.346	0.421	0.473	0.533	0.596
Luganda	0.160	0.393	0.315	0.422	0.428	0.483	0.341

Table 1: WER by language on AfriVox-v2 for the 14 languages with published baselines, which report WER only. Simba-S is our own evaluation; it does not cover three of the languages, and its macro is over the other eleven.

	SunflowerASR	OmniASR	Simba	Simba	Gemini
Language	(Ours)	LLM-7 B	S	M	2.5 Pro
Hausa	0.167	0.169	0.641	0.291	0.219
Luganda	0.220	0.171	0.227	0.357	0.421
Swahili	0.246	0.156	0.163	0.252	0.168
Kinyarwanda	0.251	0.192	0.531	0.367	0.463
Luo	0.297	0.271	0.391	0.442	0.494

Table 2: WER by language on SimbaBench over the 19 languages in common. Baselines from the public leaderboard.

Figure 3: Per-language CER improvement from GRPO against the number of hours of supervised training data for that language.

		WER			CER
Scenario	$n$	Before	After	$\Delta$	Before	After	$\Delta$
Ugandan English
Citizen feedback	12	0.324	0.282	$-13.0\%$	0.223	0.182	$-18.0\%$
Call centre	16	0.302	0.273	$-9.4\%$	0.213	0.182	$-14.8\%$
Luganda
Citizen feedback	8	0.705	0.683	$-3.0\%$	0.324	0.317	$-2.2\%$

Table 3: Effect of GRPO on the Sunbird salt-practical-eval set, by deployment scenario and language.

Variant	WER	$\Delta$	CER	$\Delta$
SFT checkpoint (initialisation)	0.4279	—	0.1783	—
$+$ 1 further SFT epoch	0.4794	$+12.0\%$	0.2089	$+17.2\%$
GRPO, random clips per subset	0.4220	$-1.4\%$	0.1714	$-3.9\%$
GRPO, 16 clips per batch	0.4243	$-0.8\%$	0.1689	$-5.3\%$
GRPO, full (SunflowerASR)	0.4201	$-1.8\%$	0.1680	$-5.8\%$

Table 4: Ablations on the noise-

1.3

set (

20,347

rows), with all variants starting from the same SFT checkpoint.

Measure	SFT	GRPO
Word substitutions	114,755	115,272
Character substitutions	165,096	160,220
Mean char. edit distance per substitution	3.437	3.385
Substitutions within $\leq 1$ edit	26.7%	27.4%
Substitutions within $\leq 2$ edits	44.9%	45.8%
Total char. cost of substitutions	394,368	390,231

Table 5: Substitution quality on the

20,124

rows where neither model loops.

Lang.	Reference	SFT heard	GRPO heard	Edits
Oromo	libiraal	liibiraaldimookiraasiitti	liibiraal	17 $\to$ 1
Rukonjo	ngerikulhakulhania	ebyo	erikulhakulhania	17 $\to$ 2
Shona	vobva	rwakachenerukira	wobva	15 $\to$ 1
Luganda	bibili	ebitakyalabirwaako	bibiri	14 $\to$ 1
Ruruuli	ebyankulakulana	mbe	ebyangulagulana	14 $\to$ 2
Kanuri	k e nasartiyende	d e	k e nasartiyande	13 $\to$ 1

Table 6: Cases where the supervised model heard a fragment or an unrelated word and GRPO heard a near-miss spelling.

Automatic Speech Recognition

Automatic speech recognition converts spoken language into text across speakers, languages, domains, and recording conditions.

See all 480 papers

Pseudo-Labels as Rewards, Not Targets: Group-Relative Policy Optimization for Semi-Supervised ASR
Oct 23, 2026Shashi Kumar, Yanis Labrak, Sergio Burdisso +4Automatic Speech Recognition Domain Adaptation for ASR
Idiap Research Institute, Martigny, Switzerland · EPFL, Lausanne, Switzerland · Brno University of Technology, Czech Republic +1
When Synthetic Speech Is All You Have: Better Call GRPO
Jul 9, 2026Shashi Kumar, Yanis Labrak, Hasindri Watawana +5Synthetic-to-Real Domain Adaptation Domain Adaptation for ASR
Idiap Research Institute, Martigny, Switzerland · EPFL, Lausanne, Switzerland · Uniphore, U.S.A. +1
MERaLiON-GR: Speech Gender Recognition Model for English and SEA Languages
Aug 5, 2026Qiongqiong Wang, Ai Ti Aw, Nancy F. Chen +18Speech Foundation Models Speech Processing
Institute of Advanced Intelligence and Computing (IAIC), A*STAR, Singapore

Oct 23, 2026J/K move · Enter open · S save

Pseudo-Labels as Rewards, Not Targets: Group-Relative Policy Optimization for Semi-Supervised ASR

Shashi Kumar, Yanis Labrak, Sergio Burdisso, Esaú Villatoro-Tello+3

Idiap Research Institute, Martigny, Switzerland · EPFL, Lausanne, Switzerland · Brno University of Technology, Czech Republic+1

Automatic Speech Recognition Domain Adaptation for ASR Group Relative Policy Optimization Noisy-Label Learning

Open paper AskPDF arXiv

Adapting ASR to a domain with only untranscribed audio typically relies on pseudo-labeling, a strong teacher transcribes the audio and a student is fine-tuned on it. Because supervised fine-tuning (SFT) treats every pseudo-label token as a target, the student inherits the teacher's errors. We show that using the same pseudo-labels as a reward reference for Group Relative Policy Optimization (GRPO) is more robust to label noise. In the banking domain, GRPO yields 29.3% to 56.2% relative WER improvements over SFT on Whisper-generated labels, and the advantage holds across four Whisper sizes, four teacher families, three domains and four languages. While SFT reproduces almost 100% of teacher errors verbatim, GRPO passes through only 27%. Rollout statistics explain why, erroneous pseudo-label words rarely appear in on-policy rollouts, so group-relative advantages stay aligned with the clean transcript, whereas SFT forces them into the policy's support and collapses the reward variance GRPO relies on.

Abstract

Automatic Speech Recognition

Explore similar work

Pseudo-Labels as Rewards, Not Targets: Group-Relative Policy Optimization for Semi-Supervised ASR

When Synthetic Speech Is All You Have: Better Call GRPO

MERaLiON-GR: Speech Gender Recognition Model for English and SEA Languages