Tone-Conditioned Curriculum Learning for Low-Resource Bantu Speech Recognition

Kesego Mokgosi; Vukosi Marivate; Sitwala Mundia; Unarine Netshifhefhe; Tsholofelo Hope Mogale; Thapelo Sindane

cs.CLJun 30, 2026

Tone-Conditioned Curriculum Learning for Low-Resource Bantu Speech Recognition

Authors: Kesego Mokgosi, Vukosi Marivate, Sitwala Mundia, Unarine Netshifhefhe, Tsholofelo Hope Mogale, Thapelo Sindane

Organizations: Technological University Dublin, Ireland · Data Science for Social Impact, University of Pretoria, South Africa · Lelapa AI

Abstract

Southern Bantu languages are spoken by over 80 million people, yet current foundation ASR models still produce zero-shot WER above 100%, which limits practical use in education and public services. We addressed this gap with a tone conditioned curriculum framework for 6 Southern Bantu languages that combined hybrid difficulty scoring, gated adapters driven by tonal statistics and staged curriculum training. We trained on a community corpus and tested transfer to NCHLT to measure robustness beyond matched evaluation. Results revealed clear interactions between architecture and language, with W2V-BERT outperforming Whisper on Nguni languages by 3 to 4 WER points whilst Whisper performed better on Sotho-Tswana languages. W2V-BERT with tone conditioning reached 28.41% average WER across datasets and 23.79% on Xitsonga transfer. No single model suited all 6 languages, so deployment should pair model selection per language with validation across corpora.

Explore similar work

DONDO: Open w2v-BERT Speech-Recognition Base Models for African Languages
Jul 23, 2026Paul Azunre, Naafi Ibrahim, Joel Budu +1African Languages Bert-Based Models
WAXAL-NET: Finetuned Edge ASR Across 19 African Languages
Jun 1, 2026Victor Tolulope Olufemi, Oreoluwa Babatunde, Ramsey Njema +28Llm-Based Automatic Speech Recognition African Languages
Morpho-VITS: Variational Inference with Morphological Modeling for End-to-End Speech Synthesis of a Tonal Bantu Language
Sep 21, 2026Antoine Nzeyimana

Jul 23, 2026 · cs.CLJ/K move · Enter open · S save

DONDO: Open w2v-BERT Speech-Recognition Base Models for African Languages

Paul Azunre, Naafi Ibrahim, Joel Budu, Lawrence Adu-Gyamfi

Khaya AI

We present DONDO, a family of open, permissively licensed automatic speech recognition (ASR) base models for African languages, built on the w2v-BERT 2.0 self-supervised speech encoder. DONDO comprises twenty-one monolingual models and five multilingual models spanning twenty-seven language varieties across Ghana, Sierra Leone, Nigeria, Senegal, Kenya and Zimbabwe. Models are fine-tuned primarily on read speech drawn from religious texts, which offer broad, license-clear and orthographically consistent coverage for languages that otherwise lack transcribed audio. We describe a two-step (and, for one family, three-step) learning-rate-annealed fine-tuning procedure that first adapts a shared multilingual model at a high learning rate and then anneals it to recover, and in several cases surpass, strong monolingual baselines. We further describe a lightweight language-conditioning mechanism that injects a one-hot language identity as a sequence of prefix frames prepended to the acoustic features, allowing a single multilingual checkpoint to be steered to a target language at inference. Across the five multilingual families the annealed models reach average word error rates (WER) of 10-13%, closing most of the gap to monolingual models while covering many languages in a single checkpoint. All models are released on the Hugging Face KhayaAI organisation under the Apache-2.0 license (attribution only) so that others may fine-tune them freely, including for commercial use. We provide a conservative estimate that the languages covered are spoken by on the order of one hundred million first-language speakers, and by substantially more when second-language use is included.

African Languages Bert-Based Models

Open paper Ask about it PDF

Abstract

Explore similar work

DONDO: Open w2v-BERT Speech-Recognition Base Models for African Languages

WAXAL-NET: Finetuned Edge ASR Across 19 African Languages

Morpho-VITS: Variational Inference with Morphological Modeling for End-to-End Speech Synthesis of a Tonal Bantu Language