Continued Pretraining for Low-Resource Swahili ASR: Achieving State-of-the-Art Performance with Minimal Labeled Data

Hillary Mutisya; John Mugane

cs.SDMar 11, 2026

Continued Pretraining for Low-Resource Swahili ASR: Achieving State-of-the-Art Performance with Minimal Labeled Data

Authors: Hillary Mutisya, John Mugane

Organizations: Thiomi-Lugha NLP · Harvard University

Abstract

We investigate continued pretraining (CPT) for adapting wav2vec2-bert-2.0 to Swahili automatic speech recognition (ASR). Our approach combines unlabeled audio with limited labeled data through pseudo-labeled CPT followed by supervised finetuning. With 20,000 labeled samples, we achieve 3.24% WER on Common Voice Swahili-an 82% relative improvement over the baseline. This result surpasses the best previously reported academic system (8.3% WER from XLS-R) by 61% relative improvement. We provide concrete data requirements and a replicable methodology applicable to other low-resource languages.

Figures & tables

Panel of Figure 1 — Figure 1: Wav2vec2-BERT 2.0 architecture showing the flow from raw audio input through convolutional feature extraction, transformer-based contextual encoding, and task-specific CTC head for ASR. The model was pretrained on 4.5M hours across 104 languages including Swahili.

Config	Baseline	Final WER	$\Delta$	Benefit
5K+CPT	17.71%	10.89%	-38.5%	✓
20K+CPT	17.71%	3.24%	-81.7%	✓

Table 1: Swahili ASR performance across training configurations. Baseline refers to the 50K comparison model trained without CPT.

\Delta

represents relative improvement over baseline.

System	Type	WER
XLS-R finetuned [ 1 ]	Academic	8.3%
SpeechBrain [ 7 ]	Academic	9-12%
Ours (5K+CPT)	Research	10.89%
Ours (20K+CPT)	Research	3.24%

Table 2: Swahili ASR benchmarks comparing our approach with published academic systems

Automatic Speech Recognition

Automatic speech recognition converts spoken language into text across speakers, languages, domains, and recording conditions.

See all 480 papers

Sequential Adapter Stacking for Cross-Lingual Low-Resource ASR
Sep 14, 2026Thai Thi Thanh Thao Dang, Mengjie Qian, Kate KnillLow-Resource Speech Recognition Parameter-Efficient Fine-Tuning
Department of Engineering, University of Cambridge, UK
Automatic Speech Recognition for Low-Resource Sinhala: A Critical Review of Methods, Challenges, and Future Directions
Oct 5, 2026Chanuka Dinuwan, Sanath Jayasena, Buddhika KarunarathneASR Evaluation Automatic Speech Recognition
MSc in Data Science and Artificial Intelligence Department of Computer Science & Engineering University of Moratuwa, Sri Lanka · University of Moratuwa, Sri Lanka
Nufo: A Benchmark for Ewe Automatic Speech Recognition
Oct 12, 2026Christ Kekeli Keli, Steve Sanogo, Patrice Sebastiano +2Automatic Speech Recognition Low-Resource Speech Recognition
Universit´e d’Avignon, LIA, France · Brigham Young University–Hawaii, Laie, Hawaii

Sep 14, 2026J/K move · Enter open · S save

Sequential Adapter Stacking for Cross-Lingual Low-Resource ASR

Thai Thi Thanh Thao Dang, Mengjie Qian, Kate Knill

Department of Engineering, University of Cambridge, UK

Low-Resource Speech Recognition Parameter-Efficient Fine-Tuning Cross-Lingual Transfer Multilingual ASR

Open paper AskPDF arXiv

Extending large-scale multilingual automatic speech recognition (ASR) models to low-resource languages remains challenging. Model performance is skewed toward high-resource languages and degrades sharply for languages with limited labeled data and pre-training exposure. To address this, we investigate parameter-efficient approaches for transferring knowledge from resource-rich source languages to low-resource target languages on Whisper. Alongside warm initialization and attention-based fusion, we propose Sequential Adapter Stacking, which places a trainable target-language adapter on top of a frozen source-language adapter. Under controlled experiments, these approaches are evaluated on three target languages unsupported by Whisper -- Asturian, Assamese, and Xhosa -- using source languages with varying degrees of relatedness. Sequential Adapter Stacking with the closest related source consistently and significantly outperforms full fine-tuning across the three targets, with 5--8% relative WER reductions. These gains largely persist with only one hour of target training data.

Abstract

Automatic Speech Recognition

Explore similar work

Sequential Adapter Stacking for Cross-Lingual Low-Resource ASR

Automatic Speech Recognition for Low-Resource Sinhala: A Critical Review of Methods, Challenges, and Future Directions

Nufo: A Benchmark for Ewe Automatic Speech Recognition