cs.LGNov 3, 2026

Text-Only Domain Adaptation for Low-Resource Medical ASR: Shallow Fusion and Density Ratio Rescoring for Hausa

Authors: Oche David Ankeli, Hassan Adelani Luqman, Theodora Ngozi Egbunike, Joshua Gottschalk, Bonaventure F. P. Dossou

Organizations: RUZIVO Research Lab, African Leadership University, Kigali, Rwanda · Stanford University, Stanford, CA, USA · Mila – Quebec AI Institute and McGill University, Montreal, Canada

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Automatic speech recognition (ASR) systems often struggle with specialised medical vocabulary, especially in low-resource languages where domain-specific speech data are scarce. We investigate text-only domain adaptation for Hausa medical ASR using the fixed MMS-1B-FL102 acoustic model and two decoding-time language-model fusion methods: Shallow Fusion (SF) and the Density Ratio Approach (DRA). A 4-gram KenLM trained on 19,421 Hausa health sentences serves as the medical target model, while a 75,372-sentence general-domain model provides the source prior for DRA. Evaluation uses 250 recordings of 25 physician-reviewed prompts from 10 speakers across six northern Nigerian states. Under prompt-disjoint five-fold cross-validation, SF and DRA reduce WER from 29.56% to 23.55% and 23.33%, respectively. Both significantly outperform greedy decoding (p < 0.001), while their difference is not significant (p = 0.423). However, general-LM-only SF achieves 23.96% WER, indicating that the medical LM's additional improvement over a general LM is modest. Medical Term Error Rate also improves under fusion, but does not establish a distinct medical-domain advantage. These findings demonstrate that lightweight, text-only language-model fusion can improve Hausa medical ASR without additional domain-specific speech collection.

Figures & tables

Figure 1: Geographic distribution of speakers. States shaded contributed speakers to the evaluation corpus.
Figure 2: Word Error Rate, Character Error Rate, and Medical Term Error Rate across Greedy, Shallow Fusion, and Density Ratio decoding, pooled out-of-fold.
Figure 3: (a) WER by sentence tier. (b) DRA’s relative WER improvement over greedy, by tier.
Figure 4: Relative improvement in MTER consistently exceeds relative improvement in WER for both fusion methods.
Figure 5: Per-speaker Word Error Rate across all three decoding strategies, showing consistent improvement across the cohort.
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Speaker IDStateRegionSexHausa ProficiencyOccupation
S04PlateauNorth-CentralMNativeNGO / Community
S06GombeNorth-EastMFluent (L2)Student
S08TarabaNorth-EastMNativeStudent
S10YobeNorth-EastMNativeNGO / Community
S11KadunaNorth-WestMNativeStudent
S13GombeNorth-EastFNativeStudent
Appendix
Table 5: Anonymised speaker demographics. 8 of 10 speakers are native Hausa speakers; 2 are fluent second-language speakers.
IDTierHausa Sentence (Validated)English Translation∗\ast
0011Ka sha ruwa mai yawa kowace rana.Drink plenty of water every day. 
0021Ina jin zafi a kirjina.I feel pain in my chest. 
0031Jikinna ya kama zafi yau sosai.My body is very hot today.∗\ast
0041Ina buƙatar magani don ciwon kai.I need medicine for a headache. 
0051Ka huta sosai don ka warke da wuri.Rest well for a quick recovery.∗\ast
0061Ina jin kishin ruwa sosai.I am feeling very thirsty.∗\ast
Appendix
Table 6: All 25 evaluation sentences in final validated form. ∗\ast = linguistically corrected by medical annotators.
Category (terms)Terms ( †\dagger =corpus-derived, ‡\ddagger =expert-validated)
Symptoms & Conditions (22)amai †‡ ciwo †‡ ciwon †‡ cizon †‡ cutar †‡ gudawa †‡ hadari †‡ haɗari †‡ karkarwa †‡ rashin †‡ rawar †‡ sauro †‡ sukari †‡ tari †‡ tsanani †‡ zazzabi †‡ zazzabin †‡ rauni † hawan ‡ kishirwa ‡ zafi ‡ ƙishirwa ‡
Medications & Treatments (12)allurar †‡ kulawa †‡ kwayar †‡ magani †‡ maganin †‡ magunguna †‡ rigakafi †‡ tiyata †‡ ƙwayar †‡ kise † sha ‡ warke ‡
Clinical Procedures (6)gwaji †‡ gwajin †‡ dawo ‡ diba ‡ duba ‡ rubuta ‡
Medical Personnel & Settings (2)asibiti †‡ likita †
Anatomy (12)jikinna †‡ jini †‡ jinina †‡ kashin †‡ ƙashin †‡ baya ‡ hannuwana ‡ jikansa ‡ jiki ‡ jikina ‡ kirji ‡ kirjina ‡
Clinical Descriptors (12)alamu †‡ iyaka †‡ makonni †‡ matakan †‡ matsin †‡ musamman †‡ tsanani †‡ bukaci † bukatar † buƙaci † buƙatar † zuciyata †
Appendix
Table 7: Medical term lexicon: 66 term strings across six clinical categories. tsanani appears in two categories (Symptoms & Conditions and Clinical Descriptors), so category slots total 66 while only 65 terms are categorised; lafiya is expert-validated but not assigned to a category (and does not occur in the 25 reference sentences).
FoldHeld-out promptsSF λ\lambdaSF β\betaGen. λ\lambdaGen. β\betaDRA λτ\lambda_{\tau}DRA λψ\lambda_{\psi} / β\beta
1010, 016, 017, 020, 0210.81.50.81.51.00.1 / 0.5
2007, 008, 011, 018, 0250.80.00.81.51.20.1 / 0.5
3001, 004, 013, 019, 0220.81.50.81.51.20.1 / 0.5
4003, 006, 012, 015, 0240.81.50.81.51.20.1 / 0.5
5002, 005, 009, 014, 0230.81.50.81.51.20.1 / 0.5
Appendix
Table 8: Per-fold selected hyperparameters. SF λ\lambda / β\beta : medical-LM Shallow Fusion weight/word score. Gen. λ\lambda / β\beta : general-LM-only SF ablation. DRA λτ\lambda_{\tau} / λψ\lambda_{\psi} / β\beta : Density Ratio weights. DRA nn -best = 50 throughout.
SpeakerGreedySFDRA
S0426.01%22.34%22.71%
S0634.80%27.84%28.21%
S0832.60%25.64%24.91%
S1026.37%20.88%20.51%
S1124.54%18.68%18.68%
S1325.64%21.61%20.15%
Appendix
Table 9: Per-speaker pooled out-of-fold WER across all three decoding strategies.
More in this topic

Domain Adaptation for ASR

Domain adaptation for automatic speech recognition (ASR) tailors speech models to new speakers, domains, languages, and acoustic conditions.

See all 55 papers

Explore similar work

CardsList
Aug 19, 2026

Understanding Multilingual Medical ASR Adaptation Through Layer-Wise Analysis

Medical automatic speech recognition (MedASR) requires adaptation to specialised terminology, limited annotated clinical data, and multilingual use cases. Although large-scale pretrained ASR models such as Whisper achieve strong generalisation, their behaviour after medical and multilingual adaptation remains insufficiently understood beyond word error rate (WER). This paper investigates how multilingual medical adaptation reshapes the internal representations of Whisper models through layer-wise encoder analysis. We compare zero-shot decoding, English-only fine-tuning, German-only diagnostic fine-tuning, two-stage EN->EN+DE continuation, and direct EN+DE fine-tuning across Whisper model sizes. Fine-tuning substantially improves MedASR performance, but the best model depends on the adaptation setting: Whisper-Medium gives the lowest English WER (7.72%) and the lowest combined EN+DE WER under direct EN+DE training (26.30%); German-only Whisper-Large-v3 gives the lowest German WER (44.96%), but as a within-corpus diagnostic on 86 single-speaker training utterances rather than robust generalisation. Layer-wise analysis of the two-stage Whisper-Small trajectory shows that English medical fine-tuning produces the dominant encoder shift, whereas multilingual continuation largely preserves the adapted representation space. Domain and language information remain highly recoverable across layers, while linearly recoverable error-predictive cues weaken as WER improves.

Souranil Kahali, Rituparna Bose, Abner Hernandez +4

Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany · School of Computer Science, University of Sheffield, Sheffield, UK · Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM), Munich, Germany

Medical Speech RecognitionAutomatic Speech RecognitionDomain Adaptation for ASRMultilingual ASRRepresentation Learning
AskPDF
Apr 7, 2026

Closing the Speech-Text Gap with Limited Audio for Effective Domain Adaptation in LLM-Based ASR

Conventional end-to-end automatic speech recognition (ASR) systems rely on paired speech-text data for domain adaptation. Recent LLM-based ASR architectures connect a speech encoder to a large language model via a projection module, enabling adaptation with text-only data. However, this introduces a modality gap, as the LLM is not exposed to the noisy representations produced by the speech projector. We investigate whether small amounts of speech can mitigate this mismatch. We compare three strategies: text-only adaptation, paired speech-text adaptation, and mixed batching (MB), which combines both. Experiments in in-domain and out-of-domain settings show that even limited speech consistently improves performance. Notably, MB using only 10% of the target-domain (less than 4 hours) speech achieves word error rates comparable to, or better than, conventional ASR fine-tuning with the full dataset, indicating that small amounts of speech provide a strong modality-alignment signal.

Thibault Bañeras-Roux, Sergio Burdisso, Esaú Villatoro-Tello +9

Idiap Research Institute, Switzerland · Laboratoire d’Informatique et des Syst`emes, France · EPFL, Switzerland +2

Domain Adaptation for ASRCross-Modal AlignmentAutomatic Speech RecognitionSpeech Language Models
AskPDF
Sep 24, 2026

Benchmarking and Domain Adaptation of Automatic Speech Recognition (ASR) for Adolescent Health Communication in Ghanaian Languages

This paper presents an end-to-end study of automatic speech recognition (ASR) for adolescent health communication in three Ghanaian languages (Twi, Dagbani, and Ewe). The work proceeds in three connected stages; First, we benchmark five ASR systems (three language-specific Wav2Vec2 models and two multimodal LLMs, Gemma 3n and Gemma 4) on a general-domain Bible corpus and a Youth Adolescent Sexual and Reproductive Health (ASRH) Domain ASR dataset, using Character and Word Error Rate (CER, WER). Second, guided by the benchmark, we perform supervised domain adaptation: although Gemma 4 was the strongest zero-shot candidate, fine-tuning it proved computationally infeasible, so we pivoted to the compact Qwen3-ASR-0.6B, fine-tuned on a large Ghana Bible corpus (~90k samples) and evaluated strictly on held-out human-collected in-domain audio. Fine-tuning reduced WER on every language, most dramatically for Ewe (WER from 109.3% to 64.8%, a drop of 44.5 pp; CER from 65.1% to 24.9%). Third, we validate the work through KasaHealth, a live voice-first ASRH application deployed in all three languages, complemented by Senti-Check, a technical evaluation harness. KasaHealth was tested by 50 community respondents and achieved a 100% chat-approval rate, a 72% Good-or-Excellent translation rating, and a 92% would-recommend rate, while surfacing the domain gaps that most constrain real-world use. Across all three stages the evidence converges: for these languages the binding constraint is validated in-domain data, not model capability or computation.

Stephen E. Moore, Akwasi Asare, Mich-Seth Owusu +3

Department of Mathematics, University of Cape Coast, Ghana · Ghana Natural Language Processing, Cape Coast, Ghana

ASR EvaluationDomain Adaptation for ASRMedical Speech RecognitionAutomatic Speech RecognitionLow-Resource Speech RecognitionMultilingual ASR
AskPDF