LLM-HB: Language-Aware LLM-Guided Hotword Biasing for Code-Switching ASR
Organizations: Auditory Cognition and Computational Acoustics Lab, Shanghai Jiao Tong University, Shanghai, China · AI research Institute, iFLYTEK Company Limited, Hefei, Anhui, China · School of Electronic Information, Wuhan University, Wuhan, Hubei, China
Abstract
Code-switching (CS) speech, characterized by language alternation within a single utterance, is prevalent in multilingual communities and poses challenges to automatic speech recognition (ASR) due to rapid language transitions. We propose LLM-HB, a language-aware large language model-guided hotword biasing framework for code-switching ASR (CS-ASR). The framework incorporates a mixture-of-experts (MoE) adaptor and an auxiliary language prediction head to better capture and differentiate multilingual representations, while hotwords are injected through LLM-based prompting. Experiments on the ASRU2019 code-switching challenge test set demonstrate that LLM-HB achieves a 20.30% relative reduction in mixed error rate (MER) over the baseline, reaching 5.85% MER under a 15-distractor hotword setting. To the best of our knowledge, this is the first work to integrate LLM-guided biasing into code-switching ASR. We further release the extracted ASRU2019 hotword list to support future research.
Figures & tables
| Method | Training Dataset | MER | B-MER | CER | B-CER | WER | B-WER |
|---|---|---|---|---|---|---|---|
| Base Model | ASRU2019-CS-Train-Set | 9.59 | 26.91 | 7.03 | 25.45 | 30.44 | 27.27 |
| + MoE Adaptor | 9.65 | 27.18 | 7.06 | 25.57 | 30.73 | 27.58 | |
| + Language Head | 9.30 | 26.27 | 6.77 | 24.50 | 29.87 | 26.72 | |
| + Hotword Prompt | 7.48 | 16.00 | 5.92 | 10.49 | 20.21 | 17.75 | |
| + MoE + LH + HP (LLM-HB) | 7.36 | 14.14 | 6.02 | 8.11 | 18.29 | 16.07 | |
| Base Model (baseline) | ASRU2019-CS- Train-Set + Librispeech + Aishell-2 | 7.34 | 22.11 | 5.04 | 19.26 | 26.05 | 22.90 |
| Configuration | MER | CER | WER |
|---|---|---|---|
| 4 choose 4 | 7.33 | 5.01 | 25.99 |
| 4 choose 2 | 7.47 | 5.10 | 26.79 |
| 2 choose 2 (Ours) | 7.19 | 4.89 | 25.94 |
| 2 choose 1 | 7.21 | 4.89 | 26.03 |
| 1 choose 1 | 7.23 | 4.90 | 26.14 |
Code-Switching Speech Recognition
Code-switching speech recognition transcribes speech that shifts between languages, preserving words, language boundaries, and writing systems.
See all 22 papers