Low-resource, agglutinative languages challenge low-latency ASR, as fixed lookahead windows fail when grapheme order diverges, suffixes cross code-switch boundaries, or code-switching data is imbalanced. We present the first comprehensive study of streaming ASR for intra-sentential Bangla-English code-switching, identifying a mismatch between strict causal streaming and long-range morphological dependencies. We introduce a Dynamic Block-Online framework with VAD-aligned inference that balances semantics, latency, and context, enabling global bidirectional attention to preserve morphological integrity. To mitigate data scarcity, we propose Script-Anchored Loanword Injection, augmenting training text with English loanwords to model phonotactic transitions. We introduce CS-WER to diagnose switch, root, and suffix errors, and show that flexible latency budgets enhance recognition of agglutinative and mixed-language speech, with successful transfer to menstrual-medical domains.
Figures & tables
Figure 1: Comparison of (a) Fixed-Latency Causal Streaming, where memory-cached attention leads to CS morphological truncation (mistranscribing and board [bord]), and (b) our proposed Dynamic Block-Processing, where VAD-aligned global attention resolves long-form agglutinative dependencies.
Figure 2: Proposed Methodology of: Script-Anchored Loanword Injection - converting Bengali to CS transcript through a loanword database M . A Dynamic Block Online model that strips audio at communicative pauses and uses an offline encoder Eoff , CIF predictor, and Decoder and how CS-WER is calculated showcased by what each of Eswitch/Eroot/Emorph looks at.
Model
Data
w
Common Voice
CER
WER
CS-WER
Whisper Offline
Largev3 [ 1 ]
-
Full
-
40.30
.94/.96/.97
Largev2 [ 1 ]
-
Full
-
62.31
.98/.94/.99
Others Offline
MBNSpeech [ 16 ]
-
Full
11.44
42.3
1/1/1
Table 1: Comparison of CER, WER, and Fine-Grained CS-WER ( Eswitch/Eroot/Emorph ) across Offline, Causal Streaming, and Dynamic Block architectures. ’B+E’, ’C’, and ’aug’ denote monolingual training, loanword injection, and noise augmentation, respectively.
Figure 3: Illustration of different model outcomes by the streaming, block online, and offline model: (a) Streaming latency against Eroot of streaming models as circles and diamonds, compared to block online (square) and offline (cross) models. (b) Error topology showcasing the Global WER of the streaming model and the dynamic block-online model (c) Code-Switched generations of dynamic block-online (red) and streaming model (blue).
Code-switching (CSW) remains challenging for large multi-lingual ASR systems in real-world deployment. While fine-tuning on synthetic CSW data is possible, it generally degrades strong monolingual baselines. Our goal is to preserve these capabilities while extending models to handle complex code-switching, including morphological variations across languages. We propose Bayesian factorized adaptation, which learns to efficiently integrate switching-relevant knowledge into strong pretrained models without overwriting existing capabilities. Requiring only a small amount of synthetic data, our approach reduces transcription errors by 32.87% on code-switched words while improving overall WER by 5.31%, all while maintaining mono-lingual performance. Our results demonstrate that effective CSW adaptation depends more on knowledge integration than data complexity.
Enes Yavuz Ugan, Alexander Waibel
Interactive Systems Lab, Karlsruhe Institute of Technology (KIT), Germany · InterACT, Carnegie Mellon University (CMU), USA
Code-switching (CS) speech, characterized by language alternation within a single utterance, is prevalent in multilingual communities and poses challenges to automatic speech recognition (ASR) due to rapid language transitions. We propose LLM-HB, a language-aware large language model-guided hotword biasing framework for code-switching ASR (CS-ASR). The framework incorporates a mixture-of-experts (MoE) adaptor and an auxiliary language prediction head to better capture and differentiate multilingual representations, while hotwords are injected through LLM-based prompting. Experiments on the ASRU2019 code-switching challenge test set demonstrate that LLM-HB achieves a 20.30% relative reduction in mixed error rate (MER) over the baseline, reaching 5.85% MER under a 15-distractor hotword setting. To the best of our knowledge, this is the first work to integrate LLM-guided biasing into code-switching ASR. We further release the extracted ASRU2019 hotword list to support future research.
Yuxuan He, Genshun Wan, Pengcheng Li +3
Auditory Cognition and Computational Acoustics Lab, Shanghai Jiao Tong University, Shanghai, China · AI research Institute, iFLYTEK Company Limited, Hefei, Anhui, China · School of Electronic Information, Wuhan University, Wuhan, Hubei, China
Lightweight speech recognition models are critical for edge deployment, yet highly optimized architectures like Moonshine often fail on morphologically rich, non-Latin languages such as Bengali. This study identifies the root cause of this failure as the model's English-centric byte-level tokenizer, which fragments Bengali words into high-fertility byte chains and triggers catastrophic autoregressive collapse during inference. To resolve this, a novel vocabulary transplantation pipeline is proposed to replace the decoder vocabulary with the native-script BanglaBERT WordPiece vocabulary and resize the corresponding token embedding matrix. Experimental results demonstrate a reduction in token fertility from 9.16 to 1.30. By decreasing autoregressive sequence length by 85.8%, decoding instability is entirely mitigated. When evaluated on the 882-hour Lipi-Ghor dataset, the modified architecture achieves a competitive 21.54% Word Error Rate (WER) and a Real-Time Factor (RTF) of 0.0053. Ultimately, this research provides a scalable, reproducible blueprint for cross-script adaptation of compact ASR models without the need for resource-intensive pre-training.
Sanjid Hasan, Md. Abdur Rahman
Khulna University of Engineering & Technology, Bangladesh · Military Institute of Science and Technology, Bangladesh