From Statistical Methods to Pre-Trained Models; A Survey on Automatic Speech Recognition for Resource Scarce Urdu Language
Organizations: State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation Chinese Academy of Sciences, Beijing 100190, P.R. China · University of Chinese Academy of Sciences, Beijing China · School of Electronics and Information Engineering, Shenzhen University · School of Artificial Intelligence, CAS Institute of Automation
Abstract
Automatic Speech Recognition (ASR) technology has witnessed significant advancements in recent years, revolutionizing human-computer interactions. While major languages have benefited from these developments, lesser-resourced languages like Urdu face unique challenges. This paper provides an extensive exploration of the dynamic landscape of ASR research, focusing particularly on the resource-constrained Urdu language, which is widely spoken across South Asian nations. It outlines current research trends, technological advancements, and potential directions for future studies in Urdu ASR, aiming to pave the way for forthcoming researchers interested in this domain. By leveraging contemporary technologies, analyzing existing datasets, and evaluating effective algorithms and tools, the paper seeks to shed light on the unique challenges and opportunities associated with Urdu language processing and its integration into the broader field of speech research.
Figures & tables
| S. No | Tool(s) | Technology | Accuracy WER | Ref. |
|---|---|---|---|---|
| 1 | Scikit, Keras | SVM, CNN | 0.97 | Chandio et al. (2021) |
| 2 | libSVM | SVM, RF, LDA | 73% | Ali et al. (2015b) |
| 3 | Sphinx, CLTK | HMM | 74% | Shaukat et al. (2016) |
| 4 | Sphinx | SVM, GMM | 92.4% | Qasim et al. (2016b) |
| 5 | Kaldi, SRILM | SGMM, MMI | 24.2 | Raza et al. (2018) |
| 6 | Sphinx4 | HMM | 10.66 | Ashraf et al. (2010) |
| S. No | Dataset | Size (utts./hrs) | Vocab | Speech Type | Public | Ref. |
|---|---|---|---|---|---|---|
| 1 | Urdu Spoken Words | 5200 utts. | 52 | Isolated | No | Ashraf et al. (2010) |
| 2 | AUDD | 25,518 utts. | - | isolated | No | Chandio et al. (2021) |
| 3 | Isolated Urdu Corpus | 250 Words | 2500 | Isolated | Yes | Ali et al. (2016) |
| 4 | PRUS Corpus | 70 Minutes | 42,289 | Read Speech | Yes | Raza et al. (2010) |
| 5 | District Names DS | 9.5 hours | 41293 | Spontaneous, isolated | Yes | Qasim et al. (2016a) |
| 6 | Urdu Mixed Datasets | 126.8 hours | - | Read Speech | Partially | Naeem et al. (2020) |
| S. No | Technique(s) | %WER/ Accuracy | Ref. |
|---|---|---|---|
| 1 | SGMMI | 16.7 | Naeem et al. (2020) |
| 2 | SVM, CNN | 0.97 | Chandio et al. (2021) |
| 3 | GMM, CMLLR | 32.6 | Shaik et al. (2015) |
| 4 | SVM, RF, LDA | 73% | Ali et al. (2015a) |
| 5 | HMM | 74% | Shaukat et al. (2016) |
| 6 | SVM, GMM | 92.4% | Qasim et al. (2016b) |
| S. No | A.M Technique(s) | Language Model LM | WER (%) Urdu | Avg WER (%) (Multi Lingual) | Ref. |
|---|---|---|---|---|---|
| 1 | SSL, IndicWav2Vec2, XLS-R | KenLM | 12.6 | 12.4 | Javed et al. (2022) |
| 2 | Wav2Vec, Conformer, CTC, OpenSLR | KenLM | - | 15.3 | Bhogale et al. (2023b) |
| 3 | IndicWav2Vec | - | 25.1 | 21.0 | Bhogale et al. (2023a) |
| IndicWhisper, etc. | 19.4 | 13.6 | |||
| Google SST | 23.3 | 23.9 | |||
| Azure SST | - | 20.0 |
| Ref. | Method | Objective | Accomplishments | Limitation |
|---|---|---|---|---|
| Javed et al. (2022) | IndicSUPERB Wav2Vec2-based multimodal ASR | • Larger dataset, 1684 hours, for 12 Indic languages including Urdu. • Benchmarks for speech tasks including ASR. • Train and evaluate self-supervised models along with FBANK. | • Able to reduce average WER for average and specific language. • Able to boost the accuracy with a Jointly fine-tuned multilingual model | • Limited to 12 Indic languages. • Test results with unknown speakers are different from those with known speakers. • Not robust in noisy environment |
| Bhogale et al. (2023b) | Shrutilipi dataset Wav2vec2-based multimodal ASR | • Proposed mining text and audio pairs of low-resource Indic languages, Shrutilipi 6400 hours, as an alternative to creating labeled datasets. • Adopt CTC with the Needleman-Wunsch algorithm. | • Able to reduce WER by 5.8% for 7 languages, can further with the Conformer model. • High quality and diverse dataset effective for ASR down streaming. • Able to Train models which are more robust to noisy input. | • Limited to 12 Indic languages • Smallest proportion of Urdu. • Unable to show language-specific accuracy |
| Bhogale et al. (2023a) | Vistaar IndicWhisper | • A set of 59 benchmarks collated from ASR systems named Vistaar. • trained for 10.7K hours long 12 languages. | • Diversity in every language. • Reduce an average of 4% WER. • High volume Urdu speech data 320 hours with 13.6% WER. | • More datasets are possible to compete with high-resource languages and multi-lingual benchmarks. |
| Radford et al. (2023) | Whisper: Wav2Vec and Seq-to-seq architecture | • Multilingual and multi-task 680,000 hours labeled datasets. • Whisper Model: Robust speech processing with pre-trained Models based on Seq-to-seq transformer architecture. | • Capable of getting an Average 29.3% WER on wav2vec without the language model, and 12.8% WER on the Whisper model. • Whisper model robust enough for long-form (few minutes to few hours) transcriptions. | • Low accuracy for low-resource languages. • Highest proportion of English • Lacks fine-tuning • Strong decoder but no CTC or wav2vec together with a language model. |
| Conneau et al. (2023) | FLEURS Few-shot learning, mSLAM, and wav2vec-Bert Models | • Consists of 102 languages with a 12-hour dataset per language including Urdu. • Attempt to catalyze resource-scarce languages in speech tech technology. | • Simple, baselines for the ASR tasks on pre-trained mSLAM and wav2vec-Bert models. • Considerable %CER specifically in Urdu with %CER reductions of 82.9 and 83.1 respectively. | • Limited datasets • No Language model |
Automatic Speech Recognition
Automatic speech recognition converts spoken language into text across speakers, languages, domains, and recording conditions.
See all 484 papers