ELARSpeech: A 30K-Hour Dataset Across 675 Underrepresented Languages for Long-Tail Multilingual Speech Technologies
Organizations: Yonsei University · McGill University · Mila - Quebec AI Institute · Canada CIFAR AI Chair
Abstract
Massively multilingual speech modeling has made remarkable progress in recent years, yet existing training resources and benchmarks still underrepresent much of the linguistic long tail. To address this gap, we introduce ELARSpeech, a large-scale multilingual speech resource comprising 30,202 hours of speech across 675 underrepresented languages, curated from the Endangered Languages Archive. Combining human-annotated and unlabeled recordings, the corpus spans diverse recording conditions, domains, and language families. Beyond corpus construction, our experiments show how ELARSpeech supports extending ASR and TTS capabilities to diverse underrepresented languages, with further benefits from pseudo-labeling unlabeled data. At the same time, our speech representation analysis reveals remaining limitations in transferring existing multilingual models to the linguistic long tail. Together, ELARSpeech provides both a resource and a testbed for extending multilingual speech technology further into the linguistic long tail. We publicly release the dataset, all trained ASR and TTS models, and the complete data construction pipeline to facilitate research on long-tail multilingual speech technologies.
Figures & tables
| Dataset | Labeled | Content | # Langs. | Size (H) |
| Closed-Source Datasets | ||||
| Google USM | Variety | 568 | 12.1M | |
| MMS-Unlab | Bible | 2809 | 8K | |
| Whisper | Variety | 98 | 680K | |
| MMS-Lab | Bible | 1107 | 45K | |
| ALLASR | Variety | 1690 | 121K | |
| Type | Split | # Lang. | # Utt. (M) | Duration (H) |
| Labeled | Train | 409 | 3.09 | 2367.08 |
| Dev | 233 | 0.17 | 132.77 | |
| Test | 233 | 0.35 | 266.14 | |
| Multi | 82 | 0.04 | 24.82 | |
| Total | 443 | 3.64 | 2790.81 | |
| Unlabeled | Train | 592 | 19.14 | 26763.37 |
| Languages with ISO 639-3 Codes | Languages without ISO 639-3 Codes | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Language | Code | Data (H) | Monolingual | Multilingual | Language | Code | Data (H) | Monolingual | Multilingual | ||||
| OmniASR | Whisper | Whisper | OmniASR | Whisper | Whisper | ||||||||
| 300M | 1B | 1.5B | 1.5B | 300M | 1B | 1.5B | 1.5B | ||||||
| Masela, Central | mxz | 136.8 | 28.90 | 26.22 | 28.39 | 28.31 | Jitotoltec | – | 18.6 | 15.16 | 15.25 | 13.31 | 20.84 |
| Ju|’hoansi | ktz | 79.9 | 41.89 | 41.71 | 42.93 | 53.06 | Zok | – | 18.4 | 35.04 | 36.95 | 38.35 | 47.55 |
| Yoloxóchitl Mixtec | xty | 66.5 | 8.76 | 8.58 | 8.70 | 16.84 | Sri Lankan Portuguese Creole | – | 17.2 | 18.78 | 17.86 | 16.89 | 22.81 |
| Language (Code) | Test | CER | ||
|---|---|---|---|---|
| Original | +L | +LU | ||
| CommonVoice 26 (CV26) | ||||
| Dawoodi (dmk) | ELAR | 93.99 | 22.68 | 22.72 |
| CV26 | 4.47 | 4.81 | 5.58 | |
| Imbabura Highland Quichua (qvi) | ELAR | 47.48 | 7.13 | 6.71 |
| CV26 | 1.28 | 1.12 | 1.10 | |
| Language | Code | Data (H) | Reference | Monolingual | Multilingual | |||
| (Ground Truth) | OmniVoice | OmniVoice | ||||||
| CER | MOS | CER | MOS | CER | MOS | |||
| Languages with ISO 639-3 Codes | ||||||||
| C. Masela | mxz | 124.9 | 28.39 | 4.04 | 14.71 | 3.88 | 12.87 | 3.79 |
| Y. Mixtec | xty | 64.8 | 8.70 | 4.12 | 4.75 | 3.58 | 4.83 | 3.82 |
| Tzeltal | tzh | 19.8 | 23.20 | 3.76 | 12.58 | 3.33 | 12.93 | 3.45 |
| Model | Data (H) | # Langs. | # ELAR | # Params. | Mono. ASR | Multi. ASR | Multi. LID | Multi. ASR + LID | Score | |
| CER | CER | Acc. | Acc. | CER | ||||||
| w2v-BERT 2.0 † | 4.5M | 143 | – | 600M | 41.0 / 29.2 | 24.6 / 20.3 | 70.0 / 86.8 | 83.2 / 90.6 | 24.2 / 20.3 | – |
| XLS-R † | 436K | 128 | – | 300M | 39.7 / 30.6 | 29.3 / 22.0 | 66.9 / 87.9 | 55.6 / 85.6 | 28.4 / 22.9 | – |
| 1B | 40.5 / 30.9 | 30.4 / 26.3 | 70.9 / 85.8 | 66.4 / 87.1 | 28.6 / 25.2 | – | ||||
| MMS † | 491K | 1,406 | – | 300M | 33.8 / 30.5 | 28.7 / 24.0 | 62.3 / 84.3 | 71.9 / 74.3 | 31.5 / 30.0 | – |
| 1B | 33.3 / 25.7 | 21.3 / 18.1 | 84.8 / 86.1 | 73.3 / 74.8 | 26.0 / 25.5 | – | ||||
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Category | Markers and scores |
|---|---|
| Exact source text | tx , txt , text , texto , transcription |
| Strong transcription | transcription , orthography , utterance , sentence , clause , intonation unit |
| Weak text | text , word , phrase , line |
| Structural bonus | Top-level tier or transcription-like parent |
| Structural penalty | Other child tier |
| Speaker condition | Non-native ; background |
| Language | ISO 639-3 | Labeled (H) | Unlabeled (H) | Language | ISO 639-3 | Labeled (H) | Unlabeled (H) | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Train | Dev | Test | Train | Dev | Test | ||||||
| Abui | abz | 10.97 | 0.64 | 1.32 | 129.08 | Ainu | ain | 0.00 | 0.00 | 0.00 | 2.78 |
| Ainu (Saru) | aib | 0.00 | 0.00 | 0.00 | 6.28 | Aiome | aki | 0.00 | 0.00 | 0.00 | 0.42 |
| Aisi | mmq | 0.00 | 0.00 | 0.00 | 5.16 | Ajumbu | muc | 0.27 | 0.00 | 0.00 | 62.98 |
| Akan | aka | 0.49 | 0.00 | 0.00 | 1.44 | Akha | ahk | 5.49 | 0.33 | 0.64 | 2.29 |
| Akpes | ibe | 10.80 | 0.63 | 1.25 | 107.75 | Aleut | ale | 0.00 | 0.00 | 0.00 | 263.10 |
| Language | Code | Data (H) | CER | Language | Code | Data (H) | CER | Language | Code | Data (H) | CER | |||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mono | Multi | Mono | Multi | Mono | Multi | |||||||||
| Masela, Central | mxz | 136.8 | 28.39 | 28.31 | Eibela | ail | 8.9 | 10.35 | 32.72 | Apurinã | apu | 3.3 | 49.09 | 55.42 |
| Ju|’hoansi | ktz | 79.9 | 42.93 | 53.06 | Mongolian, Peripheral (Jirim) | mvf | 8.8 | 45.09 | 48.78 | Waimaha | bao | 3.3 | 34.65 | 45.13 |
| Gorwaa | gow | 73.3 | 62.82 | 89.33 | Siona | snn | 8.7 | 34.99 | 32.91 | Fania | fni | 3.2 | 14.92 | 31.82 |
| Tzeltal | tzh | 69.9 | 23.20 | 30.05 | Bastimentos Creole English | – | 8.6 | 55.60 | 49.91 | Kerinci (Koto Keras) | – | 3.2 | 15.22 | 21.52 |
| Yoloxóchitl Mixtec | xty | 66.5 | 8.70 | 16.84 | Anal | anm | 8.5 | 37.23 | 43.47 | Thiek | – | 3.2 | 15.69 | 19.57 |
| Language | Code | Data (H) | Reference | OmniVoice | Language | Code | Data (H) | Reference | OmniVoice | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| (Ground Truth) | (Ground Truth) | ||||||||||||||
| CER | UTMOS | CER | UTMOS | SIM-O | CER | UTMOS | CER | UTMOS | SIM-O | ||||||
| Masela, Central | mxz | 136.8 | 28.39 | 2.660 | 12.87 | 2.786 | 0.769 | Bum | bmv | 5.5 | 10.40 | 1.715 | 16.85 | 2.165 | 0.589 |
| Ju|’hoansi | ktz | 79.9 | 42.93 | 2.072 | 46.39 | 2.635 | 0.526 | Karagas | kim | 5.4 | 29.64 | 1.905 | 12.75 | 2.293 | 0.781 |
| Gorwaa | gow | 73.3 | 62.82 | 2.220 | 70.45 | 2.463 | 0.538 | Dzardzongke | – | 5.4 | 15.70 | 2.404 | 7.95 | 2.796 | 0.712 |
| Tzeltal | tzh | 69.9 | 23.20 | 2.188 | 12.93 | 2.603 | 0.692 | Tinigua | tit | 5.3 | 27.87 | 2.135 | 20.19 | 2.219 | 0.693 |
Low-Resource Language Processing
Low-resource language processing builds and evaluates language technologies for languages with limited data, tools, or digital representation.
See all 565 papers