ELARSpeech: A 30K-Hour Dataset Across 675 Underrepresented Languages for Long-Tail Multilingual Speech Technologies

Sangmin Lee; Yejin Jeon; David Ifeoluwa Adelani

cs.CLOct 14, 2026

ELARSpeech: A 30K-Hour Dataset Across 675 Underrepresented Languages for Long-Tail Multilingual Speech Technologies

Authors: Sangmin Lee, Yejin Jeon, David Ifeoluwa Adelani

Organizations: Yonsei University · McGill University · Mila - Quebec AI Institute · Canada CIFAR AI Chair

Abstract

Massively multilingual speech modeling has made remarkable progress in recent years, yet existing training resources and benchmarks still underrepresent much of the linguistic long tail. To address this gap, we introduce ELARSpeech, a large-scale multilingual speech resource comprising 30,202 hours of speech across 675 underrepresented languages, curated from the Endangered Languages Archive. Combining human-annotated and unlabeled recordings, the corpus spans diverse recording conditions, domains, and language families. Beyond corpus construction, our experiments show how ELARSpeech supports extending ASR and TTS capabilities to diverse underrepresented languages, with further benefits from pseudo-labeling unlabeled data. At the same time, our speech representation analysis reveals remaining limitations in transferring existing multilingual models to the linguistic long tail. Together, ELARSpeech provides both a resource and a testbed for extending multilingual speech technology further into the linguistic long tail. We publicly release the dataset, all trained ASR and TTS models, and the complete data construction pipeline to facilitate research on long-tail multilingual speech technologies.

Figures & tables

Dataset	Labeled	Content	# Langs.	Size (H)
Closed-Source Datasets
Google USM	$\times$	Variety	568	12.1M
MMS-Unlab	$\times$	Bible	2809	8K
Whisper	$\checkmark$	Variety	98	680K
MMS-Lab		Bible	1107	45K
ALLASR		Variety	1690	121K

Table 1: Comparison of existing low-resource multilingual speech datasets. "

\triangle

" indicates the mixture across labeled, pseudo-labeled, and unlabeled data.

Panel of Figure 1 — Figure 1: Geographic, domain, and language family distributions of ELARSpeech.

Type	Split	# Lang.	# Utt. (M)	Duration (H)
Labeled	Train	409	3.09	2367.08
	Dev	233	0.17	132.77
	Test	233	0.35	266.14
	Multi	82	0.04	24.82
	Total	443	3.64	2790.81
Unlabeled	Train	592	19.14	26763.37

Table 2: Dataset statistics of ELARSpeech. “Multi” denotes recordings categorized as multilingual.

Languages with ISO 639-3 Codes							Languages without ISO 639-3 Codes
Language	Code	Data (H)	Monolingual			Multilingual	Language	Code	Data (H)	Monolingual			Multilingual
			OmniASR		Whisper	Whisper				OmniASR		Whisper	Whisper
			300M	1B	1.5B	1.5B				300M	1B	1.5B	1.5B
Masela, Central	mxz	136.8	28.90	26.22	28.39	28.31	Jitotoltec	–	18.6	15.16	15.25	13.31	20.84
Ju\|’hoansi	ktz	79.9	41.89	41.71	42.93	53.06	Zok	–	18.4	35.04	36.95	38.35	47.55
Yoloxóchitl Mixtec	xty	66.5	8.76	8.58	8.70	16.84	Sri Lankan Portuguese Creole	–	17.2	18.78	17.86	16.89	22.81

Table 3: Monolingual and multilingual ASR performance across 30 pivot languages of ELARSpeech. Languages are sorted by data duration within each category, and we report CER (%) for every result. Best and second-best results are shown in bold and underlined, respectively. Macro CER is averaged across languages, while win rate denotes the proportion of languages for which each model achieves the lowest CER.

Panel of Figure 2 — Figure 2: ASR performance with increasing amounts of pseudo-labeled ELARSpeech-U.

CommonVoice 26 (CV26)
Language (Code)	Test	CER $\downarrow$
Language (Code)	Test	Original	+L	+LU
Dawoodi (dmk)	ELAR	93.99	22.68	22.72
Dawoodi (dmk)	CV26	4.47	4.81	5.58
Imbabura Highland Quichua (qvi)	ELAR	47.48	7.13	6.71
Imbabura Highland Quichua (qvi)	CV26	1.28	1.12	1.10

Table 4: Cross-domain ASR performance on ELARSpeech and external test sets: CV26 and INEL.

Language	Code	Data (H)	Reference		Monolingual		Multilingual
			(Ground Truth)		OmniVoice		OmniVoice
			CER $\downarrow$	MOS $\uparrow$	CER $\downarrow$	MOS $\uparrow$	CER $\downarrow$	MOS $\uparrow$
Languages with ISO 639-3 Codes
C. Masela	mxz	124.9	28.39	4.04	14.71	3.88	12.87	3.79
Y. Mixtec	xty	64.8	8.70	4.12	4.75	3.58	4.83	3.82
Tzeltal	tzh	19.8	23.20	3.76	12.58	3.33	12.93	3.45

Table 5: Monolingual and multilingual TTS performance on pivot languages with at least 5 hours of training data, with MOS ratings from non-native listeners. Multi-word language names are abbreviated.

Panel of Figure 3 — Figure 3: TTS performance with increasing amounts of pseudo-labeled ELARSpeech-U.

Model	Data (H)	# Langs.	# ELAR	# Params.	Mono. ASR	Multi. ASR	Multi. LID	Multi. ASR + LID		Score
Model	Data (H)	# Langs.	# ELAR	# Params.	CER $\downarrow$	CER $\downarrow$	Acc. $\uparrow$	Acc. $\uparrow$	CER $\downarrow$	Score
w2v-BERT 2.0 †	4.5M	143	–	600M	41.0 / 29.2	24.6 / 20.3	70.0 / 86.8	83.2 / 90.6	24.2 / 20.3	–
XLS-R †	436K	128	–	300M	39.7 / 30.6	29.3 / 22.0	66.9 / 87.9	55.6 / 85.6	28.4 / 22.9	–
XLS-R †	436K	128	–	1B	40.5 / 30.9	30.4 / 26.3	70.9 / 85.8	66.4 / 87.1	28.6 / 25.2	–
MMS †	491K	1,406	–	300M	33.8 / 30.5	28.7 / 24.0	62.3 / 84.3	71.9 / 74.3	31.5 / 30.0	–
MMS †	491K	1,406	–	1B	33.3 / 25.7	21.3 / 18.1	84.8 / 86.1	73.3 / 74.8	26.0 / 25.5	–

Table 6: ML-SUPERB-style evaluation on ELARSpeech. Results are reported under 10-minute / 1-hour supervision. Data and # Langs. denote the amount and reported language coverage of SSL pretraining data. # ELAR denotes the number of languages in each model’s publicly available language inventory that overlap with ELARSpeech, with the percentage relative to its reported language coverage shown in parentheses. Gray italicized rows (

\dagger

) show the original (non-long-tail) ML-SUPERB results reported in Chen et al. (2024) for reference.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Language family distribution from 78 unique language families of ELARSpeech.

Category	Markers and scores
Exact source text	tx , txt , text , texto , transcription $(+90)$
Strong transcription	transcription , orthography , utterance , sentence , clause , intonation unit $(+80)$
Weak text	text , word , phrase , line $(+35)$
Structural bonus	Top-level tier or transcription-like parent $(+10)$
Structural penalty	Other child tier $(-5)$
Speaker condition	Non-native $(-15)$ ; background $(-30)$

Appendix

Algorithm 1 Automatic Transcription Tier Selection and Heuristic Scoring Rules

Figure 5: Instruction page provided to participants.

Language	ISO 639-3	Labeled (H)			Unlabeled (H)	Language	ISO 639-3	Labeled (H)			Unlabeled (H)
	ISO 639-3	Train	Dev	Test	Unlabeled (H)		ISO 639-3	Train	Dev	Test	Unlabeled (H)
Abui	abz	10.97	0.64	1.32	129.08	Ainu	ain	0.00	0.00	0.00	2.78
Ainu (Saru)	aib	0.00	0.00	0.00	6.28	Aiome	aki	0.00	0.00	0.00	0.42
Aisi	mmq	0.00	0.00	0.00	5.16	Ajumbu	muc	0.27	0.00	0.00	62.98
Akan	aka	0.49	0.00	0.00	1.44	Akha	ahk	5.49	0.33	0.64	2.29
Akpes	ibe	10.80	0.63	1.25	107.75	Aleut	ale	0.00	0.00	0.00	263.10

Appendix

Table 7: Per-language hours of labeled and unlabeled speech in ELARSpeech. Labeled data are divided into train, development, and test splits, and durations below 0.01 hours are denoted as 0.00. Groups of languages within a family are denoted as “Family” in parentheses, while dialects are denoted by their dialect names in parentheses (e.g., Saru). Multilingual subsets are excluded since their durations cannot be reliably attributed to individual languages.

Language	Code	Data (H)	CER $\downarrow$		Language	Code	Data (H)	CER $\downarrow$		Language	Code	Data (H)	CER $\downarrow$
Language	Code	Data (H)	Mono	Multi	Language	Code	Data (H)	Mono	Multi	Language	Code	Data (H)	Mono	Multi
Masela, Central	mxz	136.8	28.39	28.31	Eibela	ail	8.9	10.35	32.72	Apurinã	apu	3.3	49.09	55.42
Ju\|’hoansi	ktz	79.9	42.93	53.06	Mongolian, Peripheral (Jirim)	mvf	8.8	45.09	48.78	Waimaha	bao	3.3	34.65	45.13
Gorwaa	gow	73.3	62.82	89.33	Siona	snn	8.7	34.99	32.91	Fania	fni	3.2	14.92	31.82
Tzeltal	tzh	69.9	23.20	30.05	Bastimentos Creole English	–	8.6	55.60	49.91	Kerinci (Koto Keras)	–	3.2	15.22	21.52
Yoloxóchitl Mixtec	xty	66.5	8.70	16.84	Anal	anm	8.5	37.23	43.47	Thiek	–	3.2	15.69	19.57

Appendix

Table 8: Character Error Rate (CER) of Whisper on the ELARSpeech test set across all 233 languages. Mono denotes a monolingual Whisper model fine-tuned separately for each language; Multi denotes a single multilingual Whisper model fine-tuned on all languages. Both are evaluated on the same test utterances. Languages are sorted by training-set duration and read down each column. Data (H) denotes the amount of the human-annotated training data. Languages without an ISO 639-3 code are marked “–”.

Language	Code	Data (H)	Reference		OmniVoice			Language	Code	Data (H)	Reference		OmniVoice
			(Ground Truth)		OmniVoice						(Ground Truth)		OmniVoice
			CER $\downarrow$	UTMOS $\uparrow$	CER $\downarrow$	UTMOS $\uparrow$	SIM-O $\uparrow$				CER $\downarrow$	UTMOS $\uparrow$	CER $\downarrow$	UTMOS $\uparrow$	SIM-O $\uparrow$
Masela, Central	mxz	136.8	28.39	2.660	12.87	2.786	0.769	Bum	bmv	5.5	10.40	1.715	16.85	2.165	0.589
Ju\|’hoansi	ktz	79.9	42.93	2.072	46.39	2.635	0.526	Karagas	kim	5.4	29.64	1.905	12.75	2.293	0.781
Gorwaa	gow	73.3	62.82	2.220	70.45	2.463	0.538	Dzardzongke	–	5.4	15.70	2.404	7.95	2.796	0.712
Tzeltal	tzh	69.9	23.20	2.188	12.93	2.603	0.692	Tinigua	tit	5.3	27.87	2.135	20.19	2.219	0.693

Appendix

Table 9: CER, UTMOS, and speaker similarity (SIM-O) of the multilingual OmniVoice model on the ELARSpeech test set. UTMOS and OmniVoice results for Mezquital Otomi and Totonac, Zongozotla are unavailable because no test samples remain after SNR/C50 filtering. Languages are sorted by training-set duration and read down each column. Data (H) denotes the hours of human-annotated training data. Languages without an ISO 639-3 code are marked “–”. Reference CER and UTMOS were measured with a monolingual Whisper model and UTMOS-v2, respectively, on ground-truth speech.

Low-Resource Language Processing

Low-resource language processing builds and evaluates language technologies for languages with limited data, tools, or digital representation.

See all 565 papers

WorldSpeech: A Multilingual Speech Corpus from Around the World
May 9, 2026Antonis Asonitis, Luca A. Lanzendörfer, Frédéric Berdoz +1Automatic Speech Recognition Low-Resource Speech Recognition
ETH Zurich
PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialects
May 31, 2026Sicheng Yang, Shulan Ruan, Shiwei Wu +4Audio-Language Model Evaluation Audio Understanding
Shenzhen International Graduate School, Tsinghua University Shenzhen, China · Department of Electronic Engineering, Tsinghua University Beijing, China · JD AI Research Beijing, China
GigaAM Multilingual: Foundation Model for Underrepresented Languages
Jul 11, 2026Andrei Kuzmenko, Alexandr Maximenko, Aleksandr Kutsakov +5Speech Foundation Models Speech Processing
SaluteDevices, Russia

May 9, 2026J/K move · Enter open · S save

WorldSpeech: A Multilingual Speech Corpus from Around the World

Antonis Asonitis, Luca A. Lanzendörfer, Frédéric Berdoz, Roger Wattenhofer

ETH Zurich

Automatic Speech Recognition Low-Resource Speech Recognition Multilingual ASR Speech Corpus Construction

Open paper AskPDF arXiv

Automatic speech recognition (ASR) performs well for high-resource languages with abundant paired audio-transcript data, but its accuracy degrades sharply for most languages due to limited publicly available aligned data. To this end, we introduce WorldSpeech, a 24 kHz multilingual speech corpus comprising 65k hours of aligned audio-transcript data across 76 languages, collected from diverse public sources including parliamentary proceedings, international broadcasts, and public-domain audiobooks. For 37 languages, WorldSpeech provides more than 200 hours of aligned speech, with 28 exceeding 500 hours and 24 surpassing 1k hours. Fine-tuning existing ASR models on WorldSpeech results in an average relative Word-Error-Rate reduction of 63.5% across 11 typologically diverse languages.

Abstract

Low-Resource Language Processing

Explore similar work

WorldSpeech: A Multilingual Speech Corpus from Around the World

PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialects

GigaAM Multilingual: Foundation Model for Underrepresented Languages