We introduce RO-N3WS, a benchmark Romanian speech dataset designed to improve generalization in automatic speech recognition (ASR), particularly in low-resource and out-of-distribution (OOD) conditions. RO-N3WS comprises over 126 hours of transcribed audio collected from broadcast news, literary audiobooks, film dialogue, children's stories, and conversational podcast speech. This diversity enables robust training and fine-tuning across stylistically distinct domains. We evaluate several state-of-the-art ASR systems (Whisper, Wav2Vec 2.0) in both zero-shot and fine-tuned settings, and conduct controlled comparisons using synthetic data generated with expressive TTS models. Our results show that even limited fine-tuning on real speech from RO-N3WS yields substantial WER improvements over zero-shot baselines. We will release all models, scripts, and data splits to support reproducible research in multilingual ASR, domain adaptation, and lightweight deployment.
Figures & tables
Dataset
Total (h)
RO (h)
RO Spk.
Type
Common Voice
7,335
47
428
Read, crowdsourced
VoxPopuli
1,791
89
18
Scripted, formal
FLEURS
1,400
12
19
Read, aligned
SWARA
21
21
17
Read, phonetically balanced
RODigits
38
38
11
Read, digits
RSC
100
100
164
Read, isolated words
Table 1: Overview of multilingual and Romanian-specific speech datasets, highlighting Romanian-language coverage, speaker diversity, and recording types.
Label
Fleurs
Common Voice
Vox Populi
Echo
RO-N3WS
RO-N3WS OOD
PERSON
2.06
1.35
1.66
2.89
3.56
2.76
LOC
0.25
0.07
0.17
0.08
0.12
0.04
NAT_REL_POL
0.70
0.31
0.94
0.72
0.42
0.16
PRODUCT
1.24
0.28
0.26
1.04
1.44
0.87
DATETIME
0.93
0.45
0.49
0.83
1.46
0.50
NUMERIC_VALUE
0.81
0.45
0.67
1.23
1.61
0.46
Table 2: Named entity density (entities per 100 tokens) across Romanian speech datasets. RO-N3WS exhibits the highest density overall, highlighting its lexical richness.
Dataset
Mean Pitch (Hz)
Std. Pitch (Hz)
Pitch Range (Hz)
Mean Intensity (dB)
FLEURS
172.01
60.00
394.19
54.37
Common Voice
151.45
37.21
212.08
49.92
Vox Populi
197.99
36.49
304.07
61.45
Echo
175.10
46.67
341.19
50.95
RO-N3WS
182.19
39.33
266.32
63.65
RO-N3WS-OOD
197.72
71.92
370.07
58.19
Table 3: Prosodic statistics across Romanian ASR datasets.
Split / Source
# Files
Total Length (s)
Min (s)
Max (s)
Avg (s)
Std Dev (s)
In-domain (News)
ProTV news
32,319
196,405
3.00
11.98
6.08
1.57
Observator news
30,073
182,423
1.72
12.00
6.07
1.57
All news
62,392
378,828
1.72
12.00
6.07
1.57
Out-of-Distribution (OOD)
Audio-books
2,475
15,422
1.20
10.14
6.23
1.37
Table 4: Detailed statistics for the audio files contained in our RO-N3WS dataset.
Transcription (Romanian)
WER
Ground-truth annotation:
de la întâi ianuarie salariul minim va fi trei mii de lei brut iar în mână un angajat ar urma să primească o mie opt sute s , aizeci s , i trei de lei ("from the first of January, the minimum salary will be three thousand gross lei, and in hand, an employee would receive one thousand eight hundred sixty-three lei")
0.000
Vatis output:
De la 1 ianuarie, salariul minim va fi 3.000 lei brut, iar în mână un angajat ar urma să primească 1.863 lei.
0.387
Microsoft Transcribe output:
De la 1 ianuarie, salariul minim va fi 3.000 RON brut, iar în mână un angajat ar urma să primească 1.863 RON.
0.451
Table 5: Examples where WER is high due to formatting differences, despite semantic equivalence between output and reference. Outputs from commercial ASR APIs are post-processed to remove punctuation, normalize capitalization, and standardize spacing before WER computation.
Model
In-Domain (RO-N3WS)
Out-of-Distribution (OOD)
ProTV
Antena1
Audiobooks
Films
Stories
Podcasts
W2V2
27.7
25.6
40.8
75.4
54.1
36.3
Whisp-S
31.6
26.8
40.0
60.0
41.1
31.9
Whisp-L
12.3
5.9
14.8
27.3
10.9
11.7
Whisp-S + Echo
9.4
10.1
18.8
54.1
21.0
21.6
Microsoft Transcribe
2.9
4.8
10.6
31.1
17.6
11.5
Table 6: Zero-shot Word Error Rate (WER %) on in-domain and OOD test sets. Lower is better.
Model
In-Domain (RO-N3WS)
Out-of-Distribution (OOD)
ProTV
Antena1
Audiobooks
Films
Stories
Podcasts
W2V2 + RO-N3WS
9.8
12.1
25.6
62.4
33.6
22.8
W2V2 + ProTV
10.5
17.5
31.8
69.6
40.5
30.6
W2V2 + Antena1
14.5
13.4
28.2
62.3
36.2
23.8
Whisp-S + RO-N3WS
4.1
6.4
18.9
43.3
21.1
14.1
Whisp-S + ProTV
5.8
11.1
27.5
59.6
33.6
30.6
Table 7: WER (%) of ASR models fine-tuned on RO-N3WS and its subsets. Lower is better.
Model
In-Domain (RO-N3WS)
Out-of-Distribution (OOD)
ProTV
Antena1
Audiobooks
Films
Stories
Podcasts
Zero-shot Whisp-S
31.6
26.8
40.0
60.0
41.1
31.9
Synthetic only (TTS)
24.3
22.5
34.2
58.5
39.5
33.0
Mixed (50% TTS)
19.0
20.5
33.8
55.7
36.7
31.5
Natural only (RO-N3WS)
16.6
18.1
31.0
58.1
33.3
27.1
Table 8: WER (%) for models fine-tuned on natural, synthetic, and mixed subsets. All use Whisper Small.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 1: Recording-duration histograms (in seconds) of collected audio files from ProTV News (left) and Observator News (right).
Figure 2: Recording-duration histograms (in seconds) for out-of-distribution subsets: audiobooks, Romanian films, children’s stories and podcasts.
Whisper annotation (Romanian)
Ground-truth annotation (Romanian)
English translation (ground-truth)
Poli t , i s , tii din Cluj au dat de urma celor care în urmă cu o lună au furat din casieria Universită t , ii de S , tiin t , e Agricole 90.000 de lei
poli t , i s , tii din cluj au dat de urma celor care în urmă cu o lună au furat din casieria universită t , ii de s , tiin t , e agricole nouăzeci de mii de lei
police in cluj tracked down those who a month ago stole ninety thousand lei from the university of agricultural sciences.
După ce au sequestrat s , i legat bine pe cei doi proprietari în vârstă, ho t , ii s , ase la număr
după ce i-au sechestrat s , i legat bine pe cei doi proprietari în vârstă ho t , ii s , ase la număr
after tying up the two elderly homeowners the six robbers…
Oamenii dintr-un oras din sudul Italii au avut ocazia sa asiste la filmarile pentru cea mai noua pelicula din seria James Bond
oamenii dintr-un ora s , din sudul italiei au avut ocazia să asiste la filmările pentru cea mai nouă peliculă din seria james bond
people in a town in southern italy had the chance to witness the filming of the latest james bond movie.
Appendix
Table 9: Examples of Whisper-generated transcripts and their corrected annotations. The English translation corresponds to the ground-truth version.
Hyper-parameter
Whisper-small (ProTV / Antena 1)
Whisper-small (RO-N3WS)
Whisper-large (RO-N3WS)
Per-device batch size
16
32
32
Gradient-accumulation steps
1
2
2
Initial learning rate
1×10−5
5×10−5
2×10−5
LR scheduler
Linear
Linear
Linear
Warm-up steps
300
500
3 000
Total training steps
5 000
20 000
20 000
Appendix
Table 10: Key hyper-parameters for the Whisper configurations.
Hyper-parameter
Value
Per-device batch size
16
Gradient-accumulation steps
2
Initial learning rate
5×10−5
Weight decay
0.01
LR scheduler
Linear (default)
Warm-up steps
500
Appendix
Table 11: Key hyper-parameters for fine-tuning Wav2Vec 2.0
Figure 3: Learning curves on the ProTV and Antena 1 test sets. WER is reported after fine-tuning Wav2Vec 2.0 and Whisper Small on 5, 10, and the full 17 training chunks of either the same bulletin or the other bulletin. Best viewed in color.
Large language models (LLMs) have driven substantial advances in speech language models (SpeechLMs), yielding strong performance in automatic speech recognition (ASR) under high-resource conditions. However, existing benchmarks predominantly focus on high-resource languages, leaving the ASR behavior of SpeechLMs in low-resource languages insufficiently understood. This gap is critical, as practical ASR systems must reliably support low-resource languages and generalize across diverse language families, and it directly hinders the deployment of SpeechLM-based ASR in real-world multilingual scenarios. As a result, it is essential to evaluate SpeechLMs on low-resource languages to ensure their generalizability across different language families. To address this problem, we propose LoASR-Bench, a comprehensive benchmark designed to evaluate low-resource automatic speech recognition (ASR) of the latest SpeechLMs across diverse language families. LoASR-Bench comprises 25 languages from 9 language families, featuring both Latin and non-Latin scripts, enabling cross-linguistic and cross-script assessment of ASR performance of current SpeechLMs. Experimental results highlight the limitations of the latest SpeechLMs in handling real-world low-resource languages.
Jianan Chen, Xiaoxue Gao, Tatsuya Kawahara +1
Kyoto University, Kyoto, Japan · Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore
Automatic speech recognition (ASR) performs well for high-resource languages with abundant paired audio-transcript data, but its accuracy degrades sharply for most languages due to limited publicly available aligned data. To this end, we introduce WorldSpeech, a 24 kHz multilingual speech corpus comprising 65k hours of aligned audio-transcript data across 76 languages, collected from diverse public sources including parliamentary proceedings, international broadcasts, and public-domain audiobooks. For 37 languages, WorldSpeech provides more than 200 hours of aligned speech, with 28 exceeding 500 hours and 24 surpassing 1k hours. Fine-tuning existing ASR models on WorldSpeech results in an average relative Word-Error-Rate reduction of 63.5% across 11 typologically diverse languages.
Antonis Asonitis, Luca A. Lanzendörfer, Frédéric Berdoz +1
Automated transcription of parliamentary proceedings faces significant hurdles due to demographic bias, dialectal variation, and technical artifacts such as utterance truncation during segmentation. This paper introduces the ROManian PARliamentary Speech Corpus (ROMPAR) dataset, a 17.80-hour corpus of Romanian and Moldavian parliamentary speech, featuring double-annotated ground truth and explicit labels for reconstructed word fragments. To build a robust ASR system, we propose a multi-task adversarial training framework that enforces demographic invariance across age, gender, and dialect. We address the inherent instability of adversarial objectives in generative architectures by introducing an exponential decay mechanism for the adversarial coefficients. Furthermore, we implement an LLM-guided decoding strategy with position-dependent weighting to facilitate morphological completion of truncated terminal words. Our results demonstrate that the proposed framework significantly reduces WER and achieves an F1-score of 96.6% in morphological reconstruction.