cs.CLMar 2, 2026
SaveRO-N3WS: Enhancing Generalization in Low-Resource ASR with Diverse Romanian Speech Benchmarks
Organizations: Department of Computer Science, University of Bucharest
Abstract
We introduce RO-N3WS, a benchmark Romanian speech dataset designed to improve generalization in automatic speech recognition (ASR), particularly in low-resource and out-of-distribution (OOD) conditions. RO-N3WS comprises over 126 hours of transcribed audio collected from broadcast news, literary audiobooks, film dialogue, children's stories, and conversational podcast speech. This diversity enables robust training and fine-tuning across stylistically distinct domains. We evaluate several state-of-the-art ASR systems (Whisper, Wav2Vec 2.0) in both zero-shot and fine-tuned settings, and conduct controlled comparisons using synthetic data generated with expressive TTS models. Our results show that even limited fine-tuning on real speech from RO-N3WS yields substantial WER improvements over zero-shot baselines. We will release all models, scripts, and data splits to support reproducible research in multilingual ASR, domain adaptation, and lightweight deployment.
Figures & tables
| Dataset | Total (h) | RO (h) | RO Spk. | Type |
| Common Voice | 7,335 | 47 | 428 | Read, crowdsourced |
| VoxPopuli | 1,791 | 89 | 18 | Scripted, formal |
| FLEURS | 1,400 | 12 | 19 | Read, aligned |
| SWARA | 21 | 21 | 17 | Read, phonetically balanced |
| RODigits | 38 | 38 | 11 | Read, digits |
| RSC | 100 | 100 | 164 | Read, isolated words |
Table 1: Overview of multilingual and Romanian-specific speech datasets, highlighting Romanian-language coverage, speaker diversity, and recording types.
| Label | Fleurs | Common Voice | Vox Populi | Echo | RO-N3WS | RO-N3WS OOD |
| PERSON | 2.06 | 1.35 | 1.66 | 2.89 | 3.56 | 2.76 |
| LOC | 0.25 | 0.07 | 0.17 | 0.08 | 0.12 | 0.04 |
| NAT_REL_POL | 0.70 | 0.31 | 0.94 | 0.72 | 0.42 | 0.16 |
| PRODUCT | 1.24 | 0.28 | 0.26 | 1.04 | 1.44 | 0.87 |
| DATETIME | 0.93 | 0.45 | 0.49 | 0.83 | 1.46 | 0.50 |
| NUMERIC_VALUE | 0.81 | 0.45 | 0.67 | 1.23 | 1.61 | 0.46 |
Table 2: Named entity density (entities per 100 tokens) across Romanian speech datasets. RO-N3WS exhibits the highest density overall, highlighting its lexical richness.
| Dataset | Mean Pitch (Hz) | Std. Pitch (Hz) | Pitch Range (Hz) | Mean Intensity (dB) |
| FLEURS | 172.01 | 60.00 | 394.19 | 54.37 |
| Common Voice | 151.45 | 37.21 | 212.08 | 49.92 |
| Vox Populi | 197.99 | 36.49 | 304.07 | 61.45 |
| Echo | 175.10 | 46.67 | 341.19 | 50.95 |
| RO-N3WS | 182.19 | 39.33 | 266.32 | 63.65 |
| RO-N3WS-OOD | 197.72 | 71.92 | 370.07 | 58.19 |
Table 3: Prosodic statistics across Romanian ASR datasets.
| Split / Source | # Files | Total Length (s) | Min (s) | Max (s) | Avg (s) | Std Dev (s) |
| In-domain (News) | ||||||
| ProTV news | 32,319 | 196,405 | 3.00 | 11.98 | 6.08 | 1.57 |
| Observator news | 30,073 | 182,423 | 1.72 | 12.00 | 6.07 | 1.57 |
| All news | 62,392 | 378,828 | 1.72 | 12.00 | 6.07 | 1.57 |
| Out-of-Distribution (OOD) | ||||||
| Audio-books | 2,475 | 15,422 | 1.20 | 10.14 | 6.23 | 1.37 |
Table 4: Detailed statistics for the audio files contained in our RO-N3WS dataset.
| Transcription (Romanian) | WER |
| Ground-truth annotation: | |
| de la întâi ianuarie salariul minim va fi trei mii de lei brut iar în mână un angajat ar urma să primească o mie opt sute s , aizeci s , i trei de lei ("from the first of January, the minimum salary will be three thousand gross lei, and in hand, an employee would receive one thousand eight hundred sixty-three lei") | 0.000 |
| Vatis output: | |
| De la 1 ianuarie, salariul minim va fi 3.000 lei brut, iar în mână un angajat ar urma să primească 1.863 lei. | 0.387 |
| Microsoft Transcribe output: | |
| De la 1 ianuarie, salariul minim va fi 3.000 RON brut, iar în mână un angajat ar urma să primească 1.863 RON. | 0.451 |
Table 5: Examples where WER is high due to formatting differences, despite semantic equivalence between output and reference. Outputs from commercial ASR APIs are post-processed to remove punctuation, normalize capitalization, and standardize spacing before WER computation.
| Model | In-Domain (RO-N3WS) | Out-of-Distribution (OOD) | ||||
| ProTV | Antena1 | Audiobooks | Films | Stories | Podcasts | |
| W2V2 | 27.7 | 25.6 | 40.8 | 75.4 | 54.1 | 36.3 |
| Whisp-S | 31.6 | 26.8 | 40.0 | 60.0 | 41.1 | 31.9 |
| Whisp-L | 12.3 | 5.9 | 14.8 | 27.3 | 10.9 | 11.7 |
| Whisp-S + Echo | 9.4 | 10.1 | 18.8 | 54.1 | 21.0 | 21.6 |
| Microsoft Transcribe | 2.9 | 4.8 | 10.6 | 31.1 | 17.6 | 11.5 |
Table 6: Zero-shot Word Error Rate (WER %) on in-domain and OOD test sets. Lower is better.
| Model | In-Domain (RO-N3WS) | Out-of-Distribution (OOD) | ||||
| ProTV | Antena1 | Audiobooks | Films | Stories | Podcasts | |
| W2V2 + RO-N3WS | 9.8 | 12.1 | 25.6 | 62.4 | 33.6 | 22.8 |
| W2V2 + ProTV | 10.5 | 17.5 | 31.8 | 69.6 | 40.5 | 30.6 |
| W2V2 + Antena1 | 14.5 | 13.4 | 28.2 | 62.3 | 36.2 | 23.8 |
| Whisp-S + RO-N3WS | 4.1 | 6.4 | 18.9 | 43.3 | 21.1 | 14.1 |
| Whisp-S + ProTV | 5.8 | 11.1 | 27.5 | 59.6 | 33.6 | 30.6 |
Table 7: WER (%) of ASR models fine-tuned on RO-N3WS and its subsets. Lower is better.
| Model | In-Domain (RO-N3WS) | Out-of-Distribution (OOD) | ||||
| ProTV | Antena1 | Audiobooks | Films | Stories | Podcasts | |
| Zero-shot Whisp-S | 31.6 | 26.8 | 40.0 | 60.0 | 41.1 | 31.9 |
| Synthetic only (TTS) | 24.3 | 22.5 | 34.2 | 58.5 | 39.5 | 33.0 |
| Mixed (50% TTS) | 19.0 | 20.5 | 33.8 | 55.7 | 36.7 | 31.5 |
| Natural only (RO-N3WS) | 16.6 | 18.1 | 31.0 | 58.1 | 33.3 | 27.1 |
Table 8: WER (%) for models fine-tuned on natural, synthetic, and mixed subsets. All use Whisper Small.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 1: Recording-duration histograms (in seconds) of collected audio files from ProTV News (left) and Observator News (right).
Figure 2: Recording-duration histograms (in seconds) for out-of-distribution subsets: audiobooks, Romanian films, children’s stories and podcasts.
| Whisper annotation (Romanian) | Ground-truth annotation (Romanian) | English translation (ground-truth) |
| Poli t , i s , tii din Cluj au dat de urma celor care în urmă cu o lună au furat din casieria Universită t , ii de S , tiin t , e Agricole 90.000 de lei | poli t , i s , tii din cluj au dat de urma celor care în urmă cu o lună au furat din casieria universită t , ii de s , tiin t , e agricole nouăzeci de mii de lei | police in cluj tracked down those who a month ago stole ninety thousand lei from the university of agricultural sciences. |
| După ce au sequestrat s , i legat bine pe cei doi proprietari în vârstă, ho t , ii s , ase la număr | după ce i-au sechestrat s , i legat bine pe cei doi proprietari în vârstă ho t , ii s , ase la număr | after tying up the two elderly homeowners the six robbers… |
| Oamenii dintr-un oras din sudul Italii au avut ocazia sa asiste la filmarile pentru cea mai noua pelicula din seria James Bond | oamenii dintr-un ora s , din sudul italiei au avut ocazia să asiste la filmările pentru cea mai nouă peliculă din seria james bond | people in a town in southern italy had the chance to witness the filming of the latest james bond movie. |
Table 9: Examples of Whisper-generated transcripts and their corrected annotations. The English translation corresponds to the ground-truth version.
| Hyper-parameter | Whisper-small (ProTV / Antena 1) | Whisper-small (RO-N3WS) | Whisper-large (RO-N3WS) |
| Per-device batch size | 16 | 32 | 32 |
| Gradient-accumulation steps | 1 | 2 | 2 |
| Initial learning rate | |||
| LR scheduler | Linear | Linear | Linear |
| Warm-up steps | 300 | 500 | 3 000 |
| Total training steps | 5 000 | 20 000 | 20 000 |
Table 10: Key hyper-parameters for the Whisper configurations.
| Hyper-parameter | Value |
| Per-device batch size | 16 |
| Gradient-accumulation steps | 2 |
| Initial learning rate | |
| Weight decay | 0.01 |
| LR scheduler | Linear (default) |
| Warm-up steps | 500 |
Table 11: Key hyper-parameters for fine-tuning Wav2Vec 2.0
Figure 3: Learning curves on the ProTV and Antena 1 test sets. WER is reported after fine-tuning Wav2Vec 2.0 and Whisper Small on 5, 10, and the full 17 training chunks of either the same bulletin or the other bulletin. Best viewed in color.