Emilia2: A Large-Scale Dataset for Controllable Long-Form Speech Generation
Organizations: The Chinese University of Hong Kong, Shenzhen · University of Illinois at Urbana-Champaign
Abstract
Generating long-form speech requires models to keep content, speaker identity, turn-taking, and expressive style consistent over time. Yet most open training corpora contain examples lasting only tens of seconds and open long-form benchmarks cover at most two speakers per recording. We introduce Emilia2, a 229K-hour multilingual dataset for controllable long-form speech generation. From in-the-wild data, Emilia2 constructs short utterances, single-speaker long-form speech, and multi-speaker dialogues, preserving transcripts, speaker identities, pauses, and acoustic transitions. Examples longer than five minutes account for 50.0% of single-speaker long-form hours and 66.8% of dialogue hours. To support expressive control, Emilia2 provides natural language instructions for 4.77M examples and nonverbal-vocalization tags for 933K examples. We also introduce Emilia2-testset, 23 hours of bilingual recordings of up to 30 minutes and four speakers, each speaker paired with reference audio. We train autoregressive (AR) and non-autoregressive (NAR) models on Emilia2. Our 0.6B-backbone NAR model achieves the lowest Chinese CER and English WER on TTSD-Eval (5.27%/9.93%) among compared systems, while our AR model is on par with Qwen3-TTS-12Hz-VD on InstructTTSEval (zh/en: 76.2/78.3 vs. 77.1/77.9). These results validate Emilia2 as a comprehensive data foundation for long-form speech generation and minute-scale expressive control. Demo: https://charlesnii.github.io/Emilia2-demo/.
Figures & tables
| Resource | Hours | Language | Long | Dialogue | Instruction | NV |
|---|---|---|---|---|---|---|
| Emilia ( He et al., 2025 ) | 216,000 | 6 | ✗ | ✗ | ✗ | ✗ |
| MinutesSpeech ( Nishimura et al., 2025 ) | 40,000 | 1 | ✓ | ✗ | ✗ | ✗ |
| OpenDialog ( Zhu et al., 2026a ) | 6,800 | 2 | ✗ | ✓ | ✗ | ✗ |
| DNASpeech ( Cheng et al., 2025 ) | 18 | 1 | ✗ | ✓ † | ✓ | ✗ |
| ParaSpeechCaps ( Diwan et al., 2025 ) | 2,800 | 1 | ✗ | ✗ | ✓ | ✗ |
| CapSpeech ( Wang et al., 2026b ) | 33,600 | 1 | ✗ | ✗ | ✓ | ✓ |
| View | Speech data | Instruction | NV | |||
|---|---|---|---|---|---|---|
| Examples | Audio (h) | Examples | Audio (h) | Examples | Audio (h) | |
| Emilia2-Short | 99,841,007 | 146,325 | 3,994,566 | 5,102 | 474,078 | 854 |
| Emilia2-Long | 4,263,913 | 155,688 | 651,060 | 15,317 | 353,929 | 9,572 |
| Emilia2-Dialogue | 1,538,239 | 96,452 | 120,815 | 4,633 | 104,999 | 4,200 |
| Total | 105,643,159 | 398,467 | 4,766,441 | 25,053 | 933,006 | 14,626 |
| Language | Cases | Single-speaker | Dialogue | Audio (h) | Avg. (min) | Range (min) |
|---|---|---|---|---|---|---|
| Chinese | 59 | 18 | 41 | 11.6 | 11.84 | 0.9–29.3 |
| English | 59 | 24 | 35 | 11.7 | 11.86 | 1.0–30.0 |
| Total | 118 | 42 | 76 | 23.3 | 11.85 | 0.9–30.0 |
| Model | Params. | Chinese | English | ||||
|---|---|---|---|---|---|---|---|
| ACC | SIM | CER | ACC | SIM | WER | ||
| GT | – | 99.33 | 0.88 | 3.00 | 98.38 | 0.85 | 3.00 |
| Higgs Audio v2 | 3B | 75.00 | 0.74 | 75.46 | 72.84 | 0.67 | 81.38 |
| VibeVoice-1.5B | 1.5B | 93.78 | 0.81 | 4.61 | 95.98 | 0.75 | 5.66 |
| MOSS-TTSD | 8B | 99.28 | 0.87 | 2.18 | 99.22 | 0.81 | 5.83 |
| Emilia2-AR | 0.6B | 98.57 | 0.84 | 3.67 | 95.05 | 0.74 | 6.82 |
| Model | TTSD-Eval | Long-TTS-Eval | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Chinese | English | Chinese | English | |||||||
| ACC | SIM | CER | ACC | SIM | WER | CER | SIM | WER | SIM | |
| Higgs Audio v2 | 60.97 | 0.63 | 79.32 | 70.60 | 0.62 | 47.54 | 73.65 | 0.60 | 73.30 | 0.47 |
| VibeVoice-1.5B | 88.25 | 0.73 | 9.21 | 93.58 | 0.68 | 10.18 | 16.98 | 0.67 | 11.20 | 0.59 |
| MOSS-TTSD | 95.36 | 0.80 | 6.08 | 95.00 | 0.74 | 11.09 | 8.22 | 0.74 | 8.54 | 0.63 |
| Emilia2-AR | 92.89 | 0.75 | 6.92 | 92.66 | 0.65 | 12.25 | 14.42 | 0.75 | 17.76 | 0.67 |
| Short-form | Chinese | English | ||||||
|---|---|---|---|---|---|---|---|---|
| Model | APS | DSD | RP | Avg. | APS | DSD | RP | Avg. |
| VoxInstruct | 47.5 | 52.3 | 42.6 | 47.5 | 54.9 | 57.0 | 39.3 | 50.4 |
| FlexiVoice-Base | 56.7 | 59.1 | 59.5 | 58.4 | 63.6 | 75.0 | 60.6 | 66.4 |
| Qwen3-TTS-12Hz-VD | 85.2 | 81.1 | 65.1 | 77.1 | 82.9 | 82.4 | 68.4 | 77.9 |
| Emilia2-AR | 82.0 | 72.5 | 74.0 | 76.2 | 83.0 | 83.5 | 68.5 | 78.3 |
| Emilia2-NAR | 69.2 | 59.8 | 55.6 | 61.6 | 67.4 | 66.5 | 49.8 | 61.2 |
| Short-form | Single-label subset | Multi-label subset | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Chinese | English | Chinese | English | ||||||||
| PCER | CER | SIM | PWER | WER | SIM | PCER | CER | SIM | PWER | WER | SIM | |
| Orpheus-TTS | 88.77 | 11.36 | – | 71.92 | 9.03 | – | 84.85 | 19.83 | – | 71.46 | 8.68 | – |
| SMIIP-NV-CV2 | 75.64 | 8.80 | 0.72 | 56.80 | 17.92 | 0.58 | 77.20 | 10.66 | 0.72 | 54.49 | 20.93 | 0.58 |
| CosyVoice 3 | 57.69 | 3.85 | 0.76 | 62.75 | 7.87 | 0.70 | 61.94 | 4.75 | 0.72 | 57.84 | 6.39 | 0.72 |
| Emilia2-AR | 39.23 | 5.39 | 0.67 | 55.59 | 10.96 | 0.62 | 49.48 | 6.25 | 0.67 | 57.19 | 9.10 | 0.62 |
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Benchmark | Coverage | FireRedLID | Baseline | Baseline Acc. |
|---|---|---|---|---|
| FLEURS test | 82 langs. | 97.18 | SpeechBrain | 92.91 |
| CommonVoice test | 74 langs. | 92.07 | Whisper | 80.81 |
| Chinese dialects | 10+ dialects | 88.47 | Dolphin | 69.01 |
| Language | SpeechBrain Acc. (kept) | FireRedLID Acc. (kept) | Acc. |
|---|---|---|---|
| Bengali (bn) | 83.3 (54) | 92.1 (63) | |
| German (de) | 89.5 (76) | 87.3 (63) | |
| English (en) | 98.9 (93) | 100.0 (95) | |
| Spanish (es) | 71.7 (53) | 91.4 (70) | |
| French (fr) | 88.9 (81) | 90.3 (72) | |
| Japanese (ja) | 87.7 (73) | 95.3 (86) |
| Benchmark | Coverage | Qwen3-ASR-1.7B | Whisper-L3 |
|---|---|---|---|
| FLEURS-en | English | 3.35 | 4.08 |
| FLEURS-zh | Chinese | 2.41 | 4.09 |
| MLS | 8 langs. | 8.55 | 8.62 |
| CommonVoice | 13 langs. | 9.18 | 10.77 |
| FLEURS | 12 langs. | 4.90 | 5.27 |
| FLEURS extended | 30 langs. | 12.60 | 8.16 |
| Method | Acc. | RTF |
|---|---|---|
| WavLM | 0.938 | 0.0028 |
| wav2vec2 | 0.936 | 0.0015 |
| Qwen2-Audio | 0.919 | 0.0077 |
| Method | Acc. | UAR |
|---|---|---|
| FireRedLID Voxlect | 0.757 | 0.743 |
| Voxlect (Whisper-L3) | 0.743 | 0.733 |
| FireRedLID | 0.331 | 0.243 |
| Measure | Acc. | UAR |
|---|---|---|
| Characters/s | 0.894 | 0.937 |
| Phones/s | 0.732 | 0.830 |
| Words/s | 0.732 | 0.819 |
| Set | Lang. | No gate | |||||||
|---|---|---|---|---|---|---|---|---|---|
| WA | UA | WF1 | Cov. | WA | UA | WF1 | |||
| MSP-Podcast | en | 3,256 | 0.49 | 0.45 | 0.45 | 28% | 0.72 | 0.54 | 0.66 |
| MELD | en | 2,608 | 0.53 | 0.39 | 0.53 | 20% | 0.71 | 0.52 | 0.71 |
| M3ED | zh | 4,194 | 0.53 | 0.42 | 0.54 | 25% | 0.73 | 0.59 | 0.73 |
| BIIC-Podcast | zh | 10,149 | 0.57 | 0.37 | 0.56 | 29% | 0.79 | 0.55 | 0.77 |
| Duration | Emilia2-Long | Emilia2-Dialogue | ||||||
|---|---|---|---|---|---|---|---|---|
| Examples | (%) | Hours | (%) | Examples | (%) | Hours | (%) | |
| 30–60 s | 2,337,079 | 54.81 | 27,015 | 17.35 | 557,474 | 36.24 | 6,696 | 6.94 |
| 1–5 min | 1,571,270 | 36.85 | 50,868 | 32.67 | 737,290 | 47.93 | 25,321 | 26.25 |
| 5–10 min | 196,474 | 4.61 | 22,972 | 14.76 | 114,598 | 7.45 | 13,441 | 13.94 |
| 10–15 min | 73,138 | 1.72 | 14,827 | 9.52 | 47,164 | 3.07 | 9,589 | 9.94 |
| 15–20 min | 33,896 | 0.79 | 9,722 | 6.24 | 25,552 | 1.66 | 7,363 | 7.63 |
| Language | Emilia2-Short | Emilia2-Long | Emilia2-Dialogue | |||
|---|---|---|---|---|---|---|
| Hours | Share (%) | Hours | Share (%) | Hours | Share (%) | |
| English | 36,808 | 25.16 | 45,311 | 29.10 | 36,238 | 37.57 |
| Chinese | 36,739 | 25.11 | 54,257 | 34.85 | 17,383 | 18.02 |
| German | 24,646 | 16.84 | 19,703 | 12.66 | 21,701 | 22.50 |
| Japanese | 14,940 | 10.21 | 8,491 | 5.45 | 7,710 | 7.99 |
| Arabic | 8,400 | 5.74 | 7,928 | 5.09 | 3,253 | 3.37 |
| Benchmark | Type | Clips | Avg. dur. | Speaker | Open source | Prompt audio |
|---|---|---|---|---|---|---|
| MinutesSpeech-test [ Nishimura et al., 2025 ] | Long | 1,221 | 0.84 min | 1 | ✗ | ✓ |
| LibriSpeech-Long [ Park et al., 2024b ] | Long | 960 | 3.19 min | 1 | ✓ | ✗ |
| Long-TTS-Eval [ Wang et al., 2026a ] | Long | 694 | 13.4 min † | 1 | ✓ | ✗ |
| MultiDialog [ Park et al., 2024a ] | Dialogue | 831 | 2.46 min | 2 | ✓ | ✗ |
| ZipVoice-Dialog testset [ Zhu et al., 2026a ] | Dialogue | 637 | 0.38 min | 2 | ✓ | ✓ |
| TTSD-Eval [ Zhang et al., 2026b ] | Both | 100 | 10.0 min † | 2 | ✓ | ✓ |
| English speakers | Chinese speakers | ||||||||
| Target duration | 1 | 2 | 3 | 4 | 1 | 2 | 3 | 4 | Total |
| 5 min | 8 | 4 | 2 | 1 | 5 | 6 | 4 | 2 | 32 |
| 5–10 min | 4 | 2 | 4 | 2 | 3 | 4 | 3 | 0 | 22 |
| 10–15 min | 3 | 4 | 4 | 1 | 2 | 4 | 2 | 1 | 21 |
| 15–20 min | 3 | 4 | 1 | 0 | 3 | 2 | 2 | 3 | 18 |
| 20–25 min | 3 | 2 | 1 | 0 | 3 | 4 | 2 | 0 | 15 |
| Stage | Max. duration | Learning rate | Max. token length |
|---|---|---|---|
| Short-form | 30 s | 32,768 | |
| Long-form | 30 min | 32,768 | |
| Dialogue | 30 min | 50,480 | |
| Expressive SFT | 5 min | 50,480 |
| Model | Inference | Chinese | English | Overall N-MOS | ||
| CER | N-MOS | WER | N-MOS | |||
| Ground truth | Recorded | – | 4.55 0.31 | – | 4.55 0.27 | 4.55 0.20 |
| Emilia2-AR | Chunked | 4.36 | 3.30 0.16 | 2.44 | 4.08 0.28 | 3.69 0.24 |
| Whole sequence | 3.67 | 4.05 0.23 | 6.82 | 4.10 0.24 | 4.08 0.16 | |
| Emilia2-NAR | Chunked | 1.73 | 3.90 0.28 | 1.52 | 4.00 0.21 | 3.95 0.17 |
| Whole sequence | 8.62 | 4.05 0.31 | 6.62 | 4.10 0.38 | 4.08 0.24 | |
| Model | English | Chinese | ||
|---|---|---|---|---|
| SIM-o | WER | SIM-o | CER | |
| Ground-truth | 0.73 | 2.14 | 0.76 | 1.25 |
| Higgs Audio v2 | 0.65 | 2.60 | 0.72 | 2.62 |
| VibeVoice-1.5B | 0.69 | 3.04 | 0.74 | 1.16 |
| MOSS-TTSD | 0.68 | 9.83 | 0.76 | 5.84 |
| Emilia2-AR | 0.61 | 1.85 | 0.69 | 1.46 |
| Model | WER | SIM-o |
|---|---|---|
| Higgs Audio v2 | 49.51 | 0.70 |
| VibeVoice-1.5B | 85.41 | 0.75 |
| MOSS-TTSD | 40.25 | 0.80 |
| Emilia2-AR | 7.67 | 0.71 |
| Emilia2-NAR | 10.34 | 0.79 |
| Language | Emilia2-AR | Emilia2-NAR | ||
|---|---|---|---|---|
| WER | SIM-o | WER | SIM-o | |
| Arabic | 7.34 | 0.71 | 13.79 | 0.76 |
| Cantonese | 37.15 | 0.75 | 28.21 | 0.78 |
| Chinese | 3.72 | 0.73 | 1.25 | 0.79 |
| Czech | 11.97 | 0.70 | 18.90 | 0.76 |
| Dutch | 4.47 | 0.69 | 6.91 | 0.72 |
Controllable Speech Generation
Controllable speech generation steers synthesized speech's voice, prosody, emotion, timing, pronunciation, and conversational delivery.
See all 121 papers