OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models

Han Zhu; Lingxuan Ye; Wei Kang; Zengwei Yao; Liyong Guo; Fangjun Kuang; Zhifeng Han; Weiji Zhuang

cs.CLApr 1, 2026

OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models

Authors: Han Zhu, Lingxuan Ye, Wei Kang, Zengwei Yao, Liyong Guo, Fangjun Kuang, Zhifeng Han, Weiji Zhuang,

Organizations: Xiaomi Corp., China

Abstract

We present OmniVoice, a massively multilingual zero-shot text-to-speech (TTS) model that scales to over 600 languages. At its core is a novel diffusion language model-style discrete non-autoregressive (NAR) architecture. Unlike conventional discrete NAR models that suffer from performance bottlenecks in complex two-stage (text-to-semantic-to-acoustic) pipelines, OmniVoice directly maps text to multi-codebook acoustic tokens. This simplified approach is facilitated by two key technical innovations: (1) a full-codebook random masking strategy for efficient training, and (2) initialization from a pre-trained LLM to ensure superior intelligibility. By leveraging a 581k-hour multilingual dataset curated entirely from open-source data, OmniVoice achieves the broadest language coverage to date and delivers state-of-the-art performance across Chinese, English, and diverse multilingual benchmarks. Our code and pre-trained models are publicly available at https://github.com/k2-fsa/OmniVoice.

Figures & tables

Figure 1: Illustration of OmniVoice architecture.

Panel of Figure 2 — Figure 2: Comparison of per-layer masking and full-codebook random masking. The x-axis denotes the time dimension, the y-axis the codebook dimension, and white blocks indicate masked tokens.

Figure 3: Statistics of the multilingual training dataset.

Model	Params.	Training Data (hours)	LibriSpeech-PC test-clean
Model	Params.	Training Data (hours)	SIM-o $\uparrow$	WER $\downarrow$	UTMOS $\uparrow$
Ground-truth	-	-	0.690	1.87	4.10
AR Models
IndexTTS2	1.7B	55k Multi.	0.700	2.35	4.06
CosyVoice3	1.1B	1000k Multi.	0.694	1.59	4.28
VoxCPM	0.7B	1800k Multi.	0.717	1.74	4.18

Table 1: Objective evaluation results on Chinese and English test sets. Baseline results are obtained using official checkpoints. Params. (Parameters) denotes the total parameter size of voice cloning TTS systems (including audio tokenizer, vocoder, and other related components). Best results are highlighted in bold . Top: Results on LibriSpeech-PC. Bottom: Results on Seed-TTS test sets.

Model	CMOS $\uparrow$	SMOS $\uparrow$
Ground-truth	0.00	3.02 $\pm$ 0.20
Qwen3-TTS	0.40 $\pm$ 0.16	3.65 $\pm$ 0.18
ZipVoice	-0.30 $\pm$ 0.16	3.35 $\pm$ 0.19
MaskGCT	-0.38 $\pm$ 0.17	3.20 $\pm$ 0.18
OmniVoice-Emilia	0.42 $\pm$ 0.15	3.58 $\pm$ 0.18
OmniVoice	0.44 $\pm$ 0.16	3.80 $\pm$ 0.17

Table 2: Subjective evaluation results on Chinese and English test sets. CMOS and SMOS are used for evaluation. Best results are highlighted in bold .

Language	WER $\downarrow$			SIM-o $\uparrow$
Language	OmniVoice	MiniMax	ElevenLabs	OmniVoice	MiniMax	ElevenLabs
Arabic	1.392	1.665	1.666	0.776	0.736	0.706
Cantonese	17.709 *	34.111	51.513	0.838	0.778	0.670
Chinese	1.008	2.252	16.026	0.821	0.780	0.677
Czech	2.856	3.875	2.108	0.837	0.796	0.685
Dutch	1.358	1.143	0.803	0.813	0.738	0.680

Table 3: Evaluation on the MiniMax-Multilingual-24 test set.

Model	FLEURS-Multilingual-102
	Avg SIM-o $\uparrow$	Avg CER $\downarrow$	Languages with CER $\leq$
	Avg SIM-o $\uparrow$	Avg CER $\downarrow$	5%	10%
Ground-truth	-	5.11	75	92
OmniVoice	0.788	4.00	82	95

Table 4: Evaluation on the FLEURS-Multilingual-102 test set.

Figure 4: CERs of OmniVoice vs. ground truth across languages with varying training data durations on FLEURS-Multilingual-102. Hollow circles indicate languages where OmniVoice achieves lower CER than ground-truth, Filled circles indicate higher CER.

Masking Strategy	Librispeech-PC test-clean
Masking Strategy	SIM-o $\uparrow$	WER $\downarrow$	UTMOS $\uparrow$
SoundStorm-style mask	0.661	3.00	4.12
MaskGCT-style mask	0.660	2.04	4.17
Full-codebook random mask	0.697	1.57	4.23
- compute loss on a single codebook	0.648	2.85	4.22

Table 5: Impact of different masking strategies.

Model	Learning Rate	WER $\downarrow$
Model	Learning Rate	Libri	Seed-en	Seed-zh
LLM initialization	1e-4	1.57	1.72	0.89
Random initialization	1e-4	2.79	2.34	1.11
	2e-4	2.52	2.43	1.01
	5e-4	2.56	2.07	1.01
	1e-3	2.72	2.29	1.02

Table 6: Impact of LLM initialization on WER across datasets. Abbreviations: Libri = Librispeech-PC test-clean, Seed-en = Seed-TTS test-en, Seed-zh = Seed-TTS test-zh.

Model	Librispeech-PC test-clean
Model	SIM-o $\uparrow$	WER $\downarrow$	UTMOS $\uparrow$
w/o prompt denoise	0.697	1.57	4.23
w/ prompt denoise	0.668	1.56	4.32

Table 7: Impact of prompt denoising task.

Inference Steps	Batch Size
Inference Steps	1	2	4	8
16	0.0319	0.0263	0.0235	0.0224
32	0.0598	0.0486	0.0436	0.0414

Table 8: RTF with different inference steps and batch sizes.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

	LibriSpeech-PC test-clean			Seed-TTS test-en			Seed-TTS test-zh
Steps	SIM-o $\uparrow$	WER $\downarrow$	UTMOS $\uparrow$	SIM-o $\uparrow$	WER $\downarrow$	UTMOS $\uparrow$	SIM-o $\uparrow$	WER $\downarrow$	UTMOS $\uparrow$
64	0.729	1.28	4.30	0.742	1.60	3.92	0.777	0.81	3.13
32	0.729	1.30	4.28	0.741	1.53	3.91	0.777	0.84	3.11
16	0.728	1.50	4.23	0.735	1.72	3.92	0.773	0.99	3.00
8	0.713	2.02	4.02	0.716	1.94	3.59	0.756	1.58	2.72

Appendix

Table 9: Objective evaluation results of OmniVoice with different inference steps.

Language	Ground-truth	OmniVoice	Language	Ground-truth	OmniVoice
Afrikaans	3.54	3.08	Amharic	4.98	7.30
Armenian	1.38	1.45	Assamese	5.29	3.92
Asturian	3.79	2.07	Azerbaijani	2.60	3.20
Belarusian	1.80	1.00	Bengali	4.10	3.08
Bosnian	1.61	0.66	Bulgarian	1.66	0.87
Burmese	6.77	12.20	Cantonese	11.01	21.92

Appendix

Table 10: Per-language CERs of ground-truth and OmniVoice on FLEURS-Multilingual-102

Language	ID	ISO	Hours	Language	ID	ISO	Hours	Language	ID	ISO	Hours
Abadi	kbt	kbt	9.73	Abkhazian	ab	abk	57.27	Abron	abr	abr	9.22
Abua	abn	abn	10.27	Adamawa Fulfulde	fub	fub	13.12	Adyghe	ady	ady	32.6
Afade	aal	aal	10.19	Afrikaans	af	afr	4.4	Agwagwune	yay	yay	8.26
Aja (Benin)	ajg	ajg	5.63	Akebu	keu	keu	9.1	Alago	ala	ala	11.04
Albanian	sq	sqi	8.59	Algerian Arabic	arq	arq	9.64	Algerian Saharan Arabic	aao	aao	2.02
Ambo-Pasco Quechua	qva	qva	9.59	Ambonese Malay	abs	abs	10.03	Amdo Tibetan	adx	adx	56.94

Appendix

Table 11: Supported languages of OmniVoice along with its OmniVoice language ID, ISO 639-3 code, and training data duration (hours).

Multilingual TTS Synthesis

See all 21 papers

Voxtral TTS
Mar 26, 2026Mistral-AI, :, Alexander H. Liu +186Multilingual TTS Synthesis TTS Synthesis
DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speech
Jul 5, 2026Junwon Moon, Yejin Lee, Seungbeom Kim +4TTS Synthesis Diffusion Model Fine-Tuning
Sungkyunkwan University · University of Seoul
VoiceTTA: Enhancing Zero-Shot Text-to-Speech via Reinforcement Learning-Based Test-Time Adaptation
Jun 25, 2026Tianxin Xie, Chenxing Li, Dong Yu +1Test-Time Adaptation Reinforcement Learning
The Hong Kong University of Science and Technology (Guangzhou) · Tencent

Mar 26, 2026J/K move · Enter open · S save

Voxtral TTS

Mistral-AI, :, Alexander H. Liu, Alexis Tacnet+185

Multilingual TTS Synthesis TTS Synthesis Expressive Speech Synthesis Voice Cloning Zero-Shot TTS

Open paper AskPDF arXiv

We introduce Voxtral TTS, an expressive multilingual text-to-speech model that generates natural speech from as little as 3 seconds of reference audio. Voxtral TTS adopts a hybrid architecture that combines auto-regressive generation of semantic speech tokens with flow-matching for acoustic tokens. These tokens are encoded and decoded with Voxtral Codec, a speech tokenizer trained from scratch with a hybrid VQ-FSQ quantization scheme. In human evaluations conducted by native speakers, Voxtral TTS is preferred for multilingual voice cloning due to its naturalness and expressivity, achieving a 68.4% win rate over ElevenLabs Flash v2.5. We release the model weights under a CC BY-NC license.

Abstract

Multilingual TTS Synthesis

Explore similar work

Voxtral TTS

DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speech

VoiceTTA: Enhancing Zero-Shot Text-to-Speech via Reinforcement Learning-Based Test-Time Adaptation