Voxtral TTS

Mistral-AI; :; Alexander H. Liu; Alexis Tacnet; Andy Ehrenberg; Andy Lo; Chen-Yo Sun; Guillaume Lample

cs.AIMar 26, 2026

Voxtral TTS

Authors: Mistral-AI, :, Alexander H. Liu, Alexis Tacnet, Andy Ehrenberg, Andy Lo, Chen-Yo Sun, Guillaume Lample, Henry Lagarde, Jean-Malo Delignon, Jaeyoung Kim, John Harvill, Khyathi Raghavi Chandu, Lorenzo Signoretti, Margaret Jennings, Patrick von Platen, Pavankumar Reddy Muddireddy, Rohin Arora, Sanchit Gandhi, Samuel Humeau, Soham Ghosh, Srijan Mishra, Van Phung, Abdelaziz Bounhar, Abhinav Rastogi, Adrien Sadé, Alan Jeffares, Albert Jiang, Alexandre Cahill, Alexandre Gavaudan, Alexandre Sablayrolles, Amélie Héliou, Amos You, Andrew Bai, Andrew Zhao, Angele Lenglemetz, Anmol Agarwal, Anton Eliseev, Antonia Calvi, Arjun Majumdar, Arthur Fournier, Artjom Joosen, Avi Sooriyarachchi, Aysenur Karaduman Utkur, Baptiste Bout, Baptiste Rozière, Baudouin De Monicault, Benjamin Tibi, Bowen Yang, Charlotte Cronjäger, Clémence Lanfranchi, Connor Chen, Corentin Barreau, Corentin Sautier, Cyprien Courtot, Darius Dabert, Diego de las Casas, Elizaveta Demyanenko, Elliot Chane-Sane, Emmanuel Gottlob, Enguerrand Paquin, Etienne Goffinet, Fabien Niel, Faruk Ahmed, Federico Baldassarre, Gabrielle Berrada, Gaëtan Ecrepont, Gauthier Guinet, Genevieve Hayes, Georgii Novikov, Giada Pistilli, Guillaume Kunsch, Guillaume Martin, Guillaume Raille, Gunjan Dhanuka, Gunshi Gupta, Han Zhou, Harshil Shah, Hope McGovern, Hugo Thimonier, Indraneel Mukherjee, Irene Zhang, Jacques Sun, Jan Ludziejewski, Jason Rute, Jérémie Dentan, Joachim Studnia, Jonas Amar, Joséphine Delas, Josselin Somerville Roberts, Julien Tauran, Karmesh Yadav, Kartik Khandelwal, Kilian Tep, Kush Jain, Laurence Aitchison, Laurent Fainsin, Léonard Blier, Lingxiao Zhao, Louis Martin, Lucile Saulnier, Luyu Gao, Maarten Buyl, Manan Sharma, Marie Pellat, Mark Prins, Martin Alexandre, Mathieu Poirée, Mathieu Schmitt, Mathilde Guillaumin, Matthieu Dinot, Matthieu Futeral, Maxime Darrin, Maximilian Augustin, Mert Unsal, Mia Chiquier, Mikhail Biriuchinskii, Minh-Quang Pham, Mircea Lica, Morgane Rivière, Nathan Grinsztajn, Neha Gupta, Olivier Bousquet, Olivier Duchenne, Patricia Wang, Paul Jacob, Paul Wambergue, Paula Kurylowicz, Philippe Pinel, Philomène Chagniot, Pierre Stock, Piotr Miłoś, Prateek Gupta, Pravesh Agrawal, Quentin Torroba, Ram Ramrakhya, Randall Isenhour, Rishi Shah, Romain Sauvestre, Roman Soletskyi, Rosalie Millner, Rupert Menneer, Sagar Vaze, Samuel Barry, Samuel Belkadi, Sandeep Subramanian, Sean Cha, Shashwat Verma, Siddhant Waghjale, Siddharth Gandhi, Simon Lepage, Sumukh Aithal, Szymon Antoniak, Tarun Kumar Vangani, Teven Le Scao, Théo Cachet, Theo Simon Sorg, Thibaut Lavril, Thomas Chabal, Thomas Foubert, Thomas Robert, Thomas Wang, Tim Lawson, Tom Bewley, Tom Edwards, Tyler Wang, Umar Jamil, Umberto Tomasini, Valeriia Nemychnikova, Vedant Nanda, Victor Jouault, Vincent Maladière, Vincent Pfister, Virgile Richard, Vladislav Bataev, Wassim Bouaziz, Wen-Ding Li, William Havard, William Marshall, Xinghui Li, Xingran Guo, Xinyu Yang, Yannic Neuhaus, Yassine El Ouahidi, Yassir Bendou, Yihan Wang, Yimu Pan, Zaccharie Ramzi, Zhenlin Xu

Abstract

We introduce Voxtral TTS, an expressive multilingual text-to-speech model that generates natural speech from as little as 3 seconds of reference audio. Voxtral TTS adopts a hybrid architecture that combines auto-regressive generation of semantic speech tokens with flow-matching for acoustic tokens. These tokens are encoded and decoded with Voxtral Codec, a speech tokenizer trained from scratch with a hybrid VQ-FSQ quantization scheme. In human evaluations conducted by native speakers, Voxtral TTS is preferred for multilingual voice cloning due to its naturalness and expressivity, achieving a 68.4% win rate over ElevenLabs Flash v2.5. We release the model weights under a CC BY-NC license.

Figures & tables

Figure 1: Voxtral TTS is preferred to ElevenLabs Flash v2.5 in human evaluations. We plot the win rate for Voxtral TTS against ElevenLabs Flash v2.5 in human evaluations across two categories. For flagship voices, we use the default voices for each model and 77 unique text examples. In the voice cloning set-up, we provide a short audio reference clip and 60 text prompts. In both categories, human annotators blindly rate which audio is better between the two models. Voxtral TTS is preferred in 58.3 and 68.4% of instances.

Figure 2: Architecture overview of Voxtral TTS. A voice reference ranging from 3s-30s is fed to the Voxtral Codec encoder to obtain audio tokens at a frame rate of 12.5 Hz. Each audio frame (labeled A ) consists of a semantic token and acoustic tokens. The voice reference audio tokens along with the text prompt tokens (labeled T ) are fed to the decoder backbone. The decoder auto-regressively generates a sequence of semantic tokens until it reaches a special End of Audio token ( <EOA> ). At each timestep, the semantic token from the decoder backbone is fed to a flow-matching transformer, which is run multiple times to predict the acoustic tokens. The semantic and acoustic tokens are fed to the Voxtral Codec decoder to obtain the generated waveform.

Figure 3: Architecture overview and training of Voxtral Codec. It consists of a split semantic VQ codebook and acoustic FSQ codebooks. Both semantic and acoustic tokens are combined for reconstruction. The semantic token has an additional distillation loss from a supervised ASR model.

	Parameter	Value
Input / Preprocessing
	Sampling rate	24000
	Patch size	240
AutoEncoder
	Encoder patch projection kernel size	7
	Encoder patch projection dimension	1024

Table 1: Key hyperparameters of the Voxtral Codec.

	WER (%) $\downarrow$			UTMOS $\uparrow$			Speaker Sim $\uparrow$
Task	Voxtral	ElevenLabs v3	ElevenLabs Flash	Voxtral	ElevenLabs v3	ElevenLabs Flash	Voxtral	ElevenLabs v3	ElevenLabs Flash
MiniMax
Arabic	2.68	3.67	2.86	3.07	2.50	2.89	0.746	0.546	0.539
German	0.83	0.45	1.08	3.12	2.90	3.27	0.721	0.457	0.489
English	0.63	0.48	0.33	4.30	4.27	4.27	0.786	0.484	0.489
Spanish	0.51	0.87	0.49	3.41	3.18	2.99	0.762	0.443	0.541

Table 3: WER, UTMOS, and Speaker Similarity scores for Voxtral TTS, ElevenLabs v3, and ElevenLabs Flash v2.5.

Emotion steering	Opponent Model	Voxtral TTS Win Rate (%)
Explicit	ElevenLabs v3	51.0
Explicit	Gemini 2.5 Flash TTS	35.4
Implicit	ElevenLabs Flash v2.5	58.3
Implicit	ElevenLabs v3	55.4
	Gemini 2.5 Flash TTS	37.1

Table 4: Voxtral TTS win rates by steering type. In the explicit steering setting, Voxtral TTS is competitive with ElevenLabs v3, while having a higher win rate compared to both ElevenLabs models in the implicit steering setting.

Language	Voxtral TTS Win Rate (%)
Arabic	72.9
Dutch	49.4
English	60.8
French	54.4
German	72.0
Hindi	79.8

Table 5: Voxtral TTS win rate against ElevenLabs Flash v2.5 across languages. Voxtral TTS matches or outperforms ElevenLabs Flash v2.5 on every language, and has an overall micro-average win rate of 68.4%.

	WER (%) $\downarrow$		UTMOS $\uparrow$
Task	Pretrain	DPO	Pretrain	DPO
MiniMax
Arabic	2.80	2.68 (-0.12)	3.01	3.07 (+0.06)
German	4.08	0.83 (-3.25)	3.05	3.12 (+0.07)
English	0.84	0.63 (-0.21)	4.25	4.30 (+0.05)
Spanish	0.56	0.51 (-0.06)	3.38	3.41 (+0.04)

Table 6: DPO improves WER and UTMOS across languages.

Figure 4: Effect of NFEs and CFG on automatic evaluations. The metrics are averaged over SEED-TTS and the 9 languages in MiniMax. Increasing the NFEs from 2 to 8 improves speaker similarity and UTMOS metrics. There is a slight regression in WER as the NFEs is increased beyond this. The metrics monotonically increase with higher CFG, but human evaluations flagged regressions in text-adherence with high

\alpha

Configuration	Latency	RTF
Eager mode	133 ms	0.258
CUDA graph	70 ms	0.103

Table 7: Effect of CUDA graph acceleration on the flow-matching transformer.

Concurrency	Latency	RTF	Throughput (char/s/GPU)	Wait Rate
1	70 ms	0.103	119.14	0%
16	331 ms	0.237	879.11	0%
32	552 ms	0.302	1430.78	0%

Table 8: Serving performance of Voxtral TTS on a single H200.

Multilingual TTS Synthesis

See all 21 papers

X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning
May 7, 2026Rixi Xu, Qingyu Liu, Haitao Li +10TTS Synthesis Zero-Shot TTS
1MoE Key Lab of Artificial Intelligence, X-LANCE Lab, Shanghai Jiao Tong University · Center for Language and Speech Processing, Johns Hopkins University · 2Shanghai Innovation Institute +4
JaiTTS: A Thai Voice Cloning Model
Apr 30, 2026Jullajak Karnjanaekarin, Pontakorn Trakuekul, Narongkorn Panitsrisit +5TTS Synthesis Audio Generation
Jasmine Technology Solution · Sirindhorn International Institute of Technology · Department of Linguistics, Chulalongkorn University
Training a Compact, Strong, and Realistic Text-to-Speech Model
Oct 10, 2026Runyuan Cai, Yu Lin, Yiming Wang +1TTS Synthesis Zero-Shot TTS
AutoArk

May 7, 2026J/K move · Enter open · S save

X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning

Rixi Xu, Qingyu Liu, Haitao Li, Yushen Chen+9

1MoE Key Lab of Artificial Intelligence, X-LANCE Lab, Shanghai Jiao Tong University · Center for Language and Speech Processing, Johns Hopkins University · 2Shanghai Innovation Institute+4

TTS Synthesis Zero-Shot TTS Voice Cloning Multilingual TTS Synthesis

Open paper AskPDF arXiv

In this paper, we present X-Voice, a 0.4B multilingual zero-shot voice cloning model that clones arbitrary voices and enables everyone to speak 30 languages. X-Voice is trained on a 420K-hour multilingual corpus using the International Phonetic Alphabet (IPA) as a unified representation. To eliminate the reliance on prompt text without complex preprocessing like forced alignment, we design a two-stage training paradigm. In Stage 1, we establish X-Voice

_{\text{s1}}

through standard conditional flow-matching training and use it to synthesize 10K hours of speaker-consistent segments as audio prompts. In Stage 2, we fine-tune on these audio pairs with prompt text masked to derive X-Voice

_{\text{s2}}

, which enables zero-shot voice cloning without requiring transcripts of audio prompts. Architecturally, we extend F5-TTS by implementing a dual-level injection of language identifiers and decoupling and scheduling of Classifier-Free Guidance to facilitate multilingual speech synthesis. Subjective and objective evaluation results demonstrate that X-Voice outperforms existing flow-matching based multilingual systems like LEMAS-TTS and achieves zero-shot cross-lingual cloning capabilities comparable to billion-scale models such as Qwen3-TTS. To facilitate research transparency and community advancement, we open-source all related resources.

Abstract

Multilingual TTS Synthesis

Explore similar work

X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning

JaiTTS: A Thai Voice Cloning Model

Training a Compact, Strong, and Realistic Text-to-Speech Model