We introduce Voxtral TTS, an expressive multilingual text-to-speech model that generates natural speech from as little as 3 seconds of reference audio. Voxtral TTS adopts a hybrid architecture that combines auto-regressive generation of semantic speech tokens with flow-matching for acoustic tokens. These tokens are encoded and decoded with Voxtral Codec, a speech tokenizer trained from scratch with a hybrid VQ-FSQ quantization scheme. In human evaluations conducted by native speakers, Voxtral TTS is preferred for multilingual voice cloning due to its naturalness and expressivity, achieving a 68.4% win rate over ElevenLabs Flash v2.5. We release the model weights under a CC BY-NC license.
Figures & tables
Figure 1: Voxtral TTS is preferred to ElevenLabs Flash v2.5 in human evaluations. We plot the win rate for Voxtral TTS against ElevenLabs Flash v2.5 in human evaluations across two categories. For flagship voices, we use the default voices for each model and 77 unique text examples. In the voice cloning set-up, we provide a short audio reference clip and 60 text prompts. In both categories, human annotators blindly rate which audio is better between the two models. Voxtral TTS is preferred in 58.3 and 68.4% of instances.
Figure 2: Architecture overview of Voxtral TTS. A voice reference ranging from 3s-30s is fed to the Voxtral Codec encoder to obtain audio tokens at a frame rate of 12.5 Hz. Each audio frame (labeled A ) consists of a semantic token and acoustic tokens. The voice reference audio tokens along with the text prompt tokens (labeled T ) are fed to the decoder backbone. The decoder auto-regressively generates a sequence of semantic tokens until it reaches a special End of Audio token ( <EOA> ). At each timestep, the semantic token from the decoder backbone is fed to a flow-matching transformer, which is run multiple times to predict the acoustic tokens. The semantic and acoustic tokens are fed to the Voxtral Codec decoder to obtain the generated waveform.
Figure 3: Architecture overview and training of Voxtral Codec. It consists of a split semantic VQ codebook and acoustic FSQ codebooks. Both semantic and acoustic tokens are combined for reconstruction. The semantic token has an additional distillation loss from a supervised ASR model.
Parameter
Value
Input / Preprocessing
Sampling rate
24000
Patch size
240
AutoEncoder
Encoder patch projection kernel size
7
Encoder patch projection dimension
1024
Table 1: Key hyperparameters of the Voxtral Codec.
WER (%) ↓
UTMOS ↑
Speaker Sim ↑
Task
Voxtral
ElevenLabs v3
ElevenLabs Flash
Voxtral
ElevenLabs v3
ElevenLabs Flash
Voxtral
ElevenLabs v3
ElevenLabs Flash
MiniMax
Arabic
2.68
3.67
2.86
3.07
2.50
2.89
0.746
0.546
0.539
German
0.83
0.45
1.08
3.12
2.90
3.27
0.721
0.457
0.489
English
0.63
0.48
0.33
4.30
4.27
4.27
0.786
0.484
0.489
Spanish
0.51
0.87
0.49
3.41
3.18
2.99
0.762
0.443
0.541
Table 3: WER, UTMOS, and Speaker Similarity scores for Voxtral TTS, ElevenLabs v3, and ElevenLabs Flash v2.5.
Emotion steering
Opponent Model
Voxtral TTS Win Rate (%)
Explicit
ElevenLabs v3
51.0
Gemini 2.5 Flash TTS
35.4
Implicit
ElevenLabs Flash v2.5
58.3
ElevenLabs v3
55.4
Gemini 2.5 Flash TTS
37.1
Table 4: Voxtral TTS win rates by steering type. In the explicit steering setting, Voxtral TTS is competitive with ElevenLabs v3, while having a higher win rate compared to both ElevenLabs models in the implicit steering setting.
Language
Voxtral TTS Win Rate (%)
Arabic
72.9
Dutch
49.4
English
60.8
French
54.4
German
72.0
Hindi
79.8
Table 5: Voxtral TTS win rate against ElevenLabs Flash v2.5 across languages. Voxtral TTS matches or outperforms ElevenLabs Flash v2.5 on every language, and has an overall micro-average win rate of 68.4%.
WER (%) ↓
UTMOS ↑
Task
Pretrain
DPO
Pretrain
DPO
MiniMax
Arabic
2.80
2.68 (-0.12)
3.01
3.07 (+0.06)
German
4.08
0.83 (-3.25)
3.05
3.12 (+0.07)
English
0.84
0.63 (-0.21)
4.25
4.30 (+0.05)
Spanish
0.56
0.51 (-0.06)
3.38
3.41 (+0.04)
Table 6: DPO improves WER and UTMOS across languages.
Figure 4: Effect of NFEs and CFG on automatic evaluations. The metrics are averaged over SEED-TTS and the 9 languages in MiniMax. Increasing the NFEs from 2 to 8 improves speaker similarity and UTMOS metrics. There is a slight regression in WER as the NFEs is increased beyond this. The metrics monotonically increase with higher CFG, but human evaluations flagged regressions in text-adherence with high α .
Configuration
Latency
RTF
Eager mode
133 ms
0.258
CUDA graph
70 ms
0.103
Table 7: Effect of CUDA graph acceleration on the flow-matching transformer.
Concurrency
Latency
RTF
Throughput (char/s/GPU)
Wait Rate
1
70 ms
0.103
119.14
0%
16
331 ms
0.237
879.11
0%
32
552 ms
0.302
1430.78
0%
Table 8: Serving performance of Voxtral TTS on a single H200.
In this paper, we present X-Voice, a 0.4B multilingual zero-shot voice cloning model that clones arbitrary voices and enables everyone to speak 30 languages. X-Voice is trained on a 420K-hour multilingual corpus using the International Phonetic Alphabet (IPA) as a unified representation. To eliminate the reliance on prompt text without complex preprocessing like forced alignment, we design a two-stage training paradigm. In Stage 1, we establish X-Voices1 through standard conditional flow-matching training and use it to synthesize 10K hours of speaker-consistent segments as audio prompts. In Stage 2, we fine-tune on these audio pairs with prompt text masked to derive X-Voices2, which enables zero-shot voice cloning without requiring transcripts of audio prompts. Architecturally, we extend F5-TTS by implementing a dual-level injection of language identifiers and decoupling and scheduling of Classifier-Free Guidance to facilitate multilingual speech synthesis. Subjective and objective evaluation results demonstrate that X-Voice outperforms existing flow-matching based multilingual systems like LEMAS-TTS and achieves zero-shot cross-lingual cloning capabilities comparable to billion-scale models such as Qwen3-TTS. To facilitate research transparency and community advancement, we open-source all related resources.
Rixi Xu, Qingyu Liu, Haitao Li +10
1MoE Key Lab of Artificial Intelligence, X-LANCE Lab, Shanghai Jiao Tong University · Center for Language and Speech Processing, Johns Hopkins University · 2Shanghai Innovation Institute +4
We present JaiTTS-v1.0, a state-of-the-art Thai voice cloning text-to-speech model built through continual training on a large Thai-centric speech corpus. The model architecture is adapted from VoxCPM, a tokenizer-free autoregressive TTS model. JaiTTS-v1.0 directly processes numerals and Thai-English code-switching, which is very common in realistic settings, without explicit text normalization. We test the models on short- and long-duration speech generation, which reflects many real-world use cases. JaiTTS-v1.0 achieves a state-of-the-art CER of 1.94%, surpassing the human ground truth of 1.98% for short-duration tasks while performing on par with human ground truth for long-duration tasks. In human judgment evaluations, our model wins 283 of 400 pairwise comparisons against commercial flagships, with only 58 losses. Our code and demo are available at https://github.com/JTS-AI-Team/JaiTTS .
Text-to-speech (TTS) systems are conventionally optimized for intelligibility and speaker similarity. However, neither objective captures the prosodic detail that distinguishes human performances from synthetic speech, leaving a realism gap that larger models and higher-quality data can narrow but not close. In the absence of an objective that directly targets this gap, realism remains a persistent challenge in TTS. We present Audio8-TTS, a compact 0.6B-parameter model for zero-shot voice cloning that achieves state-of-the-art performance in intelligibility and speaker similarity, with particularly strong results in perceptual naturalness. To align the training signal with human perception rather than signal-level fidelity, we construct a quality ladder from widely used open-source TTS systems and train a realism critic to assess generated speech against matched human recordings of the same sentences. An identity-erasure objective discourages the critic from relying on generator fingerprints. We train the model in three stages: multilingual supervised fine-tuning; group relative policy optimization (GRPO) with word error rate (WER) and speaker similarity (SIM) rewards; and GRPO guided by the realism critic. The final stage improves perceived prosodic naturalness and speaker similarity while preserving WER performance.