ZONOS3-Mini Technical Report
Organizations: Zyphra · San Francisco, CA
Abstract
ZONOS3-Mini is an ultra-compact, ultra-fast, zero-shot voice-cloning TTS model. The model can run in real time on a laptop or smartphone. For its latency and size, it achieves state-of-the-art accuracy and cloning fidelity. To achieve this we propose a latent MoE backbone with only 886M parameters (100M active per token) and a new audio codec, Zyphra Audio Codec 1 (ZAC-1), which represents 44.1 kHz audio at a 15 Hz frame rate. On the new ZTTS2-Eval benchmark, ZONOS3-Mini reaches a 6.71% macro-average WER over 55 language sets (35 clean and 20 in the wild), which is competitive with larger and slower state-of-the-art models. We release the model weights, inference, and training code.
Figures & tables
| WER | Sim | ||||
| Codec | Frames/s | clean | wild | clean | wild |
| Fish S1–DAC | 21.5 | 4.30 | 5.28 | 84.8 | 79.0 |
| ZAC-1 | 15.0 | 4.06 | 5.54 | 83.8 | 76.1 |
| MOSS, 14 RVQ | 12.5 | 5.06 | 5.67 | 84.6 | 80.3 |
| DAC | 86.1 | 42.2 | 51.7 | 78.8 | 63.7 |
| Codec | Frames/s | kbit/s | PESQ | STOI | Mel | LSD | SI-SDR | Sim | UTMOS |
| ZTTS2-Eval ground truth, 1,600 clips, 16 kHz (original audio UTMOS 2.80) | |||||||||
| ZAC-1, full decoder | 15 | 1.80 | 2.88 | 0.928 | 0.179 | 0.955 | 7.83 | 88.7 | 2.63 |
| ZAC-1, keep-2 decoder | 15 | 1.80 | 2.51 | 0.911 | 0.202 | 1.005 | 5.72 | 84.9 | 2.52 |
| Fish S1– DAC | 21.5 | 2.20 | 3.10 | 0.941 | 0.149 | 0.900 | 7.25 | 90.7 | 2.66 |
| DAC | 86.1 | 7.75 | 3.89 | 0.964 | 0.132 | 0.926 | 12.94 | 93.8 | 2.71 |
| MOSS tokenizer, 14 RVQ | 12.5 | 1.75 | 3.13 | 0.940 | 0.152 | 0.911 | 8.81 | 92.3 | 2.70 |
| Clean (35 languages) | In the wild (20 languages) | Macro | |||||
| Model | Conditioning | WER | Sim | WER | Sim | WER | Sim |
| ZONOS3-Mini | embedding | 8.28 | 83.1 | 7.52 | 74.0 | 8.00 | 79.8 |
| ZONOS3-Mini | raw audio | 7.95 | 77.7 | 12.57 | 55.1 | 9.63 | 69.5 |
| ZONOS3-Mini + PT | embedding | 7.79 | 83.4 | 4.81 | 77.3 | 6.71 | 81.2 |
| ZONOS3-Mini + PT | raw audio | 7.39 | 76.5 | 5.48 | 58.1 | 6.70 | 69.8 |
| Params | Clean | In the wild | Macro | ||||
| System | total / active | WER | Sim | WER | Sim | WER | Sim |
| Ground truth | – | 9.66 | 70.7 | 9.19 | 79.3 | 9.49 | 73.8 |
| With reference transcript | |||||||
| Qwen3-TTS 1.7B [ 16 ] | 1.7B | 22.80 | 80.6 | 14.61 | 74.5 | 19.82 | 78.4 |
| Fish S2 Pro [ 24 ] | 4.4B | 11.03 | 81.2 | 5.98 | 70.3 | 9.20 | 77.2 |
| VoxCPM2 [ 58 ] | 2B | 12.47 | 86.2 | 5.12 | 77.8 | 9.80 | 83.1 |
| Clean, 4 languages | In the wild | ||||
| System | Voice | WER | Sim | WER | Sim |
| Cartesia Sonic 3.6 | clone | 2.49 | 85.0 | 3.30 | 76.4 |
| Inworld TTS 2 | clone | 2.42 | 71.5 | 3.89 | 61.4 |
| ElevenLabs v4 | clone | 2.07 | 82.9 | 2.97 | – |
| Gemini 3.8 Flash | preset | 2.55 | – | 4.67 | – |
| \zyoursrow ZONOS3-Mini + PT, embedding (ours) | clone | 2.55 | 83.1 | 5.00 | 77.5 |
| In the wild | Clean | |||||
| Arm (cumulative) | WER | Sim | UTMOS | WER | Sim | UTMOS |
| Original (ZONOS2 block) | 64.6 | 45.1 | 2.10 | 39.1 | 59.2 | 2.86 |
| Latent MoE | 83.2 | 44.2 | 1.95 | 33.7 | 69.3 | 2.87 |
| linear router, no EDA | 43.5 | 54.5 | 2.19 | 26.9 | 65.8 | 2.99 |
| quantile balancing | 39.0 | 56.3 | 2.20 | 33.7 | 60.0 | 2.93 |
| XSA | 49.1 | 53.1 | 2.19 | 31.2 | 67.0 | 2.95 |
| Configuration | Family | Frames/s | Codebooks | Codebook sizes | kbit/s | WER | Sim |
| Fish S1 codec [ 24 ] | S1– DAC | 21.53 | 10 | 2.20 | 4.79 | 81.87 | |
| ZAC-1 | dual hybrid | 15.00 | 10 | 1.80 | 4.80 | 79.94 | |
| Dual hybrid, 10 s | dual hybrid | 15.00 | 10 | 1.80 | 5.81 | 74.54 | |
| Dual hybrid R1024 | dual hybrid | 15.00 | 10 | 1.53 | 6.53 | 67.83 | |
| Hybrid, 15 Hz, 10 CB | hybrid | 15.00 | 10 | 1.80 | 7.25 | 68.37 | |
| Hybrid, 12.5 Hz, 10 CB | hybrid | 12.50 | 10 | 1.50 | 7.22 | 65.11 |
| Pre-train 375k | Fine-tune 60k | Rate stage 20k | |
| Nodes GPUs | 14 8 H100 | 14 8 H100 | 14 8 H100 |
| Positions / GPU / step | 106,496 | 106,496 | 106,496 |
| Positions / step | 11.93M | 11.93M | 11.93M |
| Steps | 375,000 | 60,000 | 20,000 |
| Muon / AdamW / bias LR | 0.02 / 0.002 / 0.002 | 0.0035 / 0.00035 / 0.00035 | 0.0035 / 0.00035 / 0.00035 |
| Warm-up / schedule | 250 / cosine to | 100 / cosine to 0 | 100 / cosine to 0 |
| Model | Metric | en | zh | de | es | fr | it | ja | ko | ru | pt | ar | hi | tr | pl | id | th | fil | nl |
| Reference | |||||||||||||||||||
| Ground truth | WER | 4.30 | 3.02 | 4.64 | 2.89 | 4.08 | 4.05 | 4.16 | 3.20 | 5.67 | 4.18 | 13.69 | 14.61 | 9.60 | 4.48 | 7.31 | 5.44 | 13.43 | 6.83 |
| Spk. sim. | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| With reference transcript | |||||||||||||||||||
| Qwen3-TTS 1.7B [ 16 ] | WER | 2.37 | 4.64 | 3.17 | 2.26 | 3.07 | 2.36 | 3.71 | 4.32 | 5.21 | 3.18 | 31.66 | 29.32 | 37.49 | 51.17 | 8.18 | 5.64 | 16.18 | 7.39 |
| Spk. sim. | 80.0 | 85.3 | 80.4 | 84.6 | 81.6 | 82.3 | 86.8 | 85.3 | 82.3 | 83.8 | 82.4 | 75.5 | 80.1 | 76.5 | 84.1 | 86.9 | 87.6 | 80.9 | |
| Model | Metric | vi | cs | sv | da | fi | el | hu | ro | uk | nb | he | bg | hr | sk | ca | ms | fa |
| Reference | ||||||||||||||||||
| Ground truth | WER | 4.35 | 12.31 | 7.99 | 11.71 | 12.87 | 16.69 | 18.73 | 6.85 | 6.31 | 11.70 | 28.97 | 14.70 | 13.35 | 12.20 | 7.26 | 8.15 | 28.50 |
| Spk. sim. | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | |
| With reference transcript | ||||||||||||||||||
| Qwen3-TTS 1.7B [ 16 ] | WER | 26.48 | 40.73 | 16.15 | 29.93 | 37.75 | 76.66 | 49.35 | 24.83 | 13.83 | 20.80 | 91.64 | 29.28 | 24.10 | 34.40 | 8.91 | 8.05 | 43.65 |
| Spk. sim. | 83.5 | 74.9 | 79.4 | 74.4 | 81.3 | 68.2 | 76.9 | 82.9 | 83.0 | 77.7 | 71.1 | 77.0 | 81.9 | 77.7 | 80.3 | 85.7 | 80.2 | |
| Model | Metric | en | zh | de | es | fr | it | ja | ko | ru | pt | ar | hi | id | tr | tl | pl | th | nl | uk | vi |
| Reference | |||||||||||||||||||||
| Ground truth | WER | 3.82 | 8.78 | 6.25 | 5.08 | 5.57 | 6.25 | 6.46 | 9.02 | 9.53 | 5.58 | 11.62 | 13.34 | 15.18 | 17.24 | 12.54 | 9.90 | 8.74 | 6.51 | 12.20 | 10.28 |
| Spk. sim. | 76.3 | 80.6 | 77.9 | 78.9 | 77.7 | 79.1 | 80.4 | 79.0 | 79.2 | 80.1 | 79.8 | 78.0 | 78.8 | 80.2 | 78.3 | 80.2 | 81.5 | 76.1 | 80.8 | 82.2 | |
| With reference transcript | |||||||||||||||||||||
| Qwen3-TTS 1.7B [ 16 ] | WER | 2.63 | 5.79 | 2.92 | 2.25 | 2.69 | 2.99 | 4.46 | 4.78 | 5.05 | 4.23 | 24.76 | 67.34 | 11.17 | 35.52 | 10.69 | 49.40 | 7.19 | 7.38 | 13.26 | 27.78 |
| Spk. sim. | 74.7 | 79.4 | 76.2 | 76.4 | 75.5 | 76.5 | 78.7 | 78.8 | 78.7 | 76.3 | 75.3 | 45.5 | 76.3 | 73.5 | 73.5 | 70.4 | 79.3 | 73.5 | 77.7 | 73.6 | |
TTS Synthesis
Text-to-speech synthesis converts written language into intelligible, natural-sounding speech with control over voice, prosody, style, and timing.
See all 291 papers