Fish Audio S2 Technical Report

Shijia Liao; Yuxuan Wang; Songting Liu; Yifan Cheng; Ruoyi Zhang; Tianyu Li; Shidong Li; Yisheng Zheng

cs.SDMar 9, 2026

Fish Audio S2 Technical Report

Authors: Shijia Liao, Yuxuan Wang, Songting Liu, Yifan Cheng, Ruoyi Zhang, Tianyu Li, Shidong Li, Yisheng Zheng,

Abstract

We introduce Fish Audio S2, an open-sourced text-to-speech system featuring multi-speaker, multi-turn generation, and, most importantly, instruction-following control via natural-language descriptions. To scale training, we develop a multi-stage training recipe together with a staged data pipeline covering video captioning and speech captioning, voice-quality assessment, and reward modeling. To push the frontier of open-source TTS, we release our model weights, fine-tuning code, and an SGLang-based inference engine. The inference engine is production-ready for streaming, achieving an RTF of 0.195 and a time-to-first-audio below 100 ms.Our code and weights are available on GitHub (https://github.com/fishaudio/fish-speech) and Hugging Face (https://huggingface.co/fishaudio/s2-pro). We highly encourage readers to visit https://fish.audio to try custom voices.

Figures & tables

Figure 1: Fish Audio S2 is a multilingual, controllable, and expressive TTS system supporting long-form, multi-speaker, multi-turn generation with ultra-low TTFA and RTF.

Figure 4: Fish Audio S2 supports multi-speaker generation with fine-grained natural language control over prosody, emotion, and speaking style.

Figure 5: Training reward curves during RL-based post-training.

Model	WER (%) $\downarrow$ test-zh \| test-en \| zh-hard
Fish Audio S2	0.54 \| 0.99 \| 5.99
Fish Audio S1	0.54 \| 1.07 \| 17.00
CosyVoice 3-1.5B ( Du et al., 2025 )	1.12 \| 2.21 \| 5.83
Qwen3-TTS ( Hu et al., 2026 )	0.77 \| 1.24 \| –
Qwen3-Omni ( Xu et al., 2025b )	1.07 \| 1.39 \| –
FireRedTTS-2 ( Xie et al., 2025 )	1.14 \| 1.95 \| –

Table 1: Results on Seed-TTS-Eval.

Model	WER (%) $\downarrow$
Model	zh	en	ja	ko	de	es	fr	it	ru	hard-zh	hard-en
Fish Audio S2	2.65	2.43	3.96	2.76	2.22	2.00	6.26	2.04	2.78	9.10	4.40
Fish Audio S1	2.98	3.00	4.54	3.19	2.78	2.84	8.15	2.94	5.18	9.72	7.26
CosyVoice2 + DiffRO	3.00	4.72	6.36	5.14	–	–	–	–	–	10.66	10.25
CosyVoice3-0.5B + DiffRO	2.89	3.68	5.15	4.02	4.51	2.99	8.56	2.94	3.79	8.26	7.60
CosyVoice3-1.5B + DiffRO	3.01	3.71	5.27	4.01	3.93	3.26	8.09	2.72	4.11	9.06	7.56

Table 2: Results on CV3-Eval multilingual voice cloning subset.

Language	WER (%) $\downarrow$				SIM $\uparrow$
Language	MiniMax	ElevenLabs	Fish Audio S2	Fish Audio S1	MiniMax	ElevenLabs	Fish Audio S2	Fish Audio S1
Arabic	1.665	1.666	3.500	6.420	0.736	0.706	0.750	0.713
Cantonese	34.111	51.513	30.670	48.800	0.778	0.670	0.805	0.773
Chinese	2.252	16.026	0.730	0.980	0.780	0.677	0.816	0.777
Czech	3.875	2.108	2.840	18.020	0.796	0.685	0.798	0.733
Dutch	1.143	0.803	0.990	2.270	0.738	0.680	0.730	0.701

Table 3: Results on the Minimax Multilingual Testset.

Model	English			Chinese
Model	WER (%) $\downarrow$	SIM-Mean $\uparrow$	SIM-Std $\downarrow$	CER (%) $\downarrow$	SIM-Mean $\uparrow$	SIM-Std $\downarrow$
Fish Audio S2	4.38	0.523	0.0761	5.95	0.557	0.0923
Fish Audio S1	6.26	0.436	0.108	6.44	0.505	0.108
Qwen3-TTS ( Hu et al., 2026 )	7.69	0.390	0.0737	8.09	0.574	0.0614
VibeVoice ( Peng et al., 2025 )	28.0	0.530	0.0572	26.2	0.609	0.0537

Table 4: Results on the Long-Audio benchmark.

Model	Mean (Std)	95 % HDI
Fish Audio S2	0.483 (0.068)	[0.477, 0.489]
Fish Audio S2 (w/ instruction)	0.515 (0.061)	[0.510, 0.521]
Fish Audio S1	0.479 (0.087)	[0.471, 0.486]
Seed-TTS ( Anastassiou et al., 2024 )	0.417 (0.011)	[0.398, 0.438]
MiniMax-Speech ( Zhang et al., 2025 )	0.387 (0.011)	[0.368, 0.407]
Step-Audio ( Huang et al., 2025 )	0.286 (0.011)	[0.266, 0.307]

Table 5: Posterior summary statistics from the ATT evaluation: means, standard deviations (Std), and 95% highest density intervals (HDI).

Model	Overall		Emotions		Foreign Words		Paralinguistics		Questions		Syntactic Comp.
	WER (%) $\downarrow$	Win (%) $\uparrow$	WER (%) $\downarrow$	Win (%) $\uparrow$	WER (%) $\downarrow$	Win (%) $\uparrow$	WER (%) $\downarrow$	Win (%) $\uparrow$	WER (%) $\downarrow$	Win (%) $\uparrow$	WER (%) $\downarrow$	Win (%) $\uparrow$
Fish Audio S2 $\star$	8.15	81.88	0.52	86.61	14.74	63.39	24.58	91.61	0.38	84.41	0.51	83.39
Fish Audio S1	7.60	36.88	0.75	24.82	15.77	53.75	20.02	37.68	0.72	43.55	0.74	24.64
Gemini-2.5-Flash-Preview-TTS $\star$	6.35	79.10	0.71	97.32	11.80	65.56	18.38	91.25	0.40	74.82	0.50	67.14
Gemini-2.5-Flash-Preview-TTS	6.59	78.00	0.60	95.00	12.99	63.03	18.25	89.10	0.36	74.82	0.73	68.03
gpt-4o-audio-preview $\star$	7.75	77.36	1.82	96.96	13.30	70.00	21.15	89.46	1.38	62.50	1.16	68.39

Table 6: Results on EmergentTTS-Eval across five dimensions, with WER and Win-Rate against baseline.

\star

indicates strong prompting.

Dataset	Metric	Fish Audio S2	Fish Audio S1
Chinese	TAR $\uparrow$	0.984	0.942
	Naturalness $\uparrow$	4.40	4.15
	Expressiveness $\uparrow$	4.94	4.65
English	TAR $\uparrow$	0.881	0.626
	Naturalness $\uparrow$	4.21	3.71
	Expressiveness $\uparrow$	4.50	3.93

Table 7: Results on the Fish Audio Instruction Benchmark.

Multi-Speaker TTS

See all 10 papers

Fish Audio Drama 3 Technical Report
Oct 13, 2026Dawei Han, Shidong Li, Shijia Liao +13TTS Synthesis Controllable Speech Generation
VoTSpeech: Decoupling Voice Design from Speech Generation in Instruction-Following TTS
Oct 10, 2026Wenbing Yang, Qihang Lu, Zihan Sun +4TTS Synthesis Controllable Speech Generation
Beijing University of Posts and Telecommunications, Beijing, China · Hello Group, Beijing, China
InstCharVoice: Grounding Natural-Language Instructions for Character-Level Control in Text-to-Speech
Sep 28, 2026Sihang Nie, Xueru Li, Xiaofen Xing +4TTS Synthesis Language Model-Based Control
South China University of Technology · Huya Inc. · The Hongkong Polytechnic University

Oct 13, 2026J/K move · Enter open · S save

Fish Audio Drama 3 Technical Report

Dawei Han, Shidong Li, Shijia Liao, Jiahua Liu+12

TTS Synthesis Controllable Speech Generation Speech Foundation Models Streaming TTS Synthesis Voice Cloning Speech Editing

Open paper AskPDF arXiv

We introduce Fish Audio Drama 3, an open-source speech generation model that turns text into directed performances, built on a 35B-parameter mixture-of-experts backbone that activates 3B parameters per token. It clones voices from multiple references, follows natural-language direction, revises a take when the next turn gives a note, voices multi-character scenes, edits existing speech by infilling, and synthesizes long-form and streaming audio. Multi-turn revision, scene dubbing, editing by infilling, and streaming text input are new in Drama 3, all in one model. Drama 3 pairs the hybrid linear-attention backbone with a new codebook multi-token-prediction audio decoder that predicts the acoustic codebooks of each frame in dedicated layers that attend over the entire sequence. The model is trained with a three-stage pre-training recipe, instruction fine-tuning, and large-batch reinforcement learning whose only reward is a single learned model of human preference. Its inference engine decodes a single stream at 271 frames per second on one H200, against 177 for Fish Audio S2, although Drama 3 has more than seven times as many parameters. The model weights are available on GitHub (https://github.com/fishaudio/fish-speech) and Hugging Face (https://huggingface.co/fishaudio/drama-3), and inference is available through SGLang-Omni (https://github.com/sgl-project/sglang-omni). We highly encourage readers to visit https://fish.audio to direct their own performances.

Abstract

Multi-Speaker TTS

Explore similar work

Fish Audio Drama 3 Technical Report

VoTSpeech: Decoupling Voice Design from Speech Generation in Instruction-Following TTS

InstCharVoice: Grounding Natural-Language Instructions for Character-Level Control in Text-to-Speech