RVCBench: Benchmarking the Robustness of Voice Cloning Across Modern Audio Generation Models

Ruinan Jin; Xinting Liao; Hanlin Yu; Deval Pandya; Xiaoxiao Li

cs.SDJan 31, 2026

RVCBench: Benchmarking the Robustness of Voice Cloning Across Modern Audio Generation Models

Authors: Ruinan Jin, Xinting Liao, Hanlin Yu, Deval Pandya, Xiaoxiao Li

Organizations: The University of British Columbia · Vector Institute

Abstract

Modern Voice Cloning (VC) can synthesize speech that closely matches a target speaker from only seconds of reference audio, enabling applications such as personalized speech interfaces and dubbing. In practical deployments, modern audio generation models inevitably encounter noisy reference audios, imperfect text prompts, multilingual and long-form generation settings, downstream post-processing, and adversarial perturbations, all of which can significantly hurt robustness. Despite rapid progress in VC driven by autoregressive codec-token language models and diffusion-based models, robustness under realistic deployment shifts remains underexplored. This paper introduces RVCBench, a comprehensive dataset and benchmark that evaluates Robustness in Voice Clone. RVCBench contributes a large-scale, task-aligned robustness dataset that instantiates realistic deployment shifts through controlled text-audio pairing, multilingual and long-form scenarios, expressive prompts, post-processing conditions, and passive or proactive audio perturbations. Covering 18 robustness evaluations, 204 unique speakers, and 14,370 utterance-level evaluation items, RVCBench enables unified evaluation of input sensitivity, generation stability, output resilience, and perturbation robustness. We evaluate 18 representative modern open-source VC models and reveal systematic vulnerabilities in content consistency, speaker similarity, long-form stability, post-processing resilience, adversarial robustness, and detector-facing separability. We open-source the toolkit and dataset to support reproducible evaluation and future research.

Figures & tables

Figure 1 : Overview of RVCBench . We benchmark robustness on VCL across the full generation pipeline along four robustness dimensions, i.e., inputs, generation, outputs, and audio perturbations.

Figure 2 : Performance over various accents. (a) WER distribution of each model over accents (violin, log scale; dots: accents, bar: median, hollow: WER

=

0); (b) MCD across accents.

Figure 3 : Prompt length and compression. (a) MCD and (b) SIM versus prompt audio length on LibriTTS; (c) MCD and (d) STOI after each codec.

Figure 4 : Performance under text-prompt shifts. Relative WER changes compared with standard VCTK prompts under (a) hallucination-style prompts and (b) scam-style prompts.

Figure 5 : Performance on long-text generation. (a) MCD and (b) WER on LibriTTS and long-form LibriSpeech.

Figure 6 : Performance on audio-text emotion alignment on RVC-Expression .

Figure 7 : Performance degradation from clean to audios with natural noises. (a) Mean change of each metric vs. clean references (%, negative is worse); (b) per-model change in MCD.

Metric	FishSpeech	Spark-TTS	MOSS-TTSD v0.7	Higgs	OZSpeech	StyleTTS2
EER median [IQR] $\downarrow$	8.00 [5.00, 13.67]	36.33 [30.00, 42.33]	37.00 [26.67, 45.67]	23.00 [18.67, 33.67]	0.67 [0.00, 1.00]	11.67 [6.67, 22.00]

Table 1 : Detector-facing separability for the 18 representative main-paper models. EER (%, lower is more separable) is summarized as the median [first quartile, third quartile] across nine detectors. The IQR describes heterogeneity among detectors, not uncertainty about a voice-cloning model. Detector-wise results for all evaluated models are reported in the appendix.

Figure 8 : Performance on multi-speaker interference. (a) MCD. (b) SIM.

Figure 9 : Performance degradation by adversarial perturbation and anti-protection robustness. (a) SIM under each perturbation (darker: better cloning). (b)-(d) MCD, MOS, and SIM change vs. clean references after SPEC and after DEMUCS denoising (DNS64); markers follow the keys under (a).

Appendix figures & tables46 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Input	Generation			Output		Perturbation
	Shift	Multi- lingual	Long form	Express.	Codec	Detect.	Passive noise	Anti- clone	Denoise
RVCBench (ours)	✓	✓	✓	✓	✓	✓	✓	✓	✓
LibriSpeech-PC [ 34 ]	–	–	–	–	–	–	–	–	–
ClonEval [ 10 ]	–	–	–	$\triangle$	–	–	–	–	–
VCC20 [ 65 ]	–	✓	–	–	–	$\triangle$	–	–	–
Seed-TTS [ 1 ]	–	$\triangle$	–	✓	–	–	–	–	–

Appendix

Table 2 : Comparison with prior VC-, TTS-, and audio-robustness evaluation benchmarks. Unlike prior resources, RVCBench explicitly evaluates robustness across the full voice-cloning pipeline, covering input shifts, challenging generation settings, output robustness, and perturbation-based protection settings.

Model	Model Size	Generation Paradigm	System Class	Core Architecture	Primary Focus	ML Support	Open	Year
Autoregressive zero-shot TTS and voice-cloning LMs
VALL-E [ 52 ]	–	AR (discrete codec tokens)	TTS + voice cloning	Neural codec language model over discrete audio codes	Zero-shot personalized TTS from short acoustic prompt	EN only	No	2023
Seed-TTS [ 1 ]	–	AR (speech tokens)	TTS + voice cloning	Autoregressive Transformer with speech tokenizer, token LM, token diffusion model, and vocoder	Human-like expressive speech, controllable TTS, voice conversion	Multilingual / cross-lingual	No	2024
FishSpeech [ 30 ]	390M	AR (discrete codec tokens)	TTS + voice cloning	Dual autoregressive neural codec LM (fast-slow)	Multilingual TTS, zero-shot cloning	EN, ZH, DE, JA, FR, ES, KO, AR	Yes	2024
XTTS [ 6 ]	443M GPT encoder	AR (discrete speech tokens)	TTS + voice cloning	Tortoise-based multilingual zero-shot TTS model	Massively multilingual zero-shot TTS	16 languages	Yes	2024
Spark-TTS [ 54 ]	0.5B	AR (discrete codec tokens)	TTS + voice cloning	LLM-based neural codec LM with single-stream tokens	Efficient LLM-TTS, zero-shot cloning	EN, ZH	Yes	2025

Appendix

Table 3 : Comparison of recent TTS and speech generation models by generation paradigm and system class , alongside model size, core architecture, primary focus, multilingual (ML) support, open-source status, and release year. (Part a: Autoregressive zero-shot TTS and voice-cloning LMs.)

Model	Model Size	Generation Paradigm	System Class	Core Architecture	Primary Focus	ML Support	Open	Year
Autoregressive speech-token, dialogue, and foundation-audio LMs
IndexTTS [ 13 ]	2.3B	AR (discrete speech tokens)	TTS + voice cloning	XTTS/Tortoise-inspired model with hybrid character+pinyin text modeling, conformer speaker encoder, and BigVGAN2	Controllable, efficient zero-shot TTS	EN, ZH	Yes	2025
MOSS-TTSD v0.7 [ 62 ]	8B	AR (discrete codec tokens)	Dialogue TTS + voice cloning	Dialogue-oriented speech generation model	Long-form spoken dialogue generation with zero-shot cloning	EN, ZH and other mainstream languages	Yes	2026
FireRedTTS-2 [ 57 ]	1.7B	AR (streaming speech tokens)	Dialogue TTS + voice cloning	12.5 Hz streaming speech tokenizer with text-speech interleaved dual Transformer	Long-form multi-speaker dialogue, low-latency streaming, zero-shot voice cloning	EN, ZH, JA, KO, FR, DE, RU	Yes	2025
Higgs Audio [ 4 ]	3B + 2.2B DualFFN	AR (text + audio tokens)	General speech/TTS + voice cloning	Foundation audio model with dual-FFN token processing	General expressive audio generation, multi-speaker speech, and zero-shot cloning	Primarily EN, ZH, KO; includes DE, ES	Yes	2025

Appendix

Table 4 : Comparison of recent TTS and speech generation models ( continued ). (Part b: Autoregressive speech-token, dialogue, and foundation-audio LMs.)

Model	Model Size	Generation Paradigm	System Class	Core Architecture	Primary Focus	ML Support	Open	Year
Non-autoregressive, diffusion, flow, and cloning-pipeline TTS
MaskGCT [ 55 ]	2.2B	NAR (masked semantic + acoustic tokens)	TTS + voice cloning	Two-stage masked generative codec Transformer	Zero-shot TTS without explicit alignment or duration modeling	Multilingual	Yes	2024
StyleTTS2 [ 29 ]	190M	Diffusion	TTS + voice cloning/adaptation	Diffusion-based TTS with adversarial training	Expressive, high-naturalness TTS with reference-style / speaker adaptation	Partial (EN; multilingual via PL-BERT)	Yes	2023
F5-TTS [ 8 ]	335.8M	Flow matching	TTS + voice cloning	Fully non-autoregressive flow-matching TTS with DiT backbone + ConvNeXt text refinement	Faithful, fluent, multilingual zero-shot TTS	Multilingual	Yes	2024
OZSpeech [ 23 ]	145M + 102M FACodec	Flow matching (continuous-time)	TTS + voice cloning	Conditional flow-matching model with learned-prior one-step sampling	Low-latency zero-shot TTS and prompt-speech cloning	EN only	Yes	2025
OpenVoice [ 39 ]	131 MB	Hybrid cloning pipeline	Voice cloning + controllable TTS	Base speaker TTS + tone-color converter for style/voice transfer	Instant voice cloning with flexible style control and cross-lingual cloning	EN, ES, FR, ZH, JA, KO (V2 native)	Yes	2023

Appendix

Table 5 : Comparison of recent TTS and speech generation models ( continued ). (Part c: Non-autoregressive, diffusion, flow, and cloning-pipeline TTS.)

Model	Model Size	Generation Paradigm	System Class	Core Architecture	Primary Focus	ML Support	Open	Year
Continuous-latent, editing, tokenizer-free, and proprietary TTS systems
PlayDiffusion	1.5B	Diffusion	Reference-conditioned TTS + speech editing/inpainting	Diffusion-based speech generation, editing, and audio inpainting	Reference-conditioned synthesis and context-preserving speech edits	N/A (English tokenizer)	Yes	2025
VibeVoice [ 37 ]	1.5B	Latent diffusion / flow (continuous latents)	Long-form TTS + reference-conditioned voice cloning	Next-token diffusion over continuous speech latents	Long-form conversational speech with public voice-prefill conditioning	EN, ZH and expanded experimental voices	Partial	2025
VoxCPM [ 66 ]	0.5B	Tokenizer-free diffusion autoregressive	TTS + voice cloning	MiniCPM-4-based hierarchical semantic-acoustic model with FSQ, residual acoustic modeling, Local DiT decoder, and causal AudioVAE	Context-aware expressive speech generation and true-to-life zero-shot voice cloning	ZH, EN	Yes	2025
GPT-4o mini TTS	–	Proprietary instruction-conditioned neural TTS	TTS only; no public voice cloning	GPT-4o-mini-powered speech endpoint with natural-language style instructions and built-in voices	Steerable text-to-speech, multilingual narration, and realtime/streaming audio output	Multilingual; voices optimized for EN	No	2025
Gemini-TTS	–	Proprietary instruction-conditioned neural TTS	TTS only; no public voice cloning	Gemini-family TTS with prebuilt voice configuration and single-/multi-speaker synthesis	Prompt-controllable speech synthesis for scripts, narration, podcasts, and multi-speaker audio	80+ locales / prebuilt voices	No	2025

Appendix

Table 6 : Comparison of recent TTS and speech generation models ( continued ). (Part d: Continuous-latent, editing, tokenizer-free, and proprietary TTS systems.)

Model	Model Size	Generation Paradigm	System Class	Core Architecture	Primary Focus	ML Support	Open	Year
Hybrid LM + Diffusion or Flow
CosyVoice 2 [ 16 ]	0.5B	Hybrid (LM + causal flow matching)	Streaming TTS + voice cloning	Streaming TTS with LM + chunk-aware causal flow matching	Scalable streaming TTS with zero-shot voice generation and cross-lingual synthesis	ZH, EN, JA, KO, DE, ES, FR, IT, RU	Yes	2024
Qwen3-TTS [ 20 ]	0.6B / 1.7B	Hybrid (dual-track LM + block-wise DiT reconstruction)	Streaming TTS + voice cloning + voice design	Dual-track LM with 25 Hz / 12 Hz speech tokenizers and block-wise DiT waveform reconstruction	Multilingual, controllable, robust, streaming TTS with voice cloning and voice design	ZH, EN, JA, KO, DE, FR, RU, PT, ES, IT	Yes	2026
MGM-Omni [ 51 ]	0.6B / 2B / 4B	Multimodal LLM + speech generation head	Speech/TTS generation + voice cloning	Omni-modal LLM with dual-track speech generation	Agentic, personalized, long-horizon speech generation with streaming zero-shot cloning	EN, ZH	Yes	2025
GLM-TTS [ 11 ]	–	Hybrid (Text $\rightarrow$ Token AR + Token $\rightarrow$ Wav diffusion)	Streaming TTS + voice cloning	Two-stage: text-to-speech-token autoregressive SpeechLM + token-to-waveform diffusion (Flow) + vocoder	Production-level controllable, emotion-expressive zero-shot TTS	ZH, EN (incl. dialect + singing data)	Yes	2025

Appendix

Table 7 : Comparison of recent TTS and speech generation models ( continued ). (Part e: Hybrid LM + Diffusion or Flow.)

Task	Dataset Name	Evaluation	Sources	Preprocess / Setup
Input Robustness
(1) Reference audio shifts	RVC-AudioShift	Demography	VCTK	Demography : VCTK speakers across 12 accents, three age groups, and gender groups.
(2) Text prompt shifts	RVC-TextShift	Hallucination , Scam	VCTK, Robocall	Hallucination : text prompts with special tokens, formatting templates, and mixed-language fragments. Scam : text prompts with persuasive or high-risk robocall-style content.
Generation Robustness
(3) Multilingual generalizability	RVC-Multilingual	Chinese-VC , English-VC , French-VC , CrossLingual	VCTK, LibriTTS, AISHELL-1, Common Voice FR, EMIME	English-VC : VCTK and LibriTTS for English in-domain VCL . Chinese-VC and French-VC : reference audio and text are in Mandarin and French, respectively. CrossLingual : text prompt and reference audio are in different languages.
(4) Long-form generation	RVC-LongContext	LongText , LongAudio	LibriTTS, LibriSpeech-Long	LongText : generate minutes-scale utterances per speaker. LongAudio : use reference audios with different durations to evaluate sensitivity to prompt-audio length.

Appendix

Table 8 : Robustness perspectives and evaluated conditions in RVCBench . The benchmark contains 18 evaluated robustness conditions; the real-audio GroundTruth set is a control for Deepfake and is not counted as a robustness condition.

Dataset	Lang	Spk	Utts	U/S	SR	Speaker overlap
Standard English Benchmarks
VCTK [ 58 ]	EN	40	4,000	100	48k	VB: 4; Robocall: 10
LibriTTS [ 60 ]	EN	40	4,000	100	24k	Multi: 5; Long: 2
Multilingual & Cross-Lingual
AISHELL-1 [ 5 ]	ZH	40	2,000	50	16k	–
Common Voice FR [ 2 ]	FR	40	2,000	50	48k	–

Appendix

Table 9 : Dataset metadata and speaker overlap. Dataset rows contain 14,370 utterance-level evaluation items and sum to 225 speaker entries before deduplication. Accounting for shared speaker identities across derived subsets gives 204 unique speakers.

Metric	What it Measures (Attributes)
Generation quality: Speaker Identity & Content
SIM	Speaker embedding cosine similarity via ECAPA-TDNN verification score (SpeechBrain)
SVA	Speaker verification accept/reject decision (SpeechBrain; converted to boolean)
WER	Linguistic content correctness via Whisper transcription + normalized WER
Generation quality: Acoustic Quality & Naturalness
MCD	Spectral distortion vs. ground truth using DTW-aligned Mel-cepstral distance

Appendix

Table 10 : Taxonomy of evaluation metrics in RVCBench : mapping metrics to speech attributes and task categories.

Figure 10 : Comparison of performance gaps. (a) gap in gender performance. (b) absolute max deviation over age.

Observed vulnerability	Potential future measure	RVCBench component
Accent & demographic sensitivity	Demographically balanced training, speaker-disentangled embeddings, and group-aware calibration	RVC-AudioShift
Noisy or multi-speaker references	Reference-quality scoring, source separation, multi-reference aggregation, and noise augmentation	RVC-PassiveNoise
Irregular or OOD text prompts	Text normalization, prompt sanitization, uncertainty-aware generation, and semantic consistency checking	RVC-TextShift
Long-form content & speaker drift	Hierarchical generation, explicit memory mechanisms, chunk-level consistency constraints, and speaker-consistency regularization	RVC-LongContext
Weak expressive or prosodic control	Prosody supervision, emotion/style disentanglement, and style-consistency objectives	RVC-Expression
Compression & channel sensitivity	Codec-aware training, compression augmentation, differentiable post-processing, and robust vocoder design	RVC-Compression

Appendix

Table 12: Potential robustness-improvement directions suggested by RVCBench .

Dataset	Baseline	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	RTF $\downarrow$	SVA $\uparrow$	EMC $\uparrow$
LibriTTS	FishSpeech	$0.47\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.37\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.17\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$6.47\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	3.61	$0.91\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.68\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	Fish Audio S2	$0.54\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.37\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$6.16\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	4.67	$0.95\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.71\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	OZSpeech	$0.39\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.21\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$6.87\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	8.75	$0.84\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.64\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	StyleTTS2	$0.23\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.30\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$6.81\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$	0.11	$0.39\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.59\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
	Spark-TTS	$0.41\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.33\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$5.83\mkern 1.0mu_{\scriptscriptstyle\pm 0.17}$	1.56	$0.76\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.67\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	MOSS-TTSD v0.7	$0.49\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.10\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.38\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$7.09\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$	0.62	$0.88\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.67\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$

Appendix

Table 13: Performance on LibriTTS and VCTK. Appendix result tables cover the 18 representative main-paper models plus four extension systems: Fish Audio S2, MOSS-TTS v1.5, dots.tts, and Higgs TTS 3.

Model	Direction	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	SVA $\uparrow$
FishSpeech	Eng $\to$ Man	$0.58\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.08\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.40\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$6.20\mkern 1.0mu_{\scriptscriptstyle\pm 0.15}$	$1.00\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$
FishSpeech	Man $\to$ Eng	$0.55\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.36\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.07\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$5.89\mkern 1.0mu_{\scriptscriptstyle\pm 0.17}$	$0.99\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$
Fish Audio S2	Eng $\to$ Man	$0.63\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.11\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.39\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$5.96\mkern 1.0mu_{\scriptscriptstyle\pm 0.16}$	$1.00\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$
Fish Audio S2	Man $\to$ Eng	$0.61\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.37\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.07\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$5.70\mkern 1.0mu_{\scriptscriptstyle\pm 0.18}$	$1.00\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$
OZSpeech	Eng $\to$ Man	$0.03\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$1.60\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$1.00\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$14.44\mkern 1.0mu_{\scriptscriptstyle\pm 0.45}$	$0.01\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$
OZSpeech	Man $\to$ Eng	$0.30\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.19\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.07\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$6.77\mkern 1.0mu_{\scriptscriptstyle\pm 0.19}$	$0.72\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$

Appendix

Table 14: Performance on Bilingual. Evaluation of cross-lingual synthesis between English (Eng) and Mandarin (Man).

Accent	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	EMC $\uparrow$
Model: FishSpeech
American	$0.42\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.31\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.03\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.01\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$	$0.83\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$
Australian	$0.43\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.23\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$3.58\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.98\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$
British	$0.62\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.35\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.03\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$3.75\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.72\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$
Canadian	$0.43\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$4.26\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$3.54\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	$0.83\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
English	$0.44\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.25\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.03\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$4.35\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$	$0.76\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$

Appendix

Table 15: Accent Robustness (Part 1). Performance evaluation across 12 English accents. N/A indicates that speaker-level uncertainty is not estimable for an accent represented by one speaker.

Accent	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	EMC $\uparrow$
Model: Spark-TTS
American	$0.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$3.82\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.21\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$3.70\mkern 1.0mu_{\scriptscriptstyle\pm 0.14}$	$0.89\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$
Australian	$0.58\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$3.84\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.12\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.35\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.98\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$
British	$0.48\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.00\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.14\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$3.60\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.50\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$
Canadian	$0.48\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$3.89\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.28\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	$3.79\mkern 1.0mu_{\scriptscriptstyle\pm 0.35}$	$0.87\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
English	$0.52\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.82\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$0.21\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$3.96\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$0.83\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$

Appendix

Table 16: Accent Robustness (Part 2). Performance evaluation across 12 English accents. N/A indicates that speaker-level uncertainty is not estimable for an accent represented by one speaker.

Accent	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	EMC $\uparrow$
Model: Higgs
American	$0.52\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$4.08\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$	$0.19\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$	$3.73\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$	$0.85\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$
Australian	$0.53\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.09\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.09\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$3.75\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.96\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$
British	$0.66\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.30\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.34\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$3.49\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.76\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$
Canadian	$0.47\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$4.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$0.43\mkern 1.0mu_{\scriptscriptstyle\pm 0.13}$	$3.36\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$	$0.84\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
English	$0.54\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.19\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.08\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.30\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	$0.78\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$

Appendix

Table 17: Accent Robustness (Part 3). Performance evaluation across 12 English accents. N/A indicates that speaker-level uncertainty is not estimable for an accent represented by one speaker.

Accent	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	EMC $\uparrow$
Model: VibeVoice
American	$0.42\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$3.98\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$0.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$5.14\mkern 1.0mu_{\scriptscriptstyle\pm 0.13}$	$0.86\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$
Australian	$0.44\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$3.90\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.07\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.55\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.98\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$
British	$0.60\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.23\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.04\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.14\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.54\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$
Canadian	$0.45\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$3.96\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.48\mkern 1.0mu_{\scriptscriptstyle\pm 0.19}$	$0.85\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$
English	$0.46\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.95\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$0.07\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$5.19\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$	$0.79\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$

Appendix

Table 18: Accent Robustness (Part 4). Performance evaluation across 12 English accents. N/A indicates that speaker-level uncertainty is not estimable for an accent represented by one speaker.

Accent	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	EMC $\uparrow$
Model: F5-TTS
American	$0.54\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.12\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$5.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.27}$	$0.81\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$
Australian	$0.56\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$3.92\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.02\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.40\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.98\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$
British	$0.68\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.28\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.00\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.31\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.66\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$
Canadian	$0.53\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$3.98\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$	$0.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.47\mkern 1.0mu_{\scriptscriptstyle\pm 0.36}$	$0.86\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
English	$0.55\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.14\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$4.88\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	$0.73\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$

Appendix

Table 19: Accent Robustness (Part 5). Performance evaluation across 12 English accents. N/A indicates that speaker-level uncertainty is not estimable for an accent represented by one speaker.

Accent	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	EMC $\uparrow$
Model: IndexTTS
American	$0.58\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.69\mkern 1.0mu_{\scriptscriptstyle\pm 0.18}$	$0.81\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$
Australian	$0.55\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$3.84\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.49\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.96\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$
British	$0.66\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$4.27\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.01\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$3.98\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$	$0.68\mkern 1.0mu_{\scriptscriptstyle\pm\mathrm{N/A}}$
Canadian	$0.58\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$3.99\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.21\mkern 1.0mu_{\scriptscriptstyle\pm 0.17}$	$0.84\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$
English	$0.57\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.03\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$5.11\mkern 1.0mu_{\scriptscriptstyle\pm 0.15}$	$0.82\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$

Appendix

Table 20: Accent Robustness (Part 6). Performance evaluation across 12 English accents. N/A indicates that speaker-level uncertainty is not estimable for an accent represented by one speaker.

Gender	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	EMC $\uparrow$
Model: FishSpeech
Female	$0.43\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.30\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.03\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$4.24\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	$0.71\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
Male	$0.43\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.23\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.03\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$3.95\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$	$0.83\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
Model: Fish Audio S2
Female	$0.51\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.30\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.01\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$5.11\mkern 1.0mu_{\scriptscriptstyle\pm 0.14}$	$0.77\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
Male	$0.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.23\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.01\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$4.62\mkern 1.0mu_{\scriptscriptstyle\pm 0.17}$	$0.90\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$

Appendix

Table 21: Performance on different gender groups (Part 1). Performance evaluation breakdown by gender (F/M).

Gender	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	EMC $\uparrow$
Model: GLM-TTS
Female	$0.58\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$4.09\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	$0.69\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
Male	$0.57\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.96\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$3.77\mkern 1.0mu_{\scriptscriptstyle\pm 0.14}$	$0.81\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
Model: VibeVoice
Female	$0.44\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.98\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$5.28\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$	$0.76\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
Male	$0.43\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.91\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.85\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	$0.85\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$

Appendix

Table 22: Performance on different gender groups (Part 2). Performance evaluation breakdown by gender (F/M).

Age Group	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	EMC $\uparrow$
Model: FishSpeech
under 20	$0.43\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.29\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.40}$	$0.69\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$
20–29	$0.43\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.27\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.03\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$4.12\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$	$0.78\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
30 and over	$0.39\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.20\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$0.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$	$0.72\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$
Model: Fish Audio S2
under 20	$0.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.28\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.01\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$4.78\mkern 1.0mu_{\scriptscriptstyle\pm 0.45}$	$0.76\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$

Appendix

Table 23: Performance over different age groups (Part 1).

Age Group	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	EMC $\uparrow$
Model: GLM-TTS
under 20	$0.56\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.03\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$	$0.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$3.61\mkern 1.0mu_{\scriptscriptstyle\pm 0.46}$	$0.71\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$
20–29	$0.57\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$3.98\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$	$0.75\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
30 and over	$0.57\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$3.88\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$3.95\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.70\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$
Model: VibeVoice
under 20	$0.41\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$3.89\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$5.51\mkern 1.0mu_{\scriptscriptstyle\pm 0.36}$	$0.76\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$

Appendix

Table 24: Performance over different age groups (Part 2).

Model	Group	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	SVA $\uparrow$	EMC $\uparrow$	EmTXT $\uparrow$	DNSMOS $\uparrow$	Dur (s)
FishSpeech	robocall	$0.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.37\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.96\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.58\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$	$1.99\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$3.48\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$21.95\mkern 1.0mu_{\scriptscriptstyle\pm 0.26}$
FishSpeech	vctk	$0.40\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.29\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.89\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.75\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$	$2.93\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$3.30\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$2.33\mkern 1.0mu_{\scriptscriptstyle\pm 0.14}$
Fish Audio S2	robocall	$0.57\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.33\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.08\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.99\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$0.64\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$	$2.08\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$3.44\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$24.85\mkern 1.0mu_{\scriptscriptstyle\pm 0.27}$
Fish Audio S2	vctk	$0.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.26\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.01\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.95\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.78\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$3.00\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$3.28\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$2.30\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$
OZSpeech	robocall	$0.36\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$2.98\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.07\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$0.83\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$0.34\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$1.94\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$3.28\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$22.35\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$
OZSpeech	vctk	$0.24\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$3.15\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.02\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.49\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$	$0.70\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$2.96\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$2.82\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$2.07\mkern 1.0mu_{\scriptscriptstyle\pm 0.13}$

Appendix

Table 25 : Performance over the fraud dataset.

	SQ-LLM			AASIST			HuBERT-ECAPA
Generated model	mDCF $\downarrow$	EER $\downarrow$	ACC $\uparrow$	mDCF $\downarrow$	EER $\downarrow$	ACC $\uparrow$	mDCF $\downarrow$	EER $\downarrow$	ACC $\uparrow$
Fish.	34.33	18.33	81.83	26.00	13.67	86.17	10.00	5.00	94.83
Fish S2	15.33	8.33	91.83	6.67	3.33	96.50	5.00	2.67	97.50
OZS.	4.00	2.33	97.50	0.00	0.00	99.83	1.33	1.00	98.83
Style.	30.33	15.33	84.83	52.67	27.33	72.83	7.00	4.00	96.17
Spark.	76.33	42.00	58.17	82.00	48.00	52.17	57.33	29.33	70.50

Appendix

Table 26 : Full deepfake detection results for generated speech models under detection methods: SQ-LLM, AASIST, HuBERT-ECAPA.

	RawGAT-ST			RawNet2			TCM-ADD
Generated model	mDCF $\downarrow$	EER $\downarrow$	ACC $\uparrow$	mDCF $\downarrow$	EER $\downarrow$	ACC $\uparrow$	mDCF $\downarrow$	EER $\downarrow$	ACC $\uparrow$
Fish.	7.67	5.33	94.83	14.33	8.00	92.17	8.00	4.67	95.50
Fish S2	4.33	3.00	97.17	13.33	7.00	93.17	1.33	1.00	98.83
OZS.	0.00	0.00	99.83	2.00	1.00	98.83	0.00	0.00	99.83
Style.	21.33	11.67	88.17	42.00	22.00	78.17	7.00	3.67	96.50
Spark.	64.00	36.33	63.83	77.00	46.00	53.83	63.67	36.33	63.83

Appendix

Table 27 : Full deepfake detection results for generated speech models under detection methods: RawGAT-ST, RawNet2, TCM-ADD.

	XLSR-SLS			Wav2Vec2-ECAPA			WavLM-ECAPA
Generated model	mDCF $\downarrow$	EER $\downarrow$	ACC $\uparrow$	mDCF $\downarrow$	EER $\downarrow$	ACC $\uparrow$	mDCF $\downarrow$	EER $\downarrow$	ACC $\uparrow$
Fish.	8.33	4.67	95.50	33.67	17.00	83.17	15.67	8.00	91.83
Fish S2	5.67	4.33	95.83	23.33	11.67	88.17	11.67	6.00	94.17
OZS.	0.00	0.00	99.83	24.00	13.67	86.17	0.67	0.67	99.17
Style.	12.67	6.67	93.50	74.67	53.00	47.17	14.00	7.33	92.83
Spark.	75.00	42.33	57.83	59.00	30.00	69.83	58.67	29.67	70.50

Appendix

Table 28 : Full deepfake detection results for generated speech models under detection methods: XLSR-SLS, Wav2Vec2-ECAPA, WavLM-ECAPA.

Model	Noise	SIM	MOS	WER	MCD	EMC	DNSMOS
FishSpeech	-5dB	$0.20\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.26\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$8.40\mkern 1.0mu_{\scriptscriptstyle\pm 0.17}$	$0.58\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$3.32\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	+0dB	$0.31\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$4.27\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$7.21\mkern 1.0mu_{\scriptscriptstyle\pm 0.19}$	$0.60\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$3.31\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	+5dB	$0.40\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$4.28\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$6.67\mkern 1.0mu_{\scriptscriptstyle\pm 0.19}$	$0.67\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$3.32\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	+10dB	$0.46\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$4.33\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$6.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.20}$	$0.69\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$3.33\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
Fish Audio S2	-5dB	$0.29\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$4.36\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$7.51\mkern 1.0mu_{\scriptscriptstyle\pm 0.22}$	$0.62\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$3.33\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$
	+0dB	$0.43\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$4.38\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.03\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$6.79\mkern 1.0mu_{\scriptscriptstyle\pm 0.16}$	$0.66\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$	$3.31\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$

Appendix

Table 29 : Performance in multi-speaker noisy environments (Part 1).

Model	Noise	SIM	MOS	WER	MCD	EMC	DNSMOS
MOSS-TTS v1.5	-5dB	$0.20\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.33\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$8.62\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$	$0.66\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$3.19\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
	+0dB	$0.32\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$4.33\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$7.62\mkern 1.0mu_{\scriptscriptstyle\pm 0.22}$	$0.64\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$3.20\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
	+5dB	$0.45\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$4.31\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$7.22\mkern 1.0mu_{\scriptscriptstyle\pm 0.19}$	$0.68\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$3.19\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	+10dB	$0.54\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$4.32\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$7.00\mkern 1.0mu_{\scriptscriptstyle\pm 0.18}$	$0.61\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$3.23\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
dots.tts	-5dB	$0.24\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$3.64\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$0.30\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$8.23\mkern 1.0mu_{\scriptscriptstyle\pm 0.33}$	$0.62\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$3.10\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
	+0dB	$0.37\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$3.62\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	$0.16\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$7.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.29}$	$0.64\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$3.11\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$

Appendix

Table 30 : Performance in multi-speaker noisy environments (Part 2).

Model	Noise	SIM	MOS	WER	MCD	EMC	DNSMOS
VibeVoice	-5dB	$0.18\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$3.35\mkern 1.0mu_{\scriptscriptstyle\pm 0.16}$	$0.14\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$8.09\mkern 1.0mu_{\scriptscriptstyle\pm 0.31}$	$0.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$3.07\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$
	+0dB	$0.27\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$3.30\mkern 1.0mu_{\scriptscriptstyle\pm 0.17}$	$0.15\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$7.87\mkern 1.0mu_{\scriptscriptstyle\pm 0.26}$	$0.55\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$3.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$
	+5dB	$0.34\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$3.25\mkern 1.0mu_{\scriptscriptstyle\pm 0.17}$	$0.17\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$7.45\mkern 1.0mu_{\scriptscriptstyle\pm 0.28}$	$0.48\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$3.01\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$
	+10dB	$0.41\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$3.42\mkern 1.0mu_{\scriptscriptstyle\pm 0.13}$	$0.12\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$7.16\mkern 1.0mu_{\scriptscriptstyle\pm 0.19}$	$0.60\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$3.10\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$
MGM-Omni	-5dB	$0.16\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$3.56\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.33\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$8.77\mkern 1.0mu_{\scriptscriptstyle\pm 0.27}$	$0.58\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$2.85\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$
	+0dB	$0.26\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$3.70\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$	$0.28\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$7.84\mkern 1.0mu_{\scriptscriptstyle\pm 0.32}$	$0.58\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$2.99\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$

Appendix

Table 31 : Performance in multi-speaker noisy environments (Part 3).

Model	Noise	SIM	MOS	WER	MCD	EMC	DNSMOS
OpenVoice	-5dB	$0.13\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.24\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$7.64\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	$0.59\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$3.39\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$
	+0dB	$0.16\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.23\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$7.44\mkern 1.0mu_{\scriptscriptstyle\pm 0.13}$	$0.62\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$3.39\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	+5dB	$0.19\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$4.24\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$7.28\mkern 1.0mu_{\scriptscriptstyle\pm 0.13}$	$0.62\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$3.37\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$
	+10dB	$0.22\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$4.26\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$7.25\mkern 1.0mu_{\scriptscriptstyle\pm 0.14}$	$0.65\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$3.39\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$
XTTS	-5dB	$0.15\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.54\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$10.64\mkern 1.0mu_{\scriptscriptstyle\pm 0.15}$	$0.48\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$3.18\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
	+0dB	$0.23\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$3.44\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$	$0.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$10.48\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	$0.47\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$3.09\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$

Appendix

Table 32 : Performance in multi-speaker noisy environments (Part 4).

Model	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	RTF $\downarrow$	SVA $\uparrow$
FishSpeech	$0.35\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.37\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.33\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.24$	$0.83\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
Fish Audio S2	$0.41\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$4.37\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.32\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$7.39$	$0.91\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
OZSpeech	$0.28\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$3.35\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$0.59\mkern 1.0mu_{\scriptscriptstyle\pm 0.17}$	$8.21$	$0.65\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$
StyleTTS2	$0.18\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.71\mkern 1.0mu_{\scriptscriptstyle\pm 0.21}$	$0.10$	$0.23\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$
Spark-TTS	$0.34\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$3.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.14}$	$0.81\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$1.60$	$0.64\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$
MOSS-TTSD v0.7	$0.42\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.99\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	$0.90$	$0.90\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$

Appendix

Table 33: Performance on Text Hallucination (Hallucination-VCTK). WER uses the assigned text; SIM/SVA use same-speaker VCTK audio. No text-matched target waveform exists, so MCD/EMC are omitted.

Protection	SNR	MCD	SIM	WER	SpeechMOS	DNSMOS
Gaussian Noise	6.10	11.25	0.71	0.19	1.39	1.98
SPEC	8.08	12.67	0.59	0.34	1.40	1.97
POP	17.08	2.08	0.92	0.18	3.00	3.02
SafeSpeech	9.48	11.51	0.62	0.26	1.48	2.06
Enkidu	8.07	8.03	0.72	0.24	1.52	2.26
Denoise-SPEC	15.30	3.39	0.51	0.29	3.23	3.15

Appendix

Table 34 : Evaluation of the protected audio on LibriTTS.

Protection	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	RTF $\downarrow$	SVA $\uparrow$	EMC $\uparrow$
Model: FishSpeech
Gaussian Noise	$0.23\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$2.90\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.08\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$10.84\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	$4.08$	$0.45\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.56\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
POP	$0.01\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$1.53\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$1.00\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$11.89\mkern 1.0mu_{\scriptscriptstyle\pm 0.32}$	$2.36$	$0.00\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$0.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$
Enkidu	$0.33\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.21\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$7.27\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$	$2.56$	$0.67\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.65\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
SafeSpeech	$0.24\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.15\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.08\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$9.60\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$	$2.03$	$0.47\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.57\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
SPEC	$0.21\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.10\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.08\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$9.99\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$	$4.16$	$0.39\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.56\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$

Appendix

Table 35: Protected Audio VC Evaluation (Part 1). Comparison of protection methods on LibriTTS (Part 1 of 3).

Protection	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	RTF $\downarrow$	SVA $\uparrow$	EMC $\uparrow$
Model: Higgs
Gaussian Noise	$0.27\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$2.55\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.34\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$10.03\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$	$3.37$	$0.56\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.52\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
POP	$0.52\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.24\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.08\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$6.39\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	$1.33$	$0.89\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.69\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
Enkidu	$0.43\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.15\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$7.08\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	$3.17$	$0.81\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.65\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
SafeSpeech	$0.26\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$2.47\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.31\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$11.07\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	$1.40$	$0.54\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.45\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
SPEC	$0.24\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$2.43\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.32\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$11.41\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	$4.63$	$0.47\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.45\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$

Appendix

Table 36: Protected Audio VC Evaluation (Part 2). Comparison of protection methods on LibriTTS (Part 2 of 3).

Protection	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	RTF $\downarrow$	SVA $\uparrow$	EMC $\uparrow$
Model: F5-TTS
Gaussian Noise	$0.14\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$2.56\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.20\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$17.78\mkern 1.0mu_{\scriptscriptstyle\pm 0.26}$	$0.27$	$0.16\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.52\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
POP	$0.52\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.96\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.11\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$7.17\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$	$0.40$	$0.88\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.67\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
Enkidu	$0.43\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.70\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.12\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$7.15\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	$0.29$	$0.81\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.65\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
SafeSpeech	$0.21\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$2.96\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.15\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$14.56\mkern 1.0mu_{\scriptscriptstyle\pm 0.25}$	$0.36$	$0.36\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.55\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
SPEC	$0.18\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$2.80\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.17\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$15.76\mkern 1.0mu_{\scriptscriptstyle\pm 0.27}$	$0.36$	$0.27\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.55\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$

Appendix

Table 37: Protected Audio VC Evaluation (Part 3). Comparison of protection methods on LibriTTS (Part 3 of 3).

Model	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	RTF $\downarrow$	SVA $\uparrow$	EMC $\uparrow$
Dataset: Long LibriSpeech
FishSpeech	$0.57\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$4.40\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.66\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$12.36\mkern 1.0mu_{\scriptscriptstyle\pm 0.80}$	$3.03$	$1.00\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$0.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$
Fish Audio S2	$0.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$4.09\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$	$0.03\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$8.83\mkern 1.0mu_{\scriptscriptstyle\pm 0.72}$	$4.47$	$0.90\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$0.55\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$
OZSpeech	$0.42\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$2.10\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	$0.07\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$9.86\mkern 1.0mu_{\scriptscriptstyle\pm 0.94}$	$1.47$	$0.95\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.45\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$
StyleTTS2	$0.20\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$4.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.73\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$12.90\mkern 1.0mu_{\scriptscriptstyle\pm 0.67}$	$0.03$	$0.35\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$	$0.25\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$
Spark-TTS	$0.35\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$3.10\mkern 1.0mu_{\scriptscriptstyle\pm 0.20}$	$0.91\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$14.64\mkern 1.0mu_{\scriptscriptstyle\pm 1.20}$	$1.26$	$0.65\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	$0.40\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$

Appendix

Table 38: Long-Context Generation. Performance evaluation on Long LibriSpeech dataset.

Model	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	RTF $\downarrow$	SVA $\uparrow$
Dataset: Common Voice (French)
FishSpeech	$0.37\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.20\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.34\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$5.71\mkern 1.0mu_{\scriptscriptstyle\pm 0.27}$	$1.94$	$0.77\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
Fish Audio S2	$0.46\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.22\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.26\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$6.03\mkern 1.0mu_{\scriptscriptstyle\pm 0.26}$	$9.28$	$0.89\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
OZSpeech	$0.11\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$2.46\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.99\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$9.24\mkern 1.0mu_{\scriptscriptstyle\pm 0.45}$	$17.50$	$0.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
StyleTTS2	$0.11\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.72\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.70\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$7.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.28}$	$0.11$	$0.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
Spark-TTS	$0.16\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$2.85\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$1.66\mkern 1.0mu_{\scriptscriptstyle\pm 0.13}$	$6.37\mkern 1.0mu_{\scriptscriptstyle\pm 0.30}$	$1.16$	$0.35\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$

Appendix

Table 39: Multilingual Generalization. Performance on French (Common Voice) and Mandarin (AISHELL-1) datasets.

Dataset	Model	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	RTF $\downarrow$	SVA $\uparrow$	EMC $\uparrow$
LibriTTS (DEMUCS on SPEC)	FishSpeech	$0.29\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.15\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.07\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$6.65\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	$2.42$	$0.64\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.63\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	Fish Audio S2	$0.32\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.13\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$6.48\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	$1.01$	$0.73\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.66\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	OZSpeech	$0.26\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.21\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.06\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$6.84\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	$12.83$	$0.53\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.62\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	StyleTTS2	$0.19\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.31\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$6.71\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$	$0.11$	$0.24\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.57\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	Spark-TTS	$0.09\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.47\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$1.30\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$3.83\mkern 1.0mu_{\scriptscriptstyle\pm 0.14}$	$2.06$	$0.20\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.54\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
	MOSS-TTSD v0.7	$0.29\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.72\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.29\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$7.22\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$	$1.70$	$0.62\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.61\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$

Appendix

Table 40: DEMUCS-on-SPEC performance on LibriTTS.

Model	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	RTF $\downarrow$	SVA $\uparrow$	EMC $\uparrow$
Condition: Background-VCTK (Clean)
FishSpeech	$0.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.28\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.01\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$5.36\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$	$4.08$	$0.95\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.82\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
Fish Audio S2	$0.58\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.27\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.01\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$5.56\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	$25.26$	$1.00\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$0.91\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
OZSpeech	$0.27\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$3.31\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.01\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$7.41\mkern 1.0mu_{\scriptscriptstyle\pm 0.14}$	$24.83$	$0.57\mkern 1.0mu_{\scriptscriptstyle\pm 0.07}$	$0.78\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$
StyleTTS2	$0.20\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.32\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.01\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$6.07\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	$0.54$	$0.27\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.55\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$
Spark-TTS	$0.59\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$3.89\mkern 1.0mu_{\scriptscriptstyle\pm 0.09}$	$0.12\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$4.82\mkern 1.0mu_{\scriptscriptstyle\pm 0.17}$	$1.97$	$0.86\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.91\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$

Appendix

Table 41: Performance on Natural Background Noises. Evaluation on Background-VCTK under clean and noisy conditions.

Model	Condition	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	RTF $\downarrow$	SVA $\uparrow$	EMC $\uparrow$
FishSpeech	Clean	$0.47\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.37\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.17\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$6.47\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	3.61	$0.91\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.68\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	SPEC	$0.21\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.10\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.08\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$9.99\mkern 1.0mu_{\scriptscriptstyle\pm 0.08}$	4.16	$0.39\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.56\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
	DEMUCS	$0.29\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.15\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.07\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$6.65\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	2.42	$0.64\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.63\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
Fish Audio S2	Clean	$0.54\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.37\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.04\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$6.16\mkern 1.0mu_{\scriptscriptstyle\pm 0.10}$	4.67	$0.95\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.71\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	SPEC	$0.30\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$2.44\mkern 1.0mu_{\scriptscriptstyle\pm 0.06}$	$0.08\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$11.10\mkern 1.0mu_{\scriptscriptstyle\pm 0.14}$	1.27	$0.63\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.47\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
	DEMUCS	$0.32\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.13\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$0.05\mkern 1.0mu_{\scriptscriptstyle\pm 0.00}$	$6.48\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	1.01	$0.73\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.66\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$

Appendix

Table 42 : Denoising performance on SPEC with DEMUCS.

Model	Condition	SIM $\uparrow$	MOS $\uparrow$	WER $\downarrow$	MCD $\downarrow$	RTF $\downarrow$	SVA $\uparrow$	EMC $\uparrow$
GLM-TTS	Clean	$0.57\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$4.08\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.09\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$6.41\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	1.74	$0.95\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.68\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	SPEC	$0.31\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$2.64\mkern 1.0mu_{\scriptscriptstyle\pm 0.05}$	$0.18\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$9.07\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$	1.15	$0.65\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.51\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
	DEMUCS	$0.32\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.41\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.12\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$6.60\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	4.61	$0.71\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.62\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
VibeVoice	Clean	$0.48\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.83\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.23\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$	$6.76\mkern 1.0mu_{\scriptscriptstyle\pm 0.12}$	1.86	$0.85\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$0.62\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$
	SPEC	$0.25\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$2.63\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.20\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$9.64\mkern 1.0mu_{\scriptscriptstyle\pm 0.14}$	2.35	$0.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.50\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$
	DEMUCS	$0.32\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$3.52\mkern 1.0mu_{\scriptscriptstyle\pm 0.04}$	$0.18\mkern 1.0mu_{\scriptscriptstyle\pm 0.01}$	$6.94\mkern 1.0mu_{\scriptscriptstyle\pm 0.11}$	1.06	$0.69\mkern 1.0mu_{\scriptscriptstyle\pm 0.03}$	$0.57\mkern 1.0mu_{\scriptscriptstyle\pm 0.02}$

Appendix

Table 43 : Denoising performance on SPEC with DEMUCS (continued).

Model	STOI $\uparrow$	MCD $\downarrow$	SIM $\uparrow$	WER $\downarrow$
Codec: AAC 32k
FishSpeech	$0.987$	$1.22$	$0.884$	$0.028$
Fish Audio S2	$0.985$	$0.99$	$0.894$	$0.013$
OZSpeech	$0.994$	$0.91$	$0.918$	$0.015$
StyleTTS2	$0.984$	$1.08$	$0.896$	$0.020$
Spark-TTS	$0.966$	$0.73$	$0.948$	$0.093$

Appendix

Table 44: Compression Robustness (Part 1). Evaluation under AAC compression.

Model	STOI $\uparrow$	MCD $\downarrow$	SIM $\uparrow$	WER $\downarrow$
Codec: MP3 32k
FishSpeech	$0.992$	$1.06$	$0.845$	$0.013$
Fish Audio S2	$0.991$	$0.89$	$0.850$	$0.017$
OZSpeech	$0.994$	$1.15$	$0.905$	$0.015$
StyleTTS2	$0.992$	$0.99$	$0.924$	$0.025$
Spark-TTS	$0.968$	$0.87$	$0.928$	$0.149$

Appendix

Table 45: Compression Robustness (Part 2). Evaluation under MP3 compression.

Model	STOI $\uparrow$	MCD $\downarrow$	SIM $\uparrow$	WER $\downarrow$
Codec: Opus 16k
FishSpeech	$0.982$	$1.47$	$0.932$	$0.025$
Fish Audio S2	$0.981$	$1.33$	$0.932$	$0.013$
OZSpeech	$0.987$	$1.52$	$0.937$	$0.012$
StyleTTS2	$0.979$	$1.42$	$0.918$	$0.020$
Spark-TTS	$0.877$	$1.12$	$0.947$	$0.138$

Appendix

Table 46: Compression Robustness (Part 3). Evaluation under Opus compression.

Model	STOI $\uparrow$	MCD $\downarrow$	SIM $\uparrow$	WER $\downarrow$
Codec: Phone NB
FishSpeech	$0.987$	$3.79$	$0.667$	$0.019$
Fish Audio S2	$0.987$	$3.49$	$0.701$	$0.011$
OZSpeech	$0.984$	$4.25$	$0.697$	$0.010$
StyleTTS2	$0.983$	$3.06$	$0.686$	$0.019$
Spark-TTS	$0.966$	$3.37$	$0.680$	$0.175$

Appendix

Table 47: Compression Robustness (Part 4). Evaluation under narrowband telephone transmission.

Explore similar work

May 15, 2026cs.SD

Voice "Cloning" is Style Transfer

Artificially generated speech is increasingly embedded in everyday life. Voice cloning in particular enables applications where identity preservation is important, such as completing a recording, dubbing in a new language, or preserving the voices of individuals with speech loss. However, in our work, we find that despite the term, voice cloning does not faithfully ''clone'' an individual's voice. Instead, we find that widely-used voice cloning models systematically apply style transfer to source voices. As rated by human annotators, cloned voices are perceived as more authoritative, warm, customer-service-like, and human-like compared to their sources. Human annotators also report greater trust in cloned voices than source voices, and a greater willingness to disclose sensitive personal information to them. Our work furthermore shows that voice cloning leads to homogenization of speaker characteristics, as measured by reduced variance in accent, speaking rate, and the audio embedding space. Together, our results highlight a new set of limitations and risks of voice cloning technology and their potential impact on human behavior.

Kaitlyn Zhou, Federico Bianchi, Martijn Bartelds +3

Cornell University · TogetherAI · Stanford University

Cross-Lingual Voice Cloning Voice Conversion Speaker Style Human Annotators

Ask PDF

Oct 13, 2025cs.SD

VCB Bench: An Evaluation Benchmark for Audio-Grounded Large Language Model Conversational Agents

Recent advances in large audio language models (LALMs) have greatly enhanced multimodal conversational systems. However, existing benchmarks remain limited -- they are mainly English-centric, rely on synthetic speech, and lack comprehensive, discriminative evaluation across multiple dimensions. To address these gaps, we present Voice Chat Bot Bench (VCB Bench) -- a high-quality Chinese benchmark built entirely on real human speech. VCB Bench evaluates LALMs from three complementary perspectives: instruction following (including speech-level control beyond text commands), knowledge understanding (general knowledge, reasoning, and daily dialogue), and robustness (stability under perturbations in content, environment, and speaker traits). Experiments on representative LALMs reveal notable performance gaps and highlight future directions for improvement. VCB Bench provides a reproducible and fine-grained evaluation framework, offering standardized methodology and practical insights for advancing Chinese voice conversational models.

Jiliang Hu, Wenfu Wang, Zuchao Li +6

School of Artificial Intelligence, Wuhan University, Wuhan, China · Tencent AI Lab, Beijing, China

Dialogue Benchmarks Large Audio Language Models Evaluation Benchmarks

Ask PDF

Jun 18, 2026eess.AS

Low-Burden Data Augmentation for Dysarthric ASR via Zero-Shot Voice Cloning

Automatic speech recognition remains unreliable for dysarthric speech due to data scarcity and high inter-speaker variability. While synthetic data can address these gaps, traditional methods often require extensive speaker-specific data, reintroducing the collection bottleneck. We investigate zero-shot voice cloning as a low-burden augmentation strategy, using Higgs Audio V2 to clone speakers in the TORGO dataset. We fine-tune (FT) Whisper-medium on cloned, real, and hybrid data and evaluate on held-out real speech. Compared to the zero-shot (31.62%), Clone FT achieved a competitive 26.00% WER, nearly matching the 24.44% and 25.12% seen with Real and Hybrid FT, respectively. Notably, Clone and Hybrid FT outperform Real FT for moderate-severe speakers. Clone FT achieves the best results (11.45% relative) in cross-corpus evaluation on the SAP-1102. These results suggest that zero-shot cloning provides scalable training data that circumvents the costly data collection bottleneck.

Satwinder Singh, Qianli Wang, Zihan Zhong +4

DeepNet Discovery Network, University of Auckland, New Zealand · University of Illinois Urbana-Champaign, USA

Cross-Lingual Voice Cloning Dysarthric Speech Data Augmentation Zero-Shot

Ask PDF