Reliable evaluation of modern zero-shot text-to-speech (TTS) models remains challenging. Subjective tests are costly and hard to reproduce, while objective metrics often saturate, failing to distinguish SOTA systems. To address this, we propose Iterate to Differentiate (I2D), an evaluation framework that recursively synthesizes speech using the model's own outputs as references. Higher-quality models exhibit greater resilience to the distributional shift induced by iterative synthesis, resulting in slower performance degradation. I2D exploits this differential degradation to amplify performance gaps and reveal robustness. By aggregating objective metrics across iterations, I2D improves discriminability and alignment with human judgments, increasing system-level SRCC from 0.118 to 0.464 for UTMOSv2. Experiments on 11 models across Chinese, English, and emotion datasets demonstrate that I2D enables more reliable automated evaluation for zero-shot TTS.
Figures & tables
Figure 1: The overall workflow of our evaluation. The dashed arrows indicate that, after the first synthesis, the reference wav and reference text are updated using the target wav and target text. The objective metrics EMO_F1 are applied only to the Emotion dataset.
Type
Metrics / Dimensions
Objective
WER/CER
SIM
DNSMOS
UTMOSv2
Emotion F1
Subjective
Content Accuracy
Table 1: Objective evaluation metrics and subjective evaluation dimensions
Figure 2: SRCC between objective metrics and subjective dimensions at utterance and system levels (1st and 10th iterations). Utterance-level SRCC is calculated from all individual sample scores, while system-level is derived from model rankings. Specifically, we compare SIM with Spk. Consistency, 1-CER with Content Acc., and UTMOSv2/DNSMOS with Naturalness.
Figure 3: Bar chart of objective metrics for all models on Chinese dataset at the 1st and 10th iterations. The dashed lines indicate the values for real audio.
Architecture
Model
AR
FireRedTTS2 [ 22 ]
Qwen3-TTS [ 4 ]
VoxCPM1.5 [ 34 ]
NAR
F5-TTS [ 20 ]
MaskGCT [ 21 ]
Hybrid
CosyVoice [ 35 ]
Table 2: Evaluated TTS Models and Architectures
Score Method
SIM
1−CER
Iter1(baseline)
0.6818
0.5103
Mean
0.6818
0.4829
LWA
0.6818
0.5194
EWA
0.7273
0.4282
AUC
0.6818
0.4282
Score Method
UTMOSv2
DNSMOS
Table 3: System-level SRCC between aggregated metrics and human evaluation on first-iteration synthesized speech. The correlation of objective metrics computed on first-iteration outputs is included as the baseline.
TTS Models
Objective Metrics (en/zh)
Subjective Metrics (iter1 / iter10)
WER / CER ↓
SIM ↑
UTMOSv2 ↑
Content Acc. ↑
Spk. Consistency ↑
Naturalness ↑
CosyVoice
8.88 / 11.18
19.34 / 23.70
2.98 / 2.76
4.75 / 3.25
3.48 / 1.01
3.93 / 1.60
CosyVoice2
2.77 / 1.51
61.34 / 56.88
3.80 / 3.22
4.84 / 4.62
4.45 / 1.53
4.08 / 2.95
CosyVoice3
2.36 / 1.44
66.88 / 62.14
3.82 / 3.33
4.80 / 4.64
4.55 / 1.90
4.04 / 3.18
CosyVoice3-RL
2.27 / 0.90
67.32 / 62.17
3.83 / 3.33
4.84 / 4.73
4.53 / 1.98
4.07 / 3.16
F5-TTS
17.40 / 5.33
48.74 / 49.23
3.09 / 2.92
4.79 / 2.78
4.49 / 1.05
4.10 / 1.43
Table 4: Objective and subjective evaluation results of different TTS models. Objective metrics are aggregated using Mean Score (Mean) and reported as en/zh . Subjective metrics are reported at the first and tenth iterations (iter1 / iter10). Bold denotes the best result in each sub-column. Underline denotes the second-best result in each sub-column.
Figure 4: System-level SRCC between aggregated UTMOSv2 scores and human ratings of first-iteration speech, computed under different maximum iteration numbers.
TTS Models
Angry
Happy
Sad
Weighted Avg
GT
75.10
85.70
78.20
79.70
CosyVoice
30.85
30.21
57.29
39.57
CosyVoice2
50.69
54.93
47.19
50.94
CosyVoice3
59.69
60.91
46.30
55.65
CosyVoice3-RL
63.87
53.51
34.00
50.47
F5-TTS
52.47
56.14
34.88
47.83
Table 5: Emotion classification F1-score (%) aggregated using the Mean Score strategy on the emotion dataset for different TTS models.
Figure 5: Cross-model iterative evaluation curves of CosyVoice3-RL and F5-TTS. Solid lines denote the original trajectories without reference swapping, while dashed lines represent the trajectories after exchanging reference audio at the 6th iteration.