cs.SDNov 8, 2026

Guidance at Pure Noise Erodes Speaker Similarity in Few-Step Token-Conditioned Flow-Matching TTS

Authors: Jaeho Lee, Han-Lim Choi

Organizations: Department of Aerospace Engineering, KAIST, Daejeon, Republic of Korea.

Abstract

Text-to-speech systems built from a language model and a flow-matching mel decoder now run the decoder with two to four Euler steps instead of the official ten or more. At two steps the word error rate (WER) is unchanged, but speaker similarity (SIM) drops by one to six points. This paper shows that most of this loss comes from classifier-free guidance on the first step. At pure noise the network can only return the conditional mean, so guidance shifts every trajectory by nearly the same vector. Later steps do not remove this shift, and its size grows with the length of the first step, so it is negligible with ten or more steps and accounts for most of the loss with two. Two training-free samplers are derived from this analysis. First-step unguided sampling (FSU) leaves the first step unguided and recovers 8484--91%91\% of the two-step SIM loss on CosyVoice2/3 with no significant WER increase. FSU-Mid replaces the two Euler steps by one midpoint step with an unguided first evaluation, recovers 7777--85%85\% on IndexTTS-2 and matches the ten-step teacher on CosyVoice2 and GLM-TTS. Both use three network function evaluations, one fewer than the official two-step sampler, and FSU-Mid gives the highest mean recovery among eight published guidance rules re-run at two steps on four decoders.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Prompt-Consistency Inference for Zero-Shot Flow-Matching Text-to-Speech Models

    Oct 3, 2026Vasily Zadorozhnyy, Can Goksen, Kazuhito Koishida +1TTS SynthesisZero-Shot TTS

  2. Fréchet Distance Loss on Speech Representations for Text-to-Speech Synthesis

    Jul 7, 2026Ho-Lam Chung, Kuan-Po Huang, Bo-Ru Lu +1Flow MatchingTTS Synthesis

  3. Joint Residual Reweighting for Classifier Free Guidance in Flow-Matching Zero-Shot TTS

    Jun 24, 2026Runwu Shi, Yujin Wang, Hongjin Song +8Flow MatchingClassifier-Free Guidance