ReaFlow-TTS: Realization-Conditioned Flow Matching for High-Quality and Controllable Speech Synthesis
Organizations: School of Information Science and Technology, Lanzhou University, China
Abstract
In flow-matching text-to-speech (TTS), different speech realizations can induce different target velocities under the same generation conditions. A deterministic velocity field trained with squared error predicts their conditional mean, thereby marginalizing realization-dependent variation. Meanwhile, modeling such variation does not inherently provide a semantically interpretable interface for attribute manipulation. We propose ReaFlow-TTS, a realization-conditioned flow-matching framework that introduces an utterance-level stochastic realization latent and uses it to condition velocity prediction throughout the generation trajectory. We further impose valence-arousal-dominance (VAD) semantics on the realization space, enabling direct and graded attribute manipulation without target speech at inference. Experiments demonstrate improved synthesis quality over a matched full-mask baseline and reproducible latent-induced pitch, energy, and timing tendencies across initial-noise samples, providing behavioral evidence that the latent is used as a reusable realization condition. Subjective evaluation further demonstrates graded VAD manipulation across generation contexts with only modest changes in naturalness.
Figures & tables
| System | Inf. params. | WER (%) | UTMOS | NMOS |
|---|---|---|---|---|
| Ground truth | — | 2.28 | 4.10 | 3.93 |
| F5-TTS | 157.97M | 2.27 | 3.91 | 3.72 |
| F5-TTS (full mask) | 157.97M | 2.32 | 3.83 | 3.67 |
| ZipVoice base | 122.66M | 2.24 | 3.87 | 3.63 |
| ReaFlow-TTS (500k) | 158.75M | 2.08 | 4.19 | 3.93 |
| ReaFlow-TTS (full) | 158.75M | 2.09 | 4.19 | 3.91 |
| Axis | Attribute score [-1pt] | NMOS [-1pt] | Ordering acc. (%) [-1pt] |
|---|---|---|---|
| V | 3.62/3.77/4.21 | 3.91/3.93/3.83 | 61/78/82 |
| A | 3.38/4.76/5.23 | 3.91/3.82/3.80 | 89/81/92 |
| D | 3.71/4.98/5.36 | 3.91/3.84/3.77 | 86/82/88 |