When the Persona Contradicts the Line: Diagnosing Plan-to-Synthesis Failures in Chain-of-Thought TTS
Organizations: Spellbrush, CA, USA
Abstract
Chain-of-thought TTS systems rely on language-model planners to translate persona and context into vocal delivery, but it is unclear where this intent is lost when the requested affect conflicts with lexical sentiment. We introduce a bilingual Japanese--English persona--line conflict benchmark in which each item pairs one line with congruent and conflicting personas and pre-declares the expected acoustic direction. We probe this direction in the planner-generated delivery phrase, the planner-written emotion vector, the QwenEmotion-generated vector, and the rendered waveform. Planner outputs are usually direction-correct, while downstream agreement can be substantially lower. Bypassing QwenEmotion provides only modest gains over the full planner-controlled Base. Supervising QwenEmotion raises the direction rate to 75% in Japanese and 84% in English. A learned text-to-latent interface reaches the highest English rate of 89% and the highest Japanese arousal rate of 88%. These results show that learned repair of the plan-to-synthesis interface can recover counter-textual delivery, although its effect differs across languages and acoustic axes.
Figures & tables
| Line | |||
| “ 美味しいです” (It’s delicious.) | a child praising the meal a parent cooked | a wedding guest eating food they cannot enjoy | val |
| “You missed the tram by about ten seconds.” | a helpful stranger on a rainy platform, to a commuter who arrived too late | a weary parent in the dark kitchen, as the exhausted teenager comes home long after curfew | ar |
| Probe point | JP | EN |
| Delivery phrase (LLM jury) | 84% | 98% |
| Planner-written emotion vector (sign map) | 86% | 89% |
| agreement with delivery phrase | 83% | 89% |
| phrase-only / vector-only correct (items) | 11 / 14 | 15 / 1 |
| QwenEmotion vector (sign map) | 46% | 83% |
| arousal / valence | 61 / 37% | 82 / 84% |
| Japanese | English | |||||||
| Condition | Dir. | Ar. | Val. | Dir. | Ar. | Val. | ||
| Base: stock module | 65% | 70% | 62% | – | 74% | 77% | 72% | – |
| Bypassed module | 68% | 80% | 61% | .58 | 79% | 80% | 79% | .26 |
| Supervised module (SFT) | 75% | 77% | 74% | .033 | 84% | 91% | 80% | .036 |
| Learned interface | 66% | 88% | 53% | .60 | 89% | 95% | 85% | .008 |
| Ablations | ||||||||
| Japanese | English | |||||||
| Condition | Judg. | Items | Agr. | Judg. | Items | Agr. | ||
| Base | 64% | 19 | .20 | 20 | 62% | 19 | .20 | 23 |
| Supervised (SFT) | 67% | 20 | .10 | 21 | 69% | 22 | .016 | 23 |
| Learned interface | 66% | 20 | .10 | 20 | 63% | 20 | .10 | 23 |
TTS Synthesis
Text-to-speech synthesis converts written language into intelligible, natural-sounding speech with control over voice, prosody, style, and timing.
See all 275 papers