Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Input

Changsong Liu; Tianrui Wang; Ye Ni; Yizhou Peng; Eng Siong Chng

cs.SDMar 6, 2026

Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Input

Authors: Changsong Liu, Tianrui Wang, Ye Ni, Yizhou Peng, Eng Siong Chng

Organizations: Nanyang Technological University, Singapore · Tianjin University, China · Southeast University, China

Abstract

Streaming TTS that receives streaming text is essential for interactive systems, yet this scheme faces two major challenges: unnatural prosody due to missing lookahead and long-form collapse due to unbounded context. We propose a prosodic-boundary-aware post-training strategy, adapting a pretrained LLM-based TTS model using weakly time-aligned data. Specifically, the model is adapted to learn early stopping at specified content boundaries when provided with limited future text. During inference, a sliding-window prompt carries forward previous text and speech tokens, ensuring bounded context and seamless concatenation. Evaluations show our method outperforms CosyVoice-Style interleaved baseline in both short and long-form scenarios. In long-text synthesis, especially, it achieves a 66.2% absolute reduction in word error rate (from 71.0% to 4.8%) and increases speaker and emotion similarity by 16.1% and 1.5% relatively, offering a robust solution for streaming TTS with incremental text.

Figures & tables

Figure 1: The proposed fine-tuning and inferencing pipeline for prosodic boundary-aware streaming LLM-based TTS generation.

System	RTF $\downarrow$	TTFA (ms) $\downarrow$
Interleaved	0.843	1414
Sliding-Window	0.718	2588
Boundary-Aware	0.782	1296

Table 1: Evaluation on streaming efficiency. Chunk size

k=5

words; lookahead

f=2

words for our proposed method.

System	WER (%) $\downarrow$		SPK-SIM $\uparrow$		EMO-SIM $\uparrow$
System	Standard	Long-form	Standard	Long-form	Standard	Long-form
Interleaved	7.48	70.97	0.53	0.56	0.899	0.899
Sliding-Window	6.03	7.83	0.57	0.22	0.912	0.857
Boundary-Aware (Ours)	4.03	4.77	0.64	0.65	0.918	0.912

Table 2: Objective quality evaluation on Seed-TTS-Eval. Chunk size

k=5

words; lookahead

f=2

words for our proposed method.

System	MOS $\uparrow$		SMOS $\uparrow$		EMOS $\uparrow$
System	Standard	Long-form	Standard	Long-form	Standard	Long-form
Interleaved	3.99 $\pm$ 0.16	3.18 $\pm$ 0.23	4.05 $\pm$ 0.15	3.24 $\pm$ 0.22	4.03 $\pm$ 0.15	3.21 $\pm$ 0.18
Sliding-Window	3.43 $\pm$ 0.20	1.60 $\pm$ 0.18	3.18 $\pm$ 0.20	1.68 $\pm$ 0.18	3.25 $\pm$ 0.18	1.67 $\pm$ 0.17
Boundary-Aware (Ours)	4.28 $\pm$ 0.14	4.13 $\pm$ 0.13	4.25 $\pm$ 0.13	4.24 $\pm$ 0.13	4.38 $\pm$ 0.12	4.19 $\pm$ 0.13

Table 3: Subjective MOS evaluation on Seed-TTS-Eval. Higher scores indicate better perceptual quality.

Figure 2: Ablation study on quality evaluation with different combinations of chunk size k and lookahead context f.

Long-Form TTS Synthesis

See all 10 papers

Streaming T5-based Text-to-Speech Synthesis with Limited Lookahead
Jun 20, 2026Muyang Du, Jason Roche, Junjie LaiTTS Synthesis Streaming TTS Synthesis
1NVIDIA, China · 2NVIDIA, USA
Speaking Longer Than Learned: Training-Free Long-Form Generation for Full-Stream Text-to-Speech
Oct 14, 2026Jeseung Lim, Jiseong Woo, Heeseung KimLong-Form TTS Synthesis TTS Synthesis
Department of Physics, University of Seoul, Seoul, Republic of Korea · Department of AI, University of Seoul, Seoul, Republic of Korea
TontaubeV1: Streaming Text-to-Speech with Hierarchical Codec Modeling and Bounded Context
Sep 8, 2026Fritz Cremer, Jonathan CremerTTS Synthesis Streaming TTS Synthesis

Jun 20, 2026J/K move · Enter open · S save

Streaming T5-based Text-to-Speech Synthesis with Limited Lookahead

Muyang Du, Jason Roche, Junjie Lai

1NVIDIA, China · 2NVIDIA, USA

TTS Synthesis Streaming TTS Synthesis

Open paper AskPDF arXiv

Streaming text-to-speech synthesis in cascaded LLM-TTS systems still faces latency challenges as most TTS models require full context before initiating generation. We present S5-TTS, a streaming variant of T5-TTS that enables low-latency, word-by-word incremental speech synthesis through encoder-decoder language modeling and monotonic alignment learning. S5-TTS begins generating speech immediately after receiving the first few words, substantially reducing end-to-end response latency. To maintain quality under limited lookahead, we introduce a lookahead-causal masking mechanism with Conv-based auxiliary attention that preserves intelligibility and speaker similarity, and employ interleaved multi-source distillation to further restore naturalness. Experiments show that S5-TTS achieves comparable quality to full-context T5-TTS, supports zero-shot synthesis with high speaker similarity, and significantly reduces end-to-end latency for practical conversational AI systems.

Abstract

Long-Form TTS Synthesis

Explore similar work

Streaming T5-based Text-to-Speech Synthesis with Limited Lookahead

Speaking Longer Than Learned: Training-Free Long-Form Generation for Full-Stream Text-to-Speech

TontaubeV1: Streaming Text-to-Speech with Hierarchical Codec Modeling and Bounded Context