Streaming text-to-speech is essential for low-latency spoken dialogue systems, yet many systems wait for sentence-level text and are therefore only pseudo-streaming. True token-level synthesis must generate speech from uncertain prefixes while maintaining perceptual continuity over an unbounded stream with bounded context. We present X2Streaming-TTS, a causal TTS framework that consumes asynchronously arriving text tokens and emits speech without accessing future input. To handle uncertain prefixes, we introduce causal commitment, which keeps ambiguous expressions provisional through uncertainty-aware buffering and performs capacity-adaptive, punctuation-aware segmentation. To preserve acoustic continuity, we further introduce causal speech-state inheritance, which carries the complete Code2Wav state and selected historical Talker states across segment boundaries. Together with an attention prior constraint, it blocks access to future positions while retaining bounded acoustic context. Experiments show that X2Streaming-TTS outperforms existing pseudo-streaming models on most subjective and objective metrics. Further analysis shows that causal commitment stabilizes online segmentation and reduces failures caused by insufficient context, while speech-state inheritance improves boundary continuity without degrading naturalness or speaker identity. X2Streaming-TTS thus achieves strict token-level synthesis with quality comparable to the evaluated offline baselines, a median time to first audio token (TTFT) of 15.8 ms for a single request, and a median TTFT of 260.8 ms at 128 concurrent requests. Our implementation is publicly available at https://github.com/X-Square-Robot/X2Streaming-TTS .
Figures & tables
Figure 1: Left: Streaming pipeline under token-level arrival. The frontend releases only TTS-ready text ( PAD when nothing is ready; ambiguous spans such as 3 in He finished 3 are held until pronunciation is resolved), the Talker emits acoustic tokens, and Code2Wav streams waveform for immediate playback. Right: Three challenges and the corresponding mechanisms—(1) first-audio latency via token-level rather than chunk-level start; (2) irreversible early commitment via uncertainty-aware buffering; (3) cross-segment discontinuity via speech-state inheritance.
Figure 2: Speech-state inheritance across a segment boundary. After a health check on Sk−1 , its Code2Wav state warm-starts Sk , while its trailing Talker states provide bounded historical context through causal-prior attention and a gated residual.
Model
Streaming
Granularity
SEED-TTS-Eval
MiniMax
Long-text extrapolation
CER ZH
WER EN
CER ZH
WER EN
1×
2×
5×
10×
Qwen3-TTS-12Hz-1.7B
✗
Offline
1.10
1.43
0.87
1.85
3.95
3.05
4.01
3.93
F5-TTS
✗
Offline
1.52
2.00
3.74
2.08
4.42
3.81
5.04
4.95
FireRedTTS-2
✓
Chunk
1.14
1.95
0.97
2.25
5.36
4.79
5.04
5.32
CosyVoice 2-S
✓
Chunk
1.45
2.57
1.98
2.38
4.76
3.83
5.05
5.48
CosyVoice 3-S
✓
Chunk
0.81
1.68
1.43
2.21
4.72
3.36
4.86
5.12
Table 1: Intelligibility and long-text robustness (% error; lower better). Bold best, underline second best. Granularity is the text available at the acoustic decision: Offline, Chunk, or Token.
Online replay
Matched budget
Policy
Seg.
Cap ↓
Util. ↑
F1 ↑
Miss ↓
Split ↓
Fixed window
401
87.13
92.48
0.017
0.983
0.983
Punct. only
3151
0
11.77
0.199
0.766
0.150
Fixed ratio
470
1.70
78.90
–
–
–
SaT-3L
–
–
–
0.940
0.068
0.050
X2Streaming-TTS
430
0.54
76.93
0.952
0.057
0.036
Table 2: Causal commitment under two protocols (Cap/Util. in %). Left: online replay on 59 passages (Seg. = segment count; Cap = hard-cap share; Util. = budget used). Right: matched-budget boundary quality vs. human annotations. Dashes: policy absent from that protocol.
System
Gran.
Δ F0 (Hz) ↓
Δ E (dB) ↓
PBD ↓
ECAPA sim. ↑
UTMOS ↑
CosyVoice 2-S
Chunk
46.89
3.39
0.3427 [0.3226, 0.3636]
0.9304 [0.9223, 0.9366]
3.1949 [3.1277, 3.2582]
CosyVoice 3-S
Chunk
47.53
3.41
0.3479 [0.3258, 0.3712]
0.9264 [0.9215, 0.9319]
2.6899 [2.6352, 2.7465]
FireRedTTS-2
Chunk
31.68
2.17
0.1915 [0.1794, 0.2041]
0.5205 [0.4165, 0.6178]
3.2704 [3.0815, 3.4657]
X2Streaming-TTS
Token
22.61
1.66
0.1092 [0.1000, 0.1192]
0.9511 [0.9454, 0.9565]
3.9200 [3.8388, 3.9960]
Table 3: Boundary continuity and long-text stability under fixed-rate token arrival. Brackets: passage-bootstrap 95% CIs; bold best. Δ F0/ Δ E/PBD: ±1000 ms windows at 954 shared boundaries (59 passages). ECAPA/UTMOS: 10 s fixed-window protocol (60 windows/system).
System
CER ↓
Read ↑
UTMOS ↑
MOS ↑
Sem. ↑
CosyVoice 2-S
33.41
0.0
3.075
3.220
0.00
CosyVoice 3-S
6.65
40.0
3.017
3.183
60.00
FireRedTTS-2
18.21
13.3
2.788
3.629
6.67
X2Streaming-TTS
2.00
73.3
4.025
3.802
93.33
Table 4: Symbol and prefix-ambiguity evaluation on identical inputs (CER/Read/Sem. in %). Read = fully correct reading; Sem. = listener-judged meaning preserved (120 listeners).
Figure 3: Time to first audio token (TTFT) under concurrent requests on one RTX 5090. Points show client-side medians across the tested serving interfaces; labels give the median values, and the shaded band spans the minimum to the 99th percentile across successful requests. Each interface was measured for 20 rounds after 3 warmup rounds, yielding n=40 , 320, 640, 1,280, 2,560, and 5,120 requests at concurrency levels 1, 8, 16, 32, 64, and 128, respectively.
Streaming text-to-speech synthesis in cascaded LLM-TTS systems still faces latency challenges as most TTS models require full context before initiating generation. We present S5-TTS, a streaming variant of T5-TTS that enables low-latency, word-by-word incremental speech synthesis through encoder-decoder language modeling and monotonic alignment learning. S5-TTS begins generating speech immediately after receiving the first few words, substantially reducing end-to-end response latency. To maintain quality under limited lookahead, we introduce a lookahead-causal masking mechanism with Conv-based auxiliary attention that preserves intelligibility and speaker similarity, and employ interleaved multi-source distillation to further restore naturalness. Experiments show that S5-TTS achieves comparable quality to full-context T5-TTS, supports zero-shot synthesis with high speaker similarity, and significantly reduces end-to-end latency for practical conversational AI systems.
Streaming TTS that receives streaming text is essential for interactive systems, yet this scheme faces two major challenges: unnatural prosody due to missing lookahead and long-form collapse due to unbounded context. We propose a prosodic-boundary-aware post-training strategy, adapting a pretrained LLM-based TTS model using weakly time-aligned data. Specifically, the model is adapted to learn early stopping at specified content boundaries when provided with limited future text. During inference, a sliding-window prompt carries forward previous text and speech tokens, ensuring bounded context and seamless concatenation. Evaluations show our method outperforms CosyVoice-Style interleaved baseline in both short and long-form scenarios. In long-text synthesis, especially, it achieves a 66.2% absolute reduction in word error rate (from 71.0% to 4.8%) and increases speaker and emotion similarity by 16.1% and 1.5% relatively, offering a robust solution for streaming TTS with incremental text.
Changsong Liu, Tianrui Wang, Ye Ni +2
Nanyang Technological University, Singapore · Tianjin University, China · Southeast University, China
Recent progress in speech dialogue systems requires Text-to-Speech (TTS) models to be faster and more responsive. Modern speech dialogue systems impose two primary requirements on TTS models: low latency and support for streaming inputs and outputs. However, most existing single-codebook LLM-based TTS methods rely on multi-stage pipelines that lack native streaming capabilities. These systems typically suffer from high end-to-end latency due to slow autoregressive prediction and multi-step flow matching. To address these limitations, we propose FlashTTS, an open-source and low-latency streaming TTS framework. FlashTTS introduces a lagged multi-track architecture that natively processes streaming text and speech inputs, thereby eliminating the need for sentence-level buffering. To accelerate acoustic generation, we integrate parallel Multi-Token Prediction (MTP) with an X-pred mean flow matching decoder. This configuration achieves high-fidelity token-to-mel generation in exactly two function evaluations (2-NFE). By jointly optimizing input processing and decoding efficiency, FlashTTS offers a practical foundation for real-time speech dialogue systems. Experiments show that FlashTTS substantially reduces First-Packet Latency to 325ms compared to robust streaming baselines, all while preserving strong zero-shot voice cloning and cross-lingual intelligibility. Speech samples are available. The model code and checkpoints will be released as open source.