Streaming TTS Synthesis

TTS: Text-to-Speech

Streaming text-to-speech studies how to turn incoming text into speech before the full utterance is available, enabling responsive assistants and live voice interaction. Common approaches use causal or limited-lookahead models, text-speech alignment and interleaving, and autoregressive, codec-based, or diffusion decoders with bounded context and cached state. Acceleration methods such as multi-token prediction, distillation, speculative decoding, and optimized inference aim to lower first-audio latency while preserving intelligibility, speaker identity, and expressive prosody. A central challenge is balancing fast output with natural phrasing and continuity, especially when text is uncertain or long-form and when systems must support interruption and precise control of emotion or speaking rate.

Latest papers

25 papers6 papers in the last four weeks, up 50% on the four weeks before. 0.0% of all new papers.

Uni-Stream: A Unified Streaming Speech Language Model for Speech Synthesis and Recognition
Oct 23, 2026Xuan Shi, Chang Zeng, Tiantian Feng +5Speech Processing Speech Language Models
University of Southern California, USA · National Institute of Informatics, Japan · Massachusetts Institute of Technology, USA +1
RubatoCodec: Budgeted Adaptive Speech Tokenization with Streaming Decoding
Oct 21, 2026Chihiro Arata, Shohei Taniguchi, Gouki Minegishi +1Neural Audio Codecs Audio Token Compression
Third Intelligence, Inc.
Speaking Longer Than Learned: Training-Free Long-Form Generation for Full-Stream Text-to-Speech
Oct 14, 2026Jeseung Lim, Jiseong Woo, Heeseung KimLong-Form TTS Synthesis TTS Synthesis
Department of Physics, University of Seoul, Seoul, Republic of Korea · Department of AI, University of Seoul, Seoul, Republic of Korea
Fish Audio Drama 3 Technical Report
Oct 13, 2026Dawei Han, Shidong Li, Shijia Liao +13TTS Synthesis Controllable Speech Generation
SCIC: Scope- and Codebook-Aware Instruction Conditioning for Speaker-Adapted Expressive TTS
Sep 30, 2026Longyu Lu, Zongwei Du, Mengtao Xing +4TTS Synthesis Streaming TTS Synthesis
TaoLive-AIGC Team Taobao & Tmall Group of Alibaba
RVQ Position Aware Speculative Decoding for On Device Text to Speech
Sep 29, 2026Berkin Durmus, Eduardo Pacheco, Zach Nagengast +1TTS Synthesis Streaming TTS Synthesis
Argmax, Inc., USA
Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis
Sep 17, 2026Zifan Guan, Longyu Lu, Junan Zhang +3Speech Generation Evaluation TTS Evaluation
The Chinese University of Hong Kong, Shenzhen, China · TaoLive-AIGC Team Taobao & Tmall Group of Alibaba
X2-NativeCursor: Native-Token Text Progress Tracking for Incremental-Text Streaming Codec TTS
Sep 9, 2026Zehan Liu, Carl Chen, Rime Wen +7TTS Synthesis Streaming TTS Synthesis
X Square Robot
TASTE2: Text-Aligned Speech Modeling and Deployment toward Full-Duplex Voice Interaction
Sep 8, 2026Yi-Chang Chen, Chun Wei Chen, Dien-Ruei Wu +7TTS Synthesis Streaming TTS Synthesis
MediaTek Research · Internship at MediaTek Research · National Taiwan University +2
TontaubeV1: Streaming Text-to-Speech with Hierarchical Codec Modeling and Bounded Context
Sep 8, 2026Fritz Cremer, Jonathan CremerTTS Synthesis Streaming TTS Synthesis
X2Streaming-TTS: Causal Token-Level Text-to-Speech from Streaming Text with Speech-State Inheritance
Aug 19, 2026Rime Wen, Zehan Liu, Shawn Qin +4TTS Synthesis Streaming TTS Synthesis
X Square Robot
VoiceChat-TTS: A Low-Latency Continuous Speech Synthesis Model for Interactive Agents
Aug 13, 2026Edresson Casanova, Jaehyeon Kim, Mariana Graterol Fuenmayor +17TTS Synthesis Streaming TTS Synthesis
NVIDIA Corporation
CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents
Aug 9, 2026Yuqian Zhang, Yao Shi, Kexin Huang +6TTS Synthesis Streaming TTS Synthesis
Work done during internship at OPPO. · OPPO AI Center · Fudan University +1
Memory Efficient Audio Synthesis with Decoupled Temporal Depth Diffusion Transformers
Jul 26, 2026Dongseong Hwang, Prasanth Yadla, Kaan Elgin +8TTS Synthesis Streaming TTS Synthesis
Apple
Faster IndexTTS-2: Accelerating and Streaming Autoregressive Zero-Shot Text-to-Speech Synthesis on GPUs
Jul 23, 2026Muyang Du, Shuang Yu, Junjie LaiTTS Synthesis Streaming TTS Synthesis
NVIDIA · Shanghai, China
ZONOS2 Technical Report
Jun 23, 2026Gabriel Clark, Sofian Mejjoute, Mohamed Osman +2TTS Synthesis Streaming TTS Synthesis
Zyphra · San Francisco, CA
Streaming T5-based Text-to-Speech Synthesis with Limited Lookahead
Jun 20, 2026Muyang Du, Jason Roche, Junjie LaiTTS Synthesis Streaming TTS Synthesis
1NVIDIA, China · 2NVIDIA, USA
FlashTTS: Fast Streaming TTS with MTP Acceleration and X-pred Mean Flow Distillation
Jun 8, 2026Hanke Xie, Xiaming Ren, Dake Guo +10Flow Matching Multi-Token Prediction
Huawei Technologies Co., Ltd
Chatterbox-Flash: Prior-Calibrated Block Diffusion for Streaming Zero-Shot TTS
May 29, 2026Deokjin Seo, Gangin Park, Kihyun NamTTS Synthesis Streaming TTS Synthesis
Seoul National University · KAIST
Self-EmoQ: Plutchik-Guided Value-based Planning to Drive Streaming Emotional TTS
Apr 21, 2026Yue Zhao, Hongyan Li, Yong Chen +1TTS Synthesis Streaming TTS Synthesis
Geely AI Lab
MambaVoiceCloning: Efficient and Expressive Text-to-Speech via State-Space Modeling and Diffusion Control
Mar 31, 2026Sahil Kumar, Namrataben Patel, Honggang Wang +1Expressive Speech Synthesis TTS Synthesis
PhD Program in Mathematics, Yeshiva University, New York, NY 10033, USA · Department of Computer Science & Engineering, Yeshiva University, New York, NY 10033, USA · School of Artificial Intelligence, Chuzhou University, Anhui, 239000, China
VoXtream2: Full-stream TTS with dynamic speaking rate control
Mar 13, 2026Nikita Torgashov, Gustav Eje Henter, Gabriel SkantzeStreaming TTS Synthesis Zero-Shot TTS
Department of Speech, Music and Hearing, KTH Royal Institute of Technology, Sweden
Fish Audio S2 Technical Report
Mar 9, 2026Shijia Liao, Yuxuan Wang, Songting Liu +11Multi-Speaker TTS TTS Synthesis
Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Input
Mar 6, 2026Changsong Liu, Tianrui Wang, Ye Ni +2Long-Form TTS Synthesis TTS Synthesis
Nanyang Technological University, Singapore · Tianjin University, China · Southeast University, China
CTC-TTS: LLM-based dual-streaming text-to-speech with CTC alignment
Feb 23, 2026Hanwen Liu, Saierdaer Yusuyin, Hao Huang +1TTS Synthesis Streaming TTS Synthesis
School of Computer Science and Technology, Xinjiang University, China · Speech Processing and Machine Intelligence (SPMI) Lab, Tsinghua University, China

Oct 23, 2026J/K move · Enter open · S save

Uni-Stream: A Unified Streaming Speech Language Model for Speech Synthesis and Recognition

Xuan Shi, Chang Zeng, Tiantian Feng, Ye Zhang+4

University of Southern California, USA · National Institute of Informatics, Japan · Massachusetts Institute of Technology, USA+1

Speech Processing Speech Language Models Streaming TTS Synthesis Streaming ASR

Open paper AskPDF arXiv

Unified streaming speech language models aim to support speech recognition and synthesis within a single model, enabling real-time human-computer interaction. However, jointly supporting these two directions of speech-text conversion while preserving task-specific performance remains challenging. We propose Uni-Stream, a unified streaming speech language model built on compact pretrained speech models. Uni-Stream processes interleaved text and speech in a streaming format and achieves performance comparable to task-specific models through two-stage training. We also develop a data annotation pipeline for streaming training and will publicly release the generated transcriptions and word-level timestamps. We further examine Uni-Stream's representations across tasks and modalities and find that speech and text residence in distinct latent feature space even when trained in a unified model architecture, while representations within each modality become more task-dependent in later layers.