Text-to-Speech

Also known as TTS

Text-to-speech systems turn text into speech that people can understand and use, with control over voice, accent, emotion, and speaking style. Common approaches include autoregressive models over speech-codec tokens, diffusion and flow-matching models, neural vocoders, and fine-tuning or preference-based post-training. Evaluation combines listening tests with measures such as word error rate and speaker similarity, using benchmarks for zero-shot cloning, multilingual synthesis, and instruction following. A central challenge is balancing intelligibility, speaker identity, and expressive appropriateness: high naturalness scores can miss pronunciation errors or poor fit to a domain, while low-resource languages and fast, robust generation remain difficult.

Latest papers

67 papers4 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

AudioCALM: Continuous Autoregressive Language Modeling for Universal Audio Generation
Jun 22, 2026Huadai Liu, Kaicheng Luo, Wen Wang +4Text-to-Audio Generation Flow-Based Generative Modeling
Hong Kong University of Science and Technology (HKUST) · Tongyi Fun Team, Alibaba Group
Bagpiper-TTS: Natural Language Guided Universal Speech Synthesis
Jun 22, 2026Jinchuan Tian, Haoran Wang, Siddhant Arora +6Controllable Speech Generation Text-to-Speech
Language Technologies Institute, Carnegie Mellon University · LY Corporation · NVIDIA Research
ISCSLP 2026 CoT-TTS Challenge: Chain-of-Thought Reasoning for Context-Aware Text-to-Speech
Jun 20, 2026Wei Xue, Junlan Feng, Shilei Zhang +9TTS Synthesis TTS Evaluation
The Hong Kong University of Science and Technology · China Mobile · Jiutian Artificial Intelligence Technology (Beijing) Co., Ltd., China Mobile +3
Exploring Pre-training Benefits on Phoneme Addition through Fine-tuning in Speech Synthesis
Jun 18, 2026Masato Murata, Koichi Miyazaki, Tomoki Koriyama +1TTS Synthesis Low-Resource TTS Synthesis
CyberAgent, Japan · Nagoya University, Japan
Reliable Neural-Codec Text-to-Speech by ASR Self-Verification and Distillation: Near-Zero Catastrophic Failures Across Models and Codecs
Jun 16, 2026Ali Asaria, Tony Salomone, Deep GandhiTTS Synthesis Text-to-Speech
Transformer Lab
Joycent: Multi-Accent TTS via Disentangled Accent Modeling and Layer-Specific Conditioning
Jun 15, 2026Xintong Wang, Junchuan Zhao, Ye WangDisentangled Representation Learning TTS Synthesis
School of Computing, National University of Singapore, Singapore
SARA: A Dual-Stream VAE for High-Fidelity Speech Generation via Integrating Semantic and Acoustic Representations
Jun 10, 2026Peijie Chen, Wenhao Guan, Weijie Wu +7Variational Autoencoders Audio Representation Learning
School of Informatics, Xiamen University, China · School of Electronic Science and Engineering, Xiamen University, China · DiDi Global Inc., Beijing, China
OpenBibleTTS: Large-Scale Speech Resources and TTS Models for Low-Resource Languages
Jun 8, 2026David Guzmán, Luel Hagos Beyene, Jesujoba Oluwadara Alabi +3TTS Synthesis TTS Evaluation
McGill University · Mila - Quebec AI Institute · AIMS Research and Innovation Centre +3
NüshuVoice: Reviving the Voice of Endangered Nüshu with Pitch-Aware Text-to-Speech
Jun 8, 2026Hongkun Yang, Xinhui Yi, Xiyan Zhao +13TTS Synthesis Low-Resource TTS Synthesis
1Ocean University of China · 2The Hong Kong Polytechnic University · 3Cornell University +5
TLDR: Compressing Audio Tokens for Efficient Autoregressive Text-to-Speech
Jun 8, 2026Yejin Lee, Junwon Moon, Hyoeun Kim +3Efficient Transformer Inference Audio Token Compression
Sungkyunkwan University · University of Seoul
dots.tts Technical Report
Jun 5, 2026Shi Lian, Changtao Li, Bohan Li +6TTS Synthesis Speech Foundation Models
VoxCPM2 Technical Report
Jun 5, 2026Yixuan Zhou, Guoyang Zeng, Xin Liu +15Speech Foundation Models Multilingual TTS Synthesis
VoxCPM Team
Leveraging Soft Distributions of SSL-Derived Discrete Speech Tokens for Downstream Inference
Jun 5, 2026Kentaro Onda, Satoru Fukayama, Daisuke Saito +1Automatic Speech Recognition Audio Tokenization
The University of Tokyo, Japan · National Institute of Advanced Industrial Science and Technology (AIST), Japan
ImmersiveTTS: Environment-Aware Text-to-Speech with Multimodal Diffusion Transformer and Domain-Specific Representation Alignment
May 29, 2026Jun-Hak Yun, Seung-Bin Kim, Seong-Whan LeeCross-Modal Representation Learning TTS Synthesis
Department of Artificial Intelligence, Korea University, Seoul, Korea
MELD: Mel-Spectrogram-Based Speech Language Modeling with Discrete Latent Variables
May 28, 2026Sung-Lin Yeh, Wei Zhou, Gil Keren +6Audio Representation Learning Autoregressive Language Modeling
University of Edinburgh · *Work done at Meta · 2Google DeepMind +1
PashtoTTS-Bench: automated screening for low-resource non-Latin-script text-to-speech
May 26, 2026Hanif RahmanSpeech Generation Evaluation TTS Evaluation
aIndependent Researcher
CosyEdit2: Speech-Editing-Oriented Reinforcement Learning Unlocks Better Zero-Shot TTS
May 25, 2026Junyang Chen, Yuhang Jia, Hui Wang +3Speech Editing Zero-Shot TTS
College of Computer Science, Nankai University · College of Artificial Intelligence, Nankai University
Eroding Trust in Real Speech: A Large-Scale Study of Human Audio Deepfake Perception
May 21, 2026Nicolas M. Müller, Wei Herng ChoongAudio Deepfake Detection Speech Processing
Fraunhofer AISEC & Resemble AI · Germany · Fraunhofer AISEC
DUET: Unified Dual-Space Emotion Control for Diffusion and Flow-Matching Driven Text-to-Speech
May 20, 2026Xu Zhang, Longbing Cao, Zhangkai WuTTS Synthesis Audio Diffusion Models
Frontier AI Research Centre, Macquarie University, Sydney, NSW, Australia
WavCube: Unifying Speech Representation for Understanding and Generation via Semantic-Acoustic Joint Modeling
May 7, 2026Guanrou Yang, Tian Tan, Qian Chen +12Audio Representation Learning TTS Synthesis
Shanghai Jiao Tong University · Independent Researcher · Shanghai Innovation Institute +4
MiniMind-O Technical Report: An Open Small-Scale Speech-Native Omni Model
May 5, 2026Jingyao GongSpeech Foundation Models Unified Multimodal Models
Independent Researcher
JaiTTS: A Thai Voice Cloning Model
Apr 30, 2026Jullajak Karnjanaekarin, Pontakorn Trakuekul, Narongkorn Panitsrisit +5TTS Synthesis Audio Generation
Jasmine Technology Solution · Sirindhorn International Institute of Technology · Department of Linguistics, Chulalongkorn University
One Voice, Many Tongues: Cross-Lingual Voice Cloning for Scientific Speech
Apr 28, 2026Amanuel Gizachew Abebe, Yasmin MoslemTTS Synthesis Text-to-Speech
Shaggar Institute of Technology · Shaggar City, Ethiopia · Trinity College Dublin +1
Praxy Voice: Voice-Prompt Recovery + BUPS for Commercial-Class Indic TTS from a Frozen Non-Indic Base at Zero Commercial-Training-Data Cost
Apr 28, 2026Venkata Pushpak Teja MentaTTS Synthesis Multilingual TTS Synthesis
Praxel Ventures
TTS-PRISM: A Perceptual Reasoning and Interpretable Speech Model for Fine-Grained Diagnosis
Apr 24, 2026Xi Wang, Jie Wang, Xingchen Song +8TTS Synthesis Speech Generation Evaluation
Tsinghua University, China · MiLM Plus, Xiaomi Inc., China · The University of Tokyo, Japan
MINT-Bench: A Comprehensive Multilingual Benchmark for Instruction-Following Text-to-Speech
Apr 20, 2026Huakang Chen, Jingbin Hu, Liumeng Xue +12TTS Synthesis Speech Generation Evaluation
School of Intelligence Science and Technology, Nanjing University, Suzhou, China · Yutu Zhineng, Beijing, China · Lingguang Zhaxian Technology, Shanghai, China
Bridging What the Model Thinks and How It Speaks: Expressive Speech Generation via Self-Aware Intent-Realization Alignment
Apr 13, 2026Kuang Wang, Lai Wei, Ping Lin +8Expressive Speech Synthesis Speech Language Models
The Chinese University of Hong Kong, Shenzhen · Tencent Ethereal Audio Lab · Southeast University +2
Sign-to-Speech Prosody Transfer via Sign Reconstruction-based GAN
Apr 12, 2026Toranosuke Manabe, Yuto Shibata, Shinnosuke Takamichi +1Generative Adversarial Networks Text-to-Speech
Keio University, Yagami Campus, Yokohama Kanagawa 223-8522, Japan
SelfTTS: cross-speaker style transfer through explicit embedding disentanglement and self-refinement using self-augmentation
Mar 23, 2026Lucas H. Ueda, João G. T. Lima, Pedro R. Corrêa +3Disentangled Representation Learning TTS Synthesis
Universidade Estadual de Campinas (UNICAMP), Brazil · CPQD, Brazil
The Binding Effect: Analyzing How Multi-Dimensional Cues Form Gender Bias in Instruction TTS
Mar 21, 2026Kuan-Yu Chen, Yi-Cheng Lin, Po-Chung Hsieh +5Text-to-Speech Social Bias in Language Models
Graduate Institute of Communication Engineering, National Taiwan University, Taiwan · AI Research Center, Inventec Corporation, Taiwan

Jun 22, 2026J/K move · Enter open · S save

AudioCALM: Continuous Autoregressive Language Modeling for Universal Audio Generation

Huadai Liu, Kaicheng Luo, Wen Wang, Qian Chen+3

Hong Kong University of Science and Technology (HKUST) · Tongyi Fun Team, Alibaba Group

Text-to-Audio Generation Flow-Based Generative Modeling Audio Generation Text-to-Speech

Open paper AskPDF arXiv

Unifying speech, sound, and music generation in one model is hindered by tradeoffs between fidelity, end-to-end training, in-context conditioning, and variable-length synthesis that no current paradigm fully resolves. To address this challenge, we present AudioCALM, a universal audio generation framework that extends autoregressive (AR) next-token prediction from discrete tokens to continuous audio latents: a thin flow-matching head replaces the softmax to predict rectified-flow velocities at each position, and a block-causal AR-Flow attention pattern produces arbitrary-length output. Joint training of multiple audio generation tasks faces an asymmetric text--audio mismatch: speech transcripts align to specific time spans and demand tight, time-aligned attention, whereas sound and music captions describe only overall semantics and rely on diffuse, holistic attention; mixing the two disproportionately degrades sound and music generation. We address this asymmetry at two levels: a data reformulation strategy that unifies all three tasks under a single description-style conditioning interface, and a novel architecture Asymmetric Mixture-of-Modality-Experts (A-MoME), which adds a dedicated residual expert for speech while sound and music share the backbone, incurring no inference overhead on non-speech inputs. Experimental results demonstrate that AudioCALM matches modality-specific state-of-the-art and outperforms prior unified baselines on speech, sound, and music generation benchmarks.