Text-to-Speech

Also known as TTS

Text-to-speech systems turn text into speech that people can understand and use, with control over voice, accent, emotion, and speaking style. Common approaches include autoregressive models over speech-codec tokens, diffusion and flow-matching models, neural vocoders, and fine-tuning or preference-based post-training. Evaluation combines listening tests with measures such as word error rate and speaker similarity, using benchmarks for zero-shot cloning, multilingual synthesis, and instruction following. A central challenge is balancing intelligibility, speaker identity, and expressive appropriateness: high naturalness scores can miss pronunciation errors or poor fit to a domain, while low-resource languages and fast, robust generation remain difficult.

Latest papers

67 papers4 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

RAF: Relativistic Adversarial Feedback For Universal Speech Synthesis
Mar 12, 2026Yongjoon Lee, Jung-Woo ChoiNeural Vocoders Speech Processing
Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea
When Fine-Tuning Fails and when it Generalises: Role of Data Diversity and Mixed Training in LLM-based TTS
Mar 11, 2026Anupam Purwar, Aditya ChoudharyTTS Synthesis Text-to-Speech
Sprinklr AI Gurugram, India
Scalable Neural Vocoder from Range-Null Space Decomposition
Mar 9, 2026Andong Li, Tong Lei, Zhihang Sun +4Neural Vocoders Speech Processing
the Key Laboratory of Noise and Vibration Research, Institute of Acoustics, Chinese Academy of Sciences, Beijing, 100190, China, and also with University of Chinese Academy of Sciences, Beijing, 100049, China. · Tencent AI Lab. · School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China.
Speech Generation Speaker Poisoning: Capability Erasure in Zero-Shot Text-to-Speech
Mar 8, 2026Thanathai Lertpetchpun, Thanapat Trachu, Sai Praneeth Karimireddy +1TTS Synthesis Zero-Shot TTS
Signal Analysis and Interpretation Laboratory, University of Southern California · Thomas Lord Department of Computer Science, University of Southern California
Hello-Chat: Towards Realistic Social Audio Interactions
Feb 16, 2026Yueran Hou, Peilei Jia, Zihan Sun +5Expressive Speech Synthesis Audio-Language Models
RVCBench: Benchmarking the Robustness of Voice Cloning Across Modern Audio Generation Models
Jan 31, 2026Ruinan Jin, Xinting Liao, Hanlin Yu +2Adversarial Robustness Audio Generation
The University of British Columbia · Vector Institute
Mechanisms of Multimodal Synchronization: Insights from Decoder-Based Video-Text-to-Speech Synthesis
Nov 26, 2024Akshita Gupta, Tatiana Likhomanenko, Karren Dai Yang +3Unified Multimodal Models Audio-Visual Synchronization
TU Darmstadt · Apple

Mar 12, 2026J/K move · Enter open · S save

RAF: Relativistic Adversarial Feedback For Universal Speech Synthesis

Yongjoon Lee, Jung-Woo Choi

Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea

Neural Vocoders Speech Processing Generative Adversarial Networks Adversarial Training Text-to-Speech

Open paper AskPDF arXiv

We propose Relativistic Adversarial Feedback (RAF), a novel training objective for GAN vocoders that improves in-domain fidelity and generalization to unseen scenarios. Although modern GAN vocoders employ advanced architectures, their training objectives often fail to promote generalizable representations. RAF addresses this problem by leveraging speech self-supervised learning models to assist discriminators in evaluating sample quality, encouraging the generator to learn richer representations. Furthermore, we utilize relativistic pairing for real and fake waveforms to improve the modeling of the training data distribution. Experiments across multiple datasets show consistent gains in both objective and subjective metrics on GAN-based vocoders. Importantly, the RAF-trained BigVGAN-base outperforms the LSGAN-trained BigVGAN in perceptual quality using only 12% of the parameters. Comparative studies further confirm the effectiveness of RAF as a training framework for GAN vocoders.