Zero-Shot TTS

TTS: Text-to-Speech

Zero-shot TTS studies how to generate speech in a speaker’s voice from a short audio reference, often across languages or without a transcript. Common approaches include autoregressive models over discrete speech tokens, flow-matching and diffusion models, and masked non-autoregressive generation; some also model waveforms directly. Current work extends voice cloning with streaming, speech editing, and controls for pronunciation, accent, emotion, and prosody. A central challenge is balancing intelligibility, naturalness, and faithful speaker identity, while evaluation metrics do not always agree with human judgments or capture these goals separately.

Latest papers

61 papers11 papers in the last four weeks, up 38% on the four weeks before. 0.1% of all new papers.

Zero-Shot Lombard Speech Synthesis with Controllable Style Embeddings
Jan 19, 2026Seymanur Akti, Alexander WaibelTTS Synthesis Zero-Shot TTS
KIT Campus Transfer GmbH (KCT) · Karlsruhe Institute of Technology (KIT) · Carnegie Mellon University (CMU)

Jan 19, 2026J/K move · Enter open · S save

Zero-Shot Lombard Speech Synthesis with Controllable Style Embeddings

Seymanur Akti, Alexander Waibel

KIT Campus Transfer GmbH (KCT) · Karlsruhe Institute of Technology (KIT) · Carnegie Mellon University (CMU)

TTS Synthesis Zero-Shot TTS Controllable Speech Generation Speech Language Models

Open paper AskPDF arXiv

The Lombard effect plays a key role in natural communication, particularly in noisy environments or when addressing hearing-impaired listeners. We present a controllable text-to-speech (TTS) system capable of synthesizing Lombard-like speech in a zero-shot manner without requiring Lombard-specific training data. Our approach extends F5-TTS with a learned style embedding representation and analyzes the resulting latent space using principal component analysis (PCA) to identify directions associated with Lombard-related attributes. By manipulating these directions, we obtain interpretable control over vocal effort and articulation and generate speech at different Lombard levels. Experimental results show that the proposed method preserves speaker identity and naturalness, improves intelligibility under noisy conditions, and generalizes to previously unseen speakers. These findings demonstrate that style-embedding manipulation provides an effective and scalable framework for controllable zero-shot Lombard speech synthesis.