EmpDPO: Enabling Emphasis Control in TTS via Direct Preference Optimization

Haonan Chen; Yunyun Wang; Megan Wei; Zeyu Jin

cs.SDOct 29, 2026

EmpDPO: Enabling Emphasis Control in TTS via Direct Preference Optimization

Authors: Haonan Chen, Yunyun Wang, Megan Wei, Zeyu Jin

Organizations: Adobe Research, USA · Brown University, USA

Abstract

Word-level emphasis conveys meaning that text alone cannot express and is therefore essential for a practical text-to-speech (TTS) system. In this work, we study how to add emphasis control post hoc to a pretrained TTS model that has never been trained with emphasis annotations. We present EmpDPO, a simple, efficient, and effective two-stage recipe that adds emphasis control to a multilingual zero-shot diffusion TTS model through supervised fine-tuning followed by direct preference optimization (DPO). The training signals for both stages come from an off-the-shelf multilingual emphasis detector, so that no manual emphasis annotation or task-specific reward design is needed. We find that DPO can substantially strengthen emphasis control with only a few thousand preference pairs and training steps. Objective and subjective evaluations against strong public systems show that our model produces perceptible emphasis while preserving naturalness and overall quality.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of EmpDPO . SFT stage: a multilingual emphasis detector labels real speech with emphasis markers. DPO stage: the same detector scores candidates sampled from the SFT model to build chosen/rejected pairs for Diffusion-DPO.

System	$\bar{\delta}\uparrow$	CER plain / emph. $\downarrow$	SIM plain / emph. $\uparrow$	UTMOS plain / emph. $\uparrow$	NISQA plain / emph. $\uparrow$
Ours-Base	—	0.008 / —	0.49 / —	3.61 / —	4.34 / —
Ours-SFT	0.069 $\pm$ 0.003	0.008 / 0.007	0.47 / 0.47	3.60 / 3.58	4.42 / 4.42
Ours-DPO	0.136 $\pm$ 0.005	0.008 / 0.008	0.49 / 0.49	3.60 / 3.55	4.36 / 4.34
Fish-S2	0.129 $\pm$ 0.005	0.039 / 0.034	0.51 / 0.51	4.20 / 4.20	4.42 / 4.39
Qwen3-TTS	0.017 $\pm$ 0.003	0.034 / 0.036	N/A	3.56 / 3.48	4.18 / 4.06
WordVoice (en/zh)	0.074 $\pm$ 0.008	0.047 / 0.040	0.46 / 0.46	3.92 / 3.86	4.77 / 4.74

Table 1: Objective evaluation results (210 sentences

\times

8 voices = 1,680 inputs per system; 560 for WordVoice).

Figure 2: Listening test interface (English version): emphasis strength of the highlighted word (Q1) and naturalness (Q2) on a 1–5 scale, and whether another word sounds emphasized (Q3).

System	$N$	Q1 $\uparrow$	Q2 $\uparrow$	Q3 $\downarrow$
Ours-Base	364	2.73 $\pm$ 0.08	3.45 $\pm$ 0.08	42%
Ours-SFT	364	3.15 $\pm$ 0.08	3.52 $\pm$ 0.08	34%
Ours-DPO	364	3.56 $\pm$ 0.07	3.47 $\pm$ 0.08	32%
Fish-S2	364	3.25 $\pm$ 0.08	2.90 $\pm$ 0.11	29%
WordVoice (en/zh)	140	3.37 $\pm$ 0.12	3.13 $\pm$ 0.15	36%

Table 2: Subjective evaluation results: emphasis strength (Q1), naturalness (Q2), and mis-emphasis rate (Q3).

N

: number of native listeners; mean

\pm

95% CI over participants.

Figure 3: Per-language subjective results: Q1 emphasis strength (top) and Q2 naturalness (bottom). WordVoice covers only en/zh. Bars: mean over participants; error bars: 95% CI.

Variant	$\bar{\delta}\uparrow$	CER $\downarrow$	LUFS	UTMOS $\uparrow$	NISQA $\uparrow$
SFT (init)	0.069 $\pm$ 0.003	0.007	$-$ 19.7	3.58	4.42
SFT $+$ 10k	0.069 $\pm$ 0.003	0.005	$-$ 19.7	3.56	4.41
DPO-chosen	0.136 $\pm$ 0.005	0.006	$-$ 14.0	3.59	3.74
DPO-corpus	0.136 $\pm$ 0.005	0.008	$-$ 19.6	3.55	4.34

Table 3: Replay strategies, evaluated on the emphasis outputs. SFT

+

10k: SFT trained for 10k more steps; corpus / chosen: replay on real speech / on self-generated chosen samples; LUFS: integrated loudness.

TTS Synthesis

Text-to-speech synthesis converts written language into intelligible, natural-sounding speech with control over voice, prosody, style, and timing.

See all 278 papers

EmphTTS: an emphasis-control TTS with reinforcement learning
Sep 23, 2026Zirui Li, Rech Silas, Lauri Juvela +2TTS Synthesis Controllable Speech Generation
Department of Information and Communications Engineering, Aalto University, Espoo, Finland
Post-Training Zero-Shot TTS for Fine-Grained Emotion and Duration Control via Natural Language
Sep 10, 2026Lianru Gao, Yujie Guo, Yong QinTTS Synthesis Zero-Shot TTS
TMCC, College of Computer Science, Nankai University, Tianjin, China
Emo-LiPO: Listwise Preference Optimization for Fine-Grained Emotion Intensity Control in LLM-based Text-to-Speech
Jun 11, 2026Yihang Lin, Li Zhou, Congwei Cao +4TTS Synthesis Preference Optimization
The Chinese University of Hong Kong, Shenzhen · Shenzhen Loop Area Institute · Agency for Science, Technology and Research +2

Sep 23, 2026J/K move · Enter open · S save

EmphTTS: an emphasis-control TTS with reinforcement learning

Zirui Li, Rech Silas, Lauri Juvela, Tom Backstrom+1

Department of Information and Communications Engineering, Aalto University, Espoo, Finland

TTS Synthesis Controllable Speech Generation Speech Prosody

Open paper AskPDF arXiv

Generating controllable and human-like emphasis remains an open challenge in text-to-speech, even when explicit emphasis control signals are provided in the text input, limiting the communicative accuracy of synthetic speech in real-world applications. Reinforcement learning has recently shown promise for post-training TTS systems to align with human preference, yet existing methods have not been applied to word-level prosodic control. We present EmphTTS, a non-autoregressive TTS system that applies Group Relative Policy Optimization (GRPO) to the duration predictor with an emphasis localization reward, enabling direct optimization for word-level emphasis. Evaluations show that EmphTTS achieves the best emphasis controllability and performs the best in emphasis objective evaluation. In subjective preference tests, EmphTTS is significantly preferred over synthetic groundtruth and most baselines. Ablation studies show that GRPO improves emphasis realization beyond supervised-finetuning-based duration modeling and simple speaking-rate adjustment, while alleviating the mismatch between the independently trained duration predictor and TTS model.

Abstract

TTS Synthesis

Explore similar work

EmphTTS: an emphasis-control TTS with reinforcement learning

Post-Training Zero-Shot TTS for Fine-Grained Emotion and Duration Control via Natural Language

Emo-LiPO: Listwise Preference Optimization for Fine-Grained Emotion Intensity Control in LLM-based Text-to-Speech