Emotional Speech Synthesis

Ask about this topic

Overview

Emotional speech synthesis studies how to make generated speech convey feelings, intensity, and changing affect while keeping the words clear and the voice consistent. Systems use emotion labels or natural-language instructions, prosody controls such as pitch and duration, and newer autoregressive, diffusion, and flow-matching models; preference learning and activation steering offer ways to refine control. Some work also uses dialogue context or visual cues to plan an emotionally appropriate response. A central challenge is balancing expressive, fine-grained control with naturalness, intelligibility, and speaker identity, while ensuring automatic emotion measures reflect human judgments.

Latest papers

CardsList

47 papers9 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

  1. EmoInstruct-TTS: Dual-Path Instruction-Guided Emotional Speech Synthesis

    Jun 8, 2026Minghui Wu, Ganjun Liu, Zikun Fang +6Representation LearningTTS Synthesis

    University of Science and Technology of China, China · iFLYTEK Research, China · Huawei Technologies Co., Ltd., China

  2. Task-Vector Arithmetic for Emotional Expressivity Control in Language-Model-Based Text-to-Speech

    Jun 3, 2026Daniel Oliveira de Brito, Arnaldo Candido JuniorTTS SynthesisSpeaker Embedding

    Instituto de Biociências, Letras e Ciências Exatas Universidade Estadual Paulista “Júlio de Mesquita Filho” (UNESP) São José do Rio Preto, SP, Brazil

  3. Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speech

    May 31, 2026Hongfei Du, Jiacheng Shi, Sidi Lu +2TTS SynthesisSparse Autoencoders

    Department of Computer Science, William & Mary, USA.

  4. Sympatheia: Emotionally Adaptive Voice Assistant with Continuous Affect Conditioning

    May 30, 2026Sukru Samet Dindar, Riki Shimizu, Xilin Jiang +1Spoken Dialogue SystemsControllable Speech Generation

    Department of Electrical Engineering Columbia University New York, NY 10025

  5. PilotTTS: A Disciplined Modular Recipe for Competitive Speech Synthesis

    May 26, 2026Bowen Li, Shaotong Guo, Zhen Wang +11TTS SynthesisZero-Shot TTS

    Amap, Alibaba Group · The Chinese University of Hong Kong, Shenzhen

  6. DUET: Unified Dual-Space Emotion Control for Diffusion and Flow-Matching Driven Text-to-Speech

    May 20, 2026Xu Zhang, Longbing Cao, Zhangkai WuTTS SynthesisAudio Diffusion Models

    Frontier AI Research Centre, Macquarie University, Sydney, NSW, Australia

  7. The False Resonance: A Critical Examination of Emotion Embedding Similarity for Speech Generation Evaluation

    Apr 29, 2026Yun-Shao Tsai, Yi-Cheng Lin, Huang-Cheng Chou +5TTS SynthesisSpeech Generation Evaluation

    Graduate Institute of Communication Engineering, National Taiwan University, Taiwan · Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, USA · Gilbert AI Lab, USA +2

  8. Self-EmoQ: Plutchik-Guided Value-based Planning to Drive Streaming Emotional TTS

    Apr 21, 2026Yue Zhao, Hongyan Li, Yong Chen +1TTS SynthesisStreaming TTS Synthesis

    Geely AI Lab

  9. ATRIE: Adaptive Tuning for Robust Inference and Emotion in Persona-Driven Speech Synthesis

    Apr 21, 2026Aoduo Li, Haoran Lv, Hongjian Xu +5TTS SynthesisControllable Speech Generation

    Guangdong University of Technology Guangzhou, China · South China University of Technology Guangzhou, China · Shenzhen Polytechnic University Shenzhen, China +2

  10. Bridging What the Model Thinks and How It Speaks: Expressive Speech Generation via Self-Aware Intent-Realization Alignment

    Apr 13, 2026Kuang Wang, Lai Wei, Ping Lin +8Expressive Speech SynthesisSpeech Language Models

    The Chinese University of Hong Kong, Shenzhen · Tencent Ethereal Audio Lab · Southeast University +2

  11. AffectSpeech: A Large-Scale Emotional Speech Dataset with Fine-Grained Textual Descriptions for Speech Emotion Captioning and Synthesis

    Apr 5, 2026Tianhua Qi, Wenming Zheng, Björn W. Schuller +2Emotional Speech SynthesisSpeech Emotion Recognition

    School of Biological Science and Medical Engineering, Southeast University, Nanjing 210096, China · Key Laboratory of Child Development and Learning Science (Southeast University), Ministry of Education, China · Chair of Health Informatics (CHI), Technical University of Munich, 80333 Munich, Germany +3

  12. MOSS-VoiceGenerator: Create Realistic Voices with Natural Language Descriptions

    Mar 30, 2026Kexin Huang, Liwei Fan, Botian Jiang +11Controllable Speech GenerationTTS Synthesis

  13. SelfTTS: cross-speaker style transfer through explicit embedding disentanglement and self-refinement using self-augmentation

    Mar 23, 2026Lucas H. Ueda, João G. T. Lima, Pedro R. Corrêa +3Disentangled Representation LearningTTS Synthesis

    Universidade Estadual de Campinas (UNICAMP), Brazil · CPQD, Brazil

  14. Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Models

    Mar 18, 2026Xiutian Zhao, Ismail Rasim Ulgen, Philipp Koehn +2Controllable Speech GenerationEmotional Speech Synthesis

    Center for Language and Speech Processing (CLSP), Johns Hopkins University, USA · Group on Language, Audio & Music (GLAM), Imperial College London, UK

  15. Affectron: Emotional Speech Synthesis with Affective and Contextually Aligned Nonverbal Vocalizations

    Mar 15, 2026Deok-Hyeon Cho, Hyung-Seok Oh, Seung-Bin Kim +1Nonverbal Vocalization GenerationEmotional Speech Synthesis

    Department of Artificial Intelligence, Korea University, Seoul, Korea

  16. Causal Prosody Mediation for Text-to-Speech:Counterfactual Training of Duration, Pitch, and Energy in FastSpeech2

    Mar 12, 2026Suvendu Sekhar MohantyTTS SynthesisSpeech Prosody

    Arlington, Virginia, USA

  17. EmoOmni: Bridging Emotional Understanding and Expression in Omni-Modal LLMs

    Feb 25, 2026Wenjie Tian, Zhixian Zhao, Jingbin Hu +4Multimodal Large Language ModelsEmotion-Aware Dialogue Generation

    Northwestern Polytechnical University, Xi’an, China · University of Surrey, Guildford, United Kingdom