Speech Generation Evaluation

Overview

Speech generation evaluation asks whether synthesized speech says the right words, sounds natural, preserves speaker identity, and follows instructions about prosody, emotion, or conversational style. Approaches include ASR-based WER and CER, acoustic and MOS-prediction metrics, speaker and emotion embeddings, human listening tests, and pairwise preference models or large audio-language model judges. Benchmarks increasingly test multilingual and domain-specific TTS, fine-grained paralinguistic control, nonverbal vocalizations, and timing and turn-taking in spoken dialogue. A central tension is that automatic scores can miss prosodic, linguistic, or contextual errors that people notice, while gains in intelligibility or semantic alignment may come at the expense of voice identity, expressiveness, or conversational naturalness.

Latest papers

CardsList

62 papers6 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

  1. Human-Likeness and Conversational Preference Diverge for Synthesized Agent Speech in Customer-Service Calls

    Nov 11, 2026Jeffrey M. Girard, Antony D'Avirro, Jason Z. Zheng +2Voice Agent EvaluationTTS Evaluation

  2. REAL: Reward Modeling through Evidence-Linked Analysis of Lifelikeness in Generated Speech

    Nov 9, 2026Yaodong Song, Jie Lian, Xusheng Yang +6Reward ModelingSpeech Generation Evaluation

  3. Beyond Utterance-Level Speaker Similarity: Perception-Aligned Segmental Evaluation of Accent in Speech Generation

    Oct 27, 2026Jinzuomu Zhong, Alice Ross, Catherine Lai +1Speech Generation EvaluationTTS Evaluation

  4. DRAMA-Eval: A Benchmark for Context-Aware, Multi-Turn, Multi-Speaker Speech Generation

    Oct 22, 2026Jingbin Hu, Zihang Yang, Xiaoge Peng +21Speech Generation EvaluationMulti-Speaker TTS

  5. Beyond the Benchmark: Conditioning, Domains and Failure Modes of Qwen3-TTS

    Oct 16, 2026Séverin Baroudi, Yanis Labrak, Petr Motlicek +2TTS SynthesisVoice Cloning

  6. Long-Prompt Zero-Shot TTS: Benchmarking Prosody Reproduction Beyond Voice Cloning

    Oct 14, 2026Dekun Chen, Renjie Xu, Yingda Shen +2Zero-Shot TTSSpeech Prosody

  7. The NVVSpeech Challenge 2026: Overview and Lessons for Speech Understanding and Generation with Nonverbal Vocalizations

    Oct 12, 2026Liumeng Xue, Yuang Cao, Jingbin Hu +12Nonverbal Vocalization GenerationAudio Understanding

  8. Refinement Buys Intelligibility, Search Buys Identity: What Test-Time Compute Buys in Masked-Diffusion TTS

    Oct 2, 2026Nityanand Mathur, Hamees Sayed, Ayush Pratap SinghTTS SynthesisText-to-Speech

  9. Intonation Perception in Real and Synthetic Speech across Varying Familiarity Levels: A Pilot Study of Equivalence Assessment

    Sep 30, 2026Hanrui Zhou, Gaoyuan Zhang, Yixiang Chen +4Speech ProsodySpeech Generation Evaluation

  10. Toward Human-Aligned Judgement of Speech Emotion Similarity

    Sep 26, 2026Yun-Shao Tsai, Yi-Cheng Lin, Chih-Kai Yang +7Speech Generation EvaluationEmotional Speech Synthesis

  11. A Comprehensive Study of Content Representations for Speech Synthesis

    Sep 25, 2026Diego Torres, Axel Roebel, Nicolas ObinAudio Representation LearningDiscrete Speech Representation Learning

  12. ττ-Multilingual: Benchmarking Voice Agents Across Languages

    Sep 21, 2026Soham Ray, Edgard dos Santos Paiva, Ruben Valenzuela +3Voice Agent EvaluationFull-Duplex Spoken Dialogue Systems

  13. XSQ-AST: An Explainable Audio Spectrogram Transformer Framework for Localising Synthetic Speech Artifacts

    Sep 21, 2026Ben Heritage, Luca Resti, Mónica Villanueva Aylagas +3Explainable Artificial IntelligenceSpeech Generation Evaluation

  14. HaikuS2S: A Cascaded System For Responding In Verse

    Sep 20, 2026Devangi Sharma, Sophia Judicke, Glenda Tan +2TTS SynthesisSpeech Generation Evaluation

  15. Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis

    Sep 17, 2026Zifan Guan, Longyu Lu, Junan Zhang +3Speech Generation EvaluationTTS Evaluation

  16. Quantifying the Generation Modality Gap in Speech-Text Language Models

    Sep 13, 2026Ju-Chieh Chou, Jiawei Zhou, Karen LivescuSpeech Generation EvaluationSpeech Language Models

  17. Voice or Stereotype? Disentangling Acoustic and Content-Based Gender in Speech-to-Speech Models

    Sep 8, 2026Xiaoqun Liu, Tanu Mitra, Harshit Rajgarhia +1Speech Generation EvaluationSpeech Processing

  18. Preference Optimization for Non-Verbal Vocalization Synthesis

    Aug 25, 2026Haoyang Li, Chenglin Xu, Junchuan Zhao +4Nonverbal Vocalization GenerationPreference Optimization

  19. MMAG: A Multi-Control Mixed Audio Generation Benchmark

    Aug 7, 2026Zihao Zheng, Xuenan Xu, Jiahao Mei +5Speech Generation EvaluationAudio Generation

  20. Domain-Specific Evaluation of Text-to-Speech Systems: A Multi-Metric Benchmarking Study

    Aug 3, 2026Ali Jafar, Amal Sarmad, Shifa Yousaf +1Speech Generation EvaluationTTS Evaluation

  21. Beyond Prompt Adherence: Auditing Attribute-Level Voice Control in Speech Generation

    Aug 1, 2026Xianhao Zhou, Jianghao WuSpeech Generation EvaluationTTS Evaluation