TTS Evaluation

TTS: Text-to-Speech

Overview

Text-to-speech evaluation asks whether generated speech is intelligible, natural, faithful to a target voice, and appropriate to its context. Common approaches combine human MOS or pairwise preference tests with objective measures such as ASR error, speaker similarity, acoustic features, and learned quality predictors. Benchmarks increasingly probe specific abilities—including accent and phonology, emotion and stress, instruction following, multilingual performance, and long-form consistency—rather than relying on a single overall score. A central challenge is that automatic evaluators often miss prosodic or linguistic errors and may disagree with listeners, so reliable assessment requires multiple dimensions, careful human validation, and tests matched to the intended setting.

Latest papers

CardsList

63 papers11 papers in the last four weeks, down 8% on the four weeks before. 0.1% of all new papers.

  1. No-Shot Text-to-Speech: Limitations of Zero-Shot TTS and its Evaluation Methods in Representing Queer and Transgender Voices

    Nov 16, 2026Juliana Francis, Robin Netzorg, Joakim Gustafson +1Zero-Shot TTSTTS Evaluation

  2. Beyond Controllability: Speaker Diversity in Instruction-Guided Text-to-Speech

    Nov 14, 2026Yi-Cheng Lin, Jia-Kai Dong, Kuan-Yu Chen +5TTS SynthesisText-to-Speech

  3. Human-Likeness and Conversational Preference Diverge for Synthesized Agent Speech in Customer-Service Calls

    Nov 11, 2026Jeffrey M. Girard, Antony D'Avirro, Jason Z. Zheng +2Voice Agent EvaluationTTS Evaluation

  4. REAL: Reward Modeling through Evidence-Linked Analysis of Lifelikeness in Generated Speech

    Nov 9, 2026Yaodong Song, Jie Lian, Xusheng Yang +6Reward ModelingSpeech Generation Evaluation

  5. NepTTS-Bench: Native-Speaker Benchmarking and NepaliMOS for Nepali Text-to-Speech

    Oct 31, 2026Ashish Thapa, Rupak Ghimire, Lekhnath Sharma PathakTTS SynthesisTTS Evaluation

  6. Scalable Audio-LLM System for Speech MOS Evaluation: A Diagnostic Case Study and Adaptation Roadmap

    Oct 30, 2026Dimitrios Kokkinis, Panagiotis Kakoulidis, Nikolaos Ellinas +10Speech Quality AssessmentTTS Evaluation

  7. Beyond Utterance-Level Speaker Similarity: Perception-Aligned Segmental Evaluation of Accent in Speech Generation

    Oct 27, 2026Jinzuomu Zhong, Alice Ross, Catherine Lai +1Speech Generation EvaluationTTS Evaluation

  8. ConformalMOS: Uncertainty-Aware MOS Prediction with Conformal Intervals and Ordinal Modeling

    Oct 26, 2026Kehinde Elelu, Joshua E. Siegel, Mohammadali Saffary +3Speech Quality AssessmentTTS Evaluation

  9. Measuring the Verifier's Floor: How Much Apparent Rare-Word Text-to-Speech Error Is ASR Error?

    Oct 22, 2026Kirill Borodin, Vasilii Kudryavtsev, Nikita Vasiliev +1TTS EvaluationAutomatic Speech Recognition

  10. Does Expert Supervision Improve Text-to-Speech? An Empirical Study of Preference Optimization

    Oct 18, 2026Minh Duc Vu, Junchuan Zhao, Ye WangTTS SynthesisText-to-Speech

  11. ShlokaDhara: A Multi-Speaker Sanskrit Speech Corpus for Learning Meter-Aware Prosody in Text-to-Recitation

    Oct 17, 2026Harshal Kausadikar, Onkar Kishor Susladkar, Prathmesh Santosh Choudhari +3Speech ProcessingTTS Synthesis

  12. Long-Prompt Zero-Shot TTS: Benchmarking Prosody Reproduction Beyond Voice Cloning

    Oct 14, 2026Dekun Chen, Renjie Xu, Yingda Shen +2Zero-Shot TTSSpeech Prosody

  13. Speaking Longer Than Learned: Training-Free Long-Form Generation for Full-Stream Text-to-Speech

    Oct 14, 2026Jeseung Lim, Jiseong Woo, Heeseung KimLong-Form TTS SynthesisTTS Synthesis

  14. Cross-Lingual TTS for Indian Languages: A Benchmark and Inference-Time Duration Estimation Strategies

    Oct 10, 2026Adarsh Arigala, Arjun Gangwar, Srinivasan UmeshMultilingual TTS SynthesisTTS Synthesis

  15. How Should We Evaluate Naturalness for Synthetic Speech with a Foreign Accent?

    Oct 9, 2026Kentaro Onda, Satoru Fukayama, Daisuke Saito +1TTS EvaluationSpeech Quality Assessment

  16. Balalaika-Longform: A Russian Speech Corpus for Continuous Long-Form Text-to-Speech

    Sep 30, 2026Nikita Vasiliev, Kirill Borodin, Vasilii Kudryavtsev +2TTS SynthesisTTS Evaluation

  17. ProxyMOS: Label-Free Speech Quality Assessment by Multi-Teacher Distillation with Adaptive Routing

    Sep 30, 2026Maxim Trokunov, Kirill Borodin, Nikita Vasiliev +1Ensemble LearningTTS Evaluation

  18. Repetition, Not Length: Isolating the Counting Failure in Neural Text-to-Speech

    Sep 29, 2026Kirill Borodin, Vasilii Kudryavtsev, Maxim Maslov +1TTS SynthesisTTS Evaluation

  19. NADI 2026: The Second Multidialectal Arabic Speech Processing Shared Task

    Sep 22, 2026Peter Sullivan, Bashar Talafha, Ahmed Ashraf +11ASR EvaluationSpoken Language Understanding

  20. GenTraceBench: A Benchmark for Tracing Audio Deepfakes Across Pre- and Post-training Stages

    Sep 18, 2026Li Wang, Kunyu Feng, Wan Lin +7Audio Deepfake DetectionAudio Deepfake Detection Robustness

  21. Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis

    Sep 17, 2026Zifan Guan, Longyu Lu, Junan Zhang +3Speech Generation EvaluationTTS Evaluation

  22. Taming Long-form Text-to-Speech

    Sep 15, 2026Rongxiang Wang, Berkin Durmus, Aysegul Orhon +2TTS SynthesisTTS Evaluation

  23. Tone on a Budget: A Reference-Free Metric for Lexical Tone in Massively Multilingual Text-to-Speech

    Sep 13, 2026Moses Daudu, Adeola Enitan Bamidele, Honor-Jesus BezaleelTTS SynthesisTTS Evaluation

  24. The VoiceMOS Challenge 2026: Evaluating Speech Enhancement, Emotional TTS and Accented TTS Systems

    Sep 12, 2026Wen-Chin Huang, Wei Wang, Marvin Sach +4Speech Quality AssessmentTTS Evaluation

  25. Complex-Text Robustness Evaluation and Failure Diagnosis for Low-Resource Multilingual Text-to-Speech

    Sep 10, 2026Tianlun Zuo, Ziyu Zhang, Tingzhi Mao +2TTS SynthesisTTS Evaluation