The VoiceMOS Challenge 2026: Evaluating Speech Enhancement, Emotional TTS and Accented TTS Systems

Wen-Chin Huang; Wei Wang; Marvin Sach; Xiaoxue Gao; Nicholas Sanders; Erica Cooper; Toda Tomoki

cs.SDSep 12, 2026

The VoiceMOS Challenge 2026: Evaluating Speech Enhancement, Emotional TTS and Accented TTS Systems

Authors: Wen-Chin Huang, Wei Wang, Marvin Sach, Xiaoxue Gao, Nicholas Sanders, Erica Cooper, Toda Tomoki

Organizations: Nagoya University, Japan · Shanghai Jiao Tong University, China · Technische Universität Braunschweig, Germany · The Chinese University of Hong Kong, Shenzhen, China ·

Abstract

We present the results of the VoiceMOS Challenge 2026, the fifth edition of a scientific challenge on automatic prediction of subjective speech assessments. After expanding the scope to music and general audio in 2025, we refocused the evaluation target on speech and organized three tracks: prediction of absolute and comparative category ratings for enhanced speech, prediction of naturalness and emotion-related tasks for emotional text-to-speech systems, and prediction of speaker and accent similarity for codec-based speech synthesis systems. The challenge attracted a total of 18 teams worldwide, with most teams successfully surpassing the provided baselines. We summarize the challenge results, representative top-performing systems, participant feedback, and directions for future editions.

Figures & tables

Track	Evaluated systems	Prediction targets	# Samples			# Systems			# ratings per sample
Track	Evaluated systems	Prediction targets	Train	Dev	Test	Train	Dev	Test	# ratings per sample
1	Speech enhancement	ACR, CCR	–	ACR: 1008 CCR: 2520	ACR: 4032 CCR:10080	–	6	6	8
2	Emotional TTS	QMOS, EMOS optional: V/A/D, perceived emotion	12746	2730	2730	10	13	14	4-9
3	Accented TTS	Speaker & accent similarity	2800	600	600	21	23	25	4.9

Table 1: Summary of each track.

Affiliation	Track
Affiliation	1	2	3
Advanced Knowledge Center for Immersive Technologies, Brazil	V
Microsoft, Germany	V
BITS Pilani, Hyderabad Campus, India	V
Paderborn University, Germany	V
UTokyo-AIST-KAIST, Japan & South Korea		V

Table 2: List of participant affiliations, in random order.

Panel of Figure 1 — Figure 1: Utterance-level SRCC results for the three tracks. Hollow gray markers denote provisional or incomplete submissions. There were 157 missing samples in the T10 submission, and metrics were calculated using the 2574 available samples. T13 only submitted ACC results.

Figure 2: Sign accuracy of Track 1 CCR predictions as a function of

|\mathrm{CCR}|

, where sign accuracy measures whether the predicted CCR has the same sign as the ground-truth CCR. Results are computed over 0.1-wide

|\mathrm{CCR}|

bins and smoothed for visualization.

Figure 3: Track 2 optional emotion-related prediction task results. Bars show UTT-SRCC for valence (V), arousal (A), and dominance (D), together with UTT-CAT-ERR for emotion-category prediction. UTT-SRCC values are shown on the left axis, while UTT-CAT-ERR is shown on the right axis; higher UTT-SRCC and lower UTT-CAT-ERR indicate better performance.

Speech Quality Assessment

Speech quality assessment measures how well speech preserves perceptual quality, intelligibility, pronunciation, naturalness, and speaker characteristics.

See all 69 papers

Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions
Aug 10, 2026Oluwanifemi Bamgbose, Jash Shah, Simon Rosen +6Audio-Language Model Evaluation Human Preference Evaluation
ServiceNow
CodecMOS-Accent: A MOS Benchmark of Resynthesized and TTS Speech from Neural Codecs Across English Accents
Mar 15, 2026Wen-Chin Huang, Nicholas Sanders, Erica CooperTTS Evaluation Speech Quality Assessment
Nagoya University, Japan · University of Edinburgh, United Kingdom · National Institute Of Information And Communications Technology, Japan
A Comprehensive Objective Evaluation of Modern Text-to-Speech for Turkish Using Speech Quality Assessment Models
Oct 5, 2026Yunus Emre Ozkose, Alperen Kahraman, Ali HaznedarogluTTS Synthesis TTS Evaluation
Sestek, Ankara, Türkiye

Aug 10, 2026J/K move · Enter open · S save

Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions

Oluwanifemi Bamgbose, Jash Shah, Simon Rosen, Lindsay Devon Brin+5

ServiceNow

Audio-Language Model Evaluation Human Preference Evaluation TTS Synthesis Speech Generation Evaluation TTS Evaluation Speech Quality Assessment+1

Open paper AskPDF arXiv

Automated Text-to-Speech (TTS) evaluation methods (Mean Opinion Score (MOS) predictors and Audio Large Language Model (Audio-LLM) judges) are expected to reflect human perception, yet it is unclear how well they capture the distinct aspects of speech that listeners actually perceive. We deconstruct "naturalness" into a linguistically grounded annotation schema spanning 10 distinct perceptual dimensions, and use it to construct the first dimension-level meta-evaluation benchmark for TTS, comprising 860 utterances annotated by trained linguist raters. Results from benchmarking four MOS predictors and four Audio-LLM judges reveal that MOS predictors are more sensitive to acoustic signal-level properties at the prosodic and paralinguistic levels, while Audio-LLM judges show selective, prompt-dependent detection that favors word-level errors and does not generalize across all dimensions. Neither class reliably captures a breadth of linguistically structured, human-perceptible speech errors. Our dataset, annotation schema, and evaluation code are publicly released to support more targeted and interpretable TTS evaluation.

Abstract

Speech Quality Assessment

Explore similar work

Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions

CodecMOS-Accent: A MOS Benchmark of Resynthesized and TTS Speech from Neural Codecs Across English Accents

A Comprehensive Objective Evaluation of Modern Text-to-Speech for Turkish Using Speech Quality Assessment Models