As modern text-to-speech systems become increasingly intelligible and natural, evaluation is shifting from detecting gross synthesis failures to identifying subtle departures from lifelike human delivery. Because large-scale listening tests are costly and slow, speech reward models can serve as scalable automatic judges for comparing generated utterances, curating data, and supporting iterative TTS development. Existing speech reward models can predict scores or pairwise preferences and may generate natural-language rationales, but their supervision is typically defined at the utterance or pair level. The connection between local acoustic phenomena and the final preference therefore remains implicit. We introduce REAL, a framework for Reward modeling through Evidence-linked Analysis of Lifelikeness in generated speech. Its central idea is to supervise not only the final pairwise preference but also a structured account of its supporting observations, linking local audio spans to perceptual phenomena, affected dimensions, and the overall judgment. Specifically, REAL-Jury constructs these evidence-linked reports through timestamp alignment, multi-model analysis, and dispute-oriented deliberation. REAL-Judge learns to generate them from paired audio inputs. REAL-Eval assesses preference prediction in two tiers, from salient perceptual differences to subtle distinctions among strong TTS systems. Instantiated with MiMo-Audio, REAL-Judge improves pairwise preference accuracy over its backbone by 16.01 and 11.41 percentage points on the two tiers, respectively, and outperforms the open-source speech reward models SpeechJudge and UniSRM on both tiers.
Figures & tables
Figure 1: Overview of REAL-Jury. Independent analyses and timestamp alignment are iteratively reconciled into structured reports, which are filtered by human preferences to construct evidence-linked training data for REAL-Judge.
Figure 2: Training pipeline of REAL-Judge. A LoRA-adapted speech language model distills REAL-Jury’s evidence-linked reports from instructions, reference text, and time-marker-augmented audio.
Figure 3: Composition of REAL-Eval. (a) REAL-Eval-Standard includes regular and expressive comparisons under cross-lingual and monolingual reference-to-target settings. (b) REAL-Eval-Fine covers six text domains and utterances generated by four strong TTS systems.
Model
REAL-Eval-Standard
REAL-Eval-Fine
AVG
Regular
Expressive
Micro
Smart Speaker
Chat
News
Customer Service
Story
Novel
Micro
Lower-CER
20.50
17.00
18.75
5.71
9.09
3.12
9.38
6.82
6.45
6.76
9.76
Gemini-3.1-Pro-Preview
82.33
77.33
79.83
53.34
67.68
72.92
61.46
75.00
72.04
67.31
70.26
Qwen3.5-Omni
81.47
78.17
79.82
65.71
70.71
71.88
62.50
59.09
79.57
67.63
71.14
Qwen3-Omni
81.00
79.83
80.42
53.33
55.56
67.71
62.50
53.79
59.14
58.29
64.11
Step-Audio-R1.1
74.00
76.67
75.33
54.29
62.63
54.16
61.46
55.31
59.14
57.65
62.21
Table 1: Preference accuracy (%) on REAL-Eval-Standard and REAL-Eval-Fine. “AVG” is the macro-average over the corresponding subsets.
Model
Standard
Fine
REAL-Judge
84.43
65.36
w/o time marker
82.92
63.77
w/ time marker (1s)
83.00
64.07
Table 2: Time-marker ablation. Values are preference accuracy (%).
Figure 4: Predicted versus human-annotated model win rates. The dashed line denotes perfect calibration.
Evaluating speech generation still relies heavily on human judgments, such as Mean Opinion Score (MOS), which are expensive, subjective, and difficult to reproduce at scale. While a few recent studies have begun to explore AudioLLM-based judge models, existing efforts typically target only a narrow set of scenarios (e.g., utterance-level quality or single-turn dialogue) and provide limited coverage of diverse speech generation tasks and evaluation dimensions. In this work, we propose UniSRM, a unified speech reward model that can support multi-dimensional, interpretable reward signals with reliable reasoning. To support training and evaluation, we introduce UniSRM-Data and UniSRM-Bench, covering speech evaluation tasks from utterance-level quality to context-level coherence. Based on this dataset, we present the unified speech reward model, UniSRM, with a two-stage pipeline that enables reasoning-based fine-grained assessment. Furthermore, we introduce Reasoning-Consistent Rewards to improve the reliability of the reasoning process. Experiments show that UniSRM delivers more reliable and human-aligned judgments across a broad range of speech evaluation tasks, offering a practical foundation for scalable and unified evaluation of speech quality.
Yuanyuan Wang, Dongchao Yang, Yayue Deng +4
The Chinese University of Hong Kong · Tsinghua University · Independent Researcher
Current Large Language Models (LLMs) are primarily optimized for written text, often producing outputs that are grammatically correct and helpful yet poorly suited for spoken delivery via Text-to-Speech (TTS). In this work, we study how to make LLMs natively generate TTS-friendly text, which we frame as a preference alignment problem: instead of relying on downstream rewriting modules, we directly align LLMs to generate text optimized for spoken delivery. We introduce two preference datasets spanning different target domains, CORA and Recipe, which contain paired TTS-friendly and TTS-unfriendly responses. We further propose an evaluation suite combining a pattern-based heuristic metric, a TTS→ASR evaluation pipeline, and a MUSHRA listening study with human judges. Our experiments compare the recently proposed Feature-aware Sampling and Tuning (FaST) framework -- leveraging interpretable features instead of a black-box reward model -- against an array of alignment baselines on the TTS-friendly generation task. Notably, we found that FaST achieves the best overall tradeoff between TTS-friendliness and helpfulness across various settings. We also identified a strong correlation between our different metrics, highlighting the ability to reliably assess TTS-friendliness via an efficient heuristic.
Automated Text-to-Speech (TTS) evaluation methods (Mean Opinion Score (MOS) predictors and Audio Large Language Model (Audio-LLM) judges) are expected to reflect human perception, yet it is unclear how well they capture the distinct aspects of speech that listeners actually perceive. We deconstruct "naturalness" into a linguistically grounded annotation schema spanning 10 distinct perceptual dimensions, and use it to construct the first dimension-level meta-evaluation benchmark for TTS, comprising 860 utterances annotated by trained linguist raters. Results from benchmarking four MOS predictors and four Audio-LLM judges reveal that MOS predictors are more sensitive to acoustic signal-level properties at the prosodic and paralinguistic levels, while Audio-LLM judges show selective, prompt-dependent detection that favors word-level errors and does not generalize across all dimensions. Neither class reliably captures a breadth of linguistically structured, human-perceptible speech errors. Our dataset, annotation schema, and evaluation code are publicly released to support more targeted and interpretable TTS evaluation.