Calibration-Reasoning Framework for Descriptive Speech Quality Assessment

Elizaveta Kostenok; Mathieu Salzmann; Milos Cernak

eess.ASMar 10, 2026

Calibration-Reasoning Framework for Descriptive Speech Quality Assessment

Authors: Elizaveta Kostenok, Mathieu Salzmann, Milos Cernak

Organizations: EPFL, Lausanne, Switzerland · Logitech, Lausanne, Switzerland

Abstract

Explainable speech quality assessment requires moving beyond Mean Opinion Scores (MOS) to analyze underlying perceptual dimensions. To address this, we introduce a novel post-training method that tailors the foundational Audio Large Language Model for multidimensional reasoning, detection and classification of audio artifacts. First, a calibration stage aligns the model to predict predefined perceptual dimensions. Second, a reinforcement learning stage leverages Group Relative Policy Optimization (GRPO) with dimension-specific rewards to heavily enhance accuracy of descriptions and temporal localization of quality issues. With this approach we reach state-of-the-art results of 0.71 mean PCC score on the multidimensional QualiSpeech benchmark and 13% improvement in MOS prediction driven by RL-based reasoning. Furthermore, our fine-grained GRPO rewards substantially advance the model's ability to pinpoint and classify audio artifacts in time.

Figures & tables

Figure 1: Overview of end-to-end fine-tuning pipeline.

Figure 2: Dimension-wise LLM-judge reward.

Baselines
Reasoning method	Learned Enc.	Calibration	Nat.	Noise	Distort.	Effort	Cont.	MOS	Avg.
QualiSpeech-FT-SALMONN	x	✓	0.56	0.65	0.57	0.61	0.45	0.63	0.57
QualiSpeech-FT	x	✓	0.61	0.68	0.58	0.60	0.49	0.64	0.60
SQ-LLM	✓	✓	0.71	0.72	0.60	0.65	0.44	0.68	0.63
Ours
Acc.+Sem. dim.-wise	✓	✓	0.70	0.78	0.59	0.69	0.55	0.72	0.68

Table 1: Pearson Correlation Coefficients (PCC) for dimension-wise speech quality assessment across various reasoning strategies. The ’Nat.’ to ’MOS’ columns represent specific perceptual dimensions from the QualiSpeech dataset, ’Avg.’ refers to the mean of the dimension-wise PCC values.

FT Stage	N iter.	Gr. s.	B.s.	L.r.
Calibration	10k	–	8	1.5e-5
SFT Reasoning	10k	–	8	1.5e-5
GRPO Reasoning	200	4	4	5e-6

Table 2: Hyperparameter configurations across different fine-tuning stages. ’Gr.s.’, ’B.s.’, ’L.r.’ refer to group size in GRPO, batch size and learning rate.

Baselines
Reasoning method	Brief desc.									Long-form desc.
	Noise			Distortion			Unnatural Pause			ROUGE-L	Corr.
	F1	Corr.	IoU	F1	Corr.	IoU	F1	Corr.	IoU	ROUGE-L	Corr.
QualiSpeech-FT-SALMONN	0.44	0.49	0.49	0.49	0.67	0.78	0.53	0.40	0.34	0.38	0.75
QualiSpeech-FT	0.56	0.52	0.44	0.73	0.71	0.81	0.56	0.40	0.25	0.40	0.78
SQ-LLM	0.72	0.57	0.56	0.82	0.68	0.74	0.61	0.39	0.33	0.45	0.82

Table 3: Descriptive accuracy of various reasoning strategies. We report F1 score for artifact detection, Correlation score (Corr.) estimated by GPT-4o for artifact descriptions, and Intersection over Union (IoU) for overlap between detected and reference time intervals; ROUGE-L and Correlation score for long-form descriptions.

Speech Quality Assessment

Speech quality assessment measures how well speech preserves perceptual quality, intelligibility, pronunciation, naturalness, and speaker characteristics.

See all 69 papers

Few-Shot and Pseudo-Label Guided Speech Quality Evaluation with Large Language Models
Apr 15, 2026Ryandhimas E. Zezario, Dyah A. M. G. Wisnu, Szu-Wei Fu +3Speech Quality Assessment
Academia Sinica · National Chengchi University · NVIDIA +1
Investigating Human-Model Discrepancies in Speech Quality Assessment via Acoustic and Prosodic Perturbations
Jun 18, 2026Masato Takagi, Masaya Kawamura, Reo Shimizu +1Human Preference Evaluation Speech Generation Evaluation
Nagoya Institute of Technology, Japan · LY Corporation, Japan
DAMOS: Learning Distortion-Aware Speech Quality Assessment through Explicit Distortion Localization
Aug 21, 2026Naiyuan Li, Li Dong, Diqun YanSpeech Quality Assessment
College of Information Science and Engineering, Ningbo University, Ningbo, China · College of Artificial Intelligence, Ningbo University of Finance and Economics, Zhejiang Provincial Collaborative Innovation Center for Digital Supply Chain and Artificial Intelligence of Bulk Commodities

Apr 15, 2026J/K move · Enter open · S save

Few-Shot and Pseudo-Label Guided Speech Quality Evaluation with Large Language Models

Ryandhimas E. Zezario, Dyah A. M. G. Wisnu, Szu-Wei Fu, Sabato Marco Siniscalchi+2

Academia Sinica · National Chengchi University · NVIDIA+1

Speech Quality Assessment

Open paper AskPDF arXiv

In this paper, we introduce GatherMOS, a novel framework that leverages large language models (LLM) as meta-evaluators to aggregate diverse signals into quality predictions. GatherMOS integrates lightweight acoustic descriptors with pseudo-labels from DNSMOS and VQScore, enabling the LLM to reason over heterogeneous inputs and infer perceptual mean opinion scores (MOS). We further explore both zero-shot and few-shot in-context learning setups, showing that zero-shot GatherMOS maintains stable performance across diverse conditions, while few-shot guidance yields large gains when support samples match the test conditions. Experiments on the VoiceBank-DEMAND dataset demonstrate that GatherMOS consistently outperforms DNSMOS, VQScore, naive score averaging, and even learning-based models such as CNN-BLSTM and MOS-SSL when trained under limited labeled-data conditions. These results highlight the potential of LLM-based aggregation as a practical strategy for non-intrusive speech quality evaluation.

Abstract

Speech Quality Assessment

Explore similar work

Few-Shot and Pseudo-Label Guided Speech Quality Evaluation with Large Language Models

Investigating Human-Model Discrepancies in Speech Quality Assessment via Acoustic and Prosodic Perturbations

DAMOS: Learning Distortion-Aware Speech Quality Assessment through Explicit Distortion Localization