Designing a speech quality assessment (SQA) system for estimating mean-opinion-score (MOS) of multi-rate speech with varying sampling frequency (16-48 kHz) is a challenging task. The challenge arises due to the limited availability of a MOS-labeled training dataset comprising multi-rate speech samples. While self-supervised learning (SSL) models have been widely adopted in SQA to boost performance, a key limitation is that they are pretrained on 16 kHz speech and therefore discard high-frequency information present in higher sampling rates. To address this issue, we propose a spectrogram-augmented SSL method that incorporates high-frequency features (up to 48 kHz sampling rate) through a parallel-branch architecture. We further introduce a two-step training scheme: the model is first pre-trained on a large 48 kHz dataset and then fine-tuned on a smaller multi-rate dataset. Experimental results show that leveraging high-frequency information overlooked by SSL features is crucial for accurate multi-rate SQA, and that the proposed two-step training substantially improves generalization when multi-rate data is limited.
Figures & tables
Figure 1: Architecture comparison between the existing SSL-Layer-MOS (left) and the proposed SA-SSL-MOS (right).
Figure 2: Detailed layer-wise architecture of SA-SSL-MOS.
Dataset
Purpose
Sampling Rate
Language
# Samples
Ratings/Clip
AudioMOS_train [ 10 ]
train, val
16/24/48kHz
English
320/80
10
AudioMOS_test [ 10 ]
test
16/24/48kHz
English
400
10
NISQA_TRAIN_(SIM+LIVE) [ 14 ]
train
48kHz
English
10000+1020
∼ 5
NISQA_VAL_(SIM+LIVE) [ 14 ]
val
48kHz
English
2500+200
∼ 5
NISQA_TEST_LIVETALK [ 14 ]
test
48kHz
German
232
24
NISQA_TEST_FOR [ 14 ]
test
48kHz
Australian English
240
∼ 30
Table 1: Overview of datasets.
test data
train data
model
MSE ↓
LCC ↑
SRCC ↑
Tencent w/o R
AudioMOS_train
baseline
1.002 ± 0.054
0.691 ± 0.023
0.687 ± 0.024
SA-SSL-MOS
1.097 ± 0.057
0.669 ± 0.035
0.666 ± 0.033
NISQA+AudioMOS_train
baseline
0.751 ± 0.043
0.917 ± 0.009
0.901 ± 0.006
SA-SSL-MOS
1.192 ± 0.124
0.877 ± 0.024
0.891 ± 0.010
Tencent w R
AudioMOS_train
baseline
0.577 ± 0.047
0.638 ± 0.029
0.542 ± 0.036
SA-SSL-MOS
0.712 ± 0.081
0.630 ± 0.054
0.544 ± 0.056
Table 3: Results for generalization ability test. Metrics for utterance level only, best performance marked bold .
Self-supervised learning (SSL) has advanced speech processing. However, existing speech SSL methods typically assume a single sampling rate and struggle with mixed-rate data due to temporal resolution mismatch. To address this limitation, we propose MSRHuBERT, a multi-sampling-rate adaptive pre-training method. Building on HuBERT, we replace its single-rate downsampling CNN with a multi-sampling-rate adaptive downsampling CNN that maps raw waveforms from different sampling rates to a shared temporal resolution without resampling. This design enables unified mixed-rate pre-training and fine-tuning. In experiments spanning 16 to 48 kHz, MSRHuBERT outperforms HuBERT on speech recognition and full-band speech reconstruction, preserving high-frequency detail while modeling low-frequency semantic structure. Moreover, MSRHuBERT retains HuBERT's mask-prediction objective and Transformer encoder, so existing analyses and improvements that were developed for HuBERT can apply directly.
Zikang Huang, Meng Ge, Tianrui Wang +4
Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin University, China · Huiyan Technology Company, Ltd., China · Chinese Academy of Sciences, China
Speech Quality Assessment (SQA) is essential for modern speech technologies, and recent non-intrusive SQA predictors increasingly rely on Speech Foundation Models (SFMs). However, because SFMs expose representations from many layers, it remains unclear which depths are most informative for MOS prediction and how multi-layer information should be combined reliably across backbones and datasets. We benchmark ten SFMs on four MOS datasets under three regimes: full fine-tuning, last-layer probing with a frozen encoder, and naive cross-layer weighted aggregation. We find that the best layer is strongly backbone- and dataset-dependent, and that naive weighted fusion can be unstable across settings. We further evaluate a layer-calibrated aggregation variant that applies per-layer adapters before pooling, which improves the robustness of multi-layer fusion and narrows the gap to full fine-tuning while keeping the backbone frozen.
Alef Iury Siqueira Ferreira, Pedro Lustosa Rege Botelho, Fernanda Silva +5
AKCIT, Brazil · Federal University of Rio Grande do Norte (UFRN), Brazil · Federal University of Technology (UTFPR), Brazil +1
Mean opinion scores (MOS) are widely used for speech quality assessment, yet scalar labels are sensitive to rater variability and listening test differences. This introduces labeling noise, which limits the reliability of MOS prediction. Preference prediction reduces this variability as listeners compare signals directly, producing cleaner labels. We study MOS-free preference prediction and propose PrefSQA, which incorporates uncertainty-aware logits, an impairment attention head, and a module based on non-matching-reference comparisons. We use and refine five datasets, including MOS-derived and low-noise simulated sets with matching and non-matching content, experiment with human preference sets, and test on unseen data. Experiments show small improvements on MOS-derived data, while other sets reveal clear improvement over the baselines, highlighting the value of high-quality preference data and demonstrating the effectiveness of the proposed method.
Junyi Fan, Donald S. Williamson
Department of Computer Science and Engineering, The Ohio State University, USA