SA-SSL-MOS: Self-supervised Learning MOS Prediction with Spectral Augmentation for Generalized Multi-Rate Speech Assessment

Fengyuan Cao; Xinyu Liang; Fredrik Cumlin; Victor Ungureanu; Chandan K. A. Reddy; Christian Schuldt; Saikat Chatterjee

eess.ASFeb 16, 2026

SA-SSL-MOS: Self-supervised Learning MOS Prediction with Spectral Augmentation for Generalized Multi-Rate Speech Assessment

Authors: Fengyuan Cao, Xinyu Liang, Fredrik Cumlin, Victor Ungureanu, Chandan K. A. Reddy, Christian Schuldt, Saikat Chatterjee

Organizations: KTH Royal Institute of Technology, Stockholm, Sweden · Google LLC

Abstract

Designing a speech quality assessment (SQA) system for estimating mean-opinion-score (MOS) of multi-rate speech with varying sampling frequency (16-48 kHz) is a challenging task. The challenge arises due to the limited availability of a MOS-labeled training dataset comprising multi-rate speech samples. While self-supervised learning (SSL) models have been widely adopted in SQA to boost performance, a key limitation is that they are pretrained on 16 kHz speech and therefore discard high-frequency information present in higher sampling rates. To address this issue, we propose a spectrogram-augmented SSL method that incorporates high-frequency features (up to 48 kHz sampling rate) through a parallel-branch architecture. We further introduce a two-step training scheme: the model is first pre-trained on a large 48 kHz dataset and then fine-tuned on a smaller multi-rate dataset. Experimental results show that leveraging high-frequency information overlooked by SSL features is crucial for accurate multi-rate SQA, and that the proposed two-step training substantially improves generalization when multi-rate data is limited.

Figures & tables

Figure 1: Architecture comparison between the existing SSL-Layer-MOS (left) and the proposed SA-SSL-MOS (right).

Figure 2: Detailed layer-wise architecture of SA-SSL-MOS.

Dataset	Purpose	Sampling Rate	Language	# Samples	Ratings/Clip
AudioMOS_train [ 10 ]	train, val	16/24/48kHz	English	320/80	10
AudioMOS_test [ 10 ]	test	16/24/48kHz	English	400	10
NISQA_TRAIN_(SIM+LIVE) [ 14 ]	train	48kHz	English	10000+1020	$\sim$ 5
NISQA_VAL_(SIM+LIVE) [ 14 ]	val	48kHz	English	2500+200	$\sim$ 5
NISQA_TEST_LIVETALK [ 14 ]	test	48kHz	German	232	24
NISQA_TEST_FOR [ 14 ]	test	48kHz	Australian English	240	$\sim$ 30

Table 1: Overview of datasets.

test data	train data	model	MSE $\downarrow$	LCC $\uparrow$	SRCC $\uparrow$
Tencent w/o R	AudioMOS_train	baseline	1.002 $\pm$ 0.054	0.691 $\pm$ 0.023	0.687 $\pm$ 0.024
	AudioMOS_train	SA-SSL-MOS	1.097 $\pm$ 0.057	0.669 $\pm$ 0.035	0.666 $\pm$ 0.033
	NISQA+AudioMOS_train	baseline	0.751 $\pm$ 0.043	0.917 $\pm$ 0.009	0.901 $\pm$ 0.006
	NISQA+AudioMOS_train	SA-SSL-MOS	1.192 $\pm$ 0.124	0.877 $\pm$ 0.024	0.891 $\pm$ 0.010
Tencent w R	AudioMOS_train	baseline	0.577 $\pm$ 0.047	0.638 $\pm$ 0.029	0.542 $\pm$ 0.036
	AudioMOS_train	SA-SSL-MOS	0.712 $\pm$ 0.081	0.630 $\pm$ 0.054	0.544 $\pm$ 0.056

Table 3: Results for generalization ability test. Metrics for utterance level only, best performance marked bold .

Speech Quality Assessment

Speech quality assessment measures how well speech preserves perceptual quality, intelligibility, pronunciation, naturalness, and speaker characteristics.

See all 69 papers

MSR-HuBERT: Self-supervised Pre-training for Adaptation to Multiple Sampling Rates
Mar 24, 2026Zikang Huang, Meng Ge, Tianrui Wang +4Self-Supervised Speech Representation Learning Automatic Speech Recognition
Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin University, China · Huiyan Technology Company, Ltd., China · Chinese Academy of Sciences, China
CAL-MOS: Bridging Layers with Adapters for Robust MOS Prediction Across Speech Foundation Models
Sep 14, 2026Alef Iury Siqueira Ferreira, Pedro Lustosa Rege Botelho, Fernanda Silva +5Speech Foundation Models Speech Quality Assessment
AKCIT, Brazil · Federal University of Rio Grande do Norte (UFRN), Brazil · Federal University of Technology (UTFPR), Brazil +1
PrefSQA: Pairwise Preference Prediction for Speech Quality Assessment and the Critical Role of High Quality Datasets
Jun 17, 2026Junyi Fan, Donald S. WilliamsonPairwise Preference Learning Pairwise Preference Evaluation
Department of Computer Science and Engineering, The Ohio State University, USA

Mar 24, 2026J/K move · Enter open · S save

MSR-HuBERT: Self-supervised Pre-training for Adaptation to Multiple Sampling Rates

Zikang Huang, Meng Ge, Tianrui Wang, Xuanchen Li+3

Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin University, China · Huiyan Technology Company, Ltd., China · Chinese Academy of Sciences, China

Self-Supervised Speech Representation Learning Automatic Speech Recognition

Open paper AskPDF arXiv

Self-supervised learning (SSL) has advanced speech processing. However, existing speech SSL methods typically assume a single sampling rate and struggle with mixed-rate data due to temporal resolution mismatch. To address this limitation, we propose MSRHuBERT, a multi-sampling-rate adaptive pre-training method. Building on HuBERT, we replace its single-rate downsampling CNN with a multi-sampling-rate adaptive downsampling CNN that maps raw waveforms from different sampling rates to a shared temporal resolution without resampling. This design enables unified mixed-rate pre-training and fine-tuning. In experiments spanning 16 to 48 kHz, MSRHuBERT outperforms HuBERT on speech recognition and full-band speech reconstruction, preserving high-frequency detail while modeling low-frequency semantic structure. Moreover, MSRHuBERT retains HuBERT's mask-prediction objective and Transformer encoder, so existing analyses and improvements that were developed for HuBERT can apply directly.

Abstract

Speech Quality Assessment

Explore similar work

MSR-HuBERT: Self-supervised Pre-training for Adaptation to Multiple Sampling Rates

CAL-MOS: Bridging Layers with Adapters for Robust MOS Prediction Across Speech Foundation Models

PrefSQA: Pairwise Preference Prediction for Speech Quality Assessment and the Critical Role of High Quality Datasets