Bottleneck Transformer-Based Approach for Improved Automatic STOI Score Prediction

Amartyaveer; Murali Kadambi; Chandra Mohan Sharma; Anupam Mondal; Prasanta Kumar Ghosh

eess.ASFeb 17, 2026

Bottleneck Transformer-Based Approach for Improved Automatic STOI Score Prediction

Authors: Amartyaveer, Murali Kadambi, Chandra Mohan Sharma, Anupam Mondal, Prasanta Kumar Ghosh

Organizations: Spire Lab, Dept of Electrical Engg Indian Institue of Science (IISc) Bengaluru, India · Center for Artificial Intelligence and Robotics (CAIR), DRDO India

Abstract

In this study, we have presented a novel approach to predict the Short-Time Objective Intelligibility (STOI) metric using a bottleneck transformer architecture. Traditional methods for calculating STOI typically requires clean reference speech, which limits their applicability in the real world. To address this, numerous deep learning-based nonintrusive speech assessment models have garnered significant interest. Many studies have achieved commendable performance, but there is room for further improvement. We propose the use of bottleneck transformer, incorporating convolution blocks for learning frame-level features and a multi-head self-attention (MHSA) layer to aggregate the information. These components enable the transformer to focus on the key aspects of the input data. Our model has shown higher correlation and lower mean squared error for both seen and unseen scenarios compared to the state-of-the-art model using self-supervised learning (SSL) and spectral features as inputs.

Figures & tables

Fig. 1: Proposed Architecture: Left- Architecture of the full model. Right- Architecture of the Bottleneck Transformer

Features	Proposed Model	STOI-Net
wav2vec 2.0	727,233	967,938
HuBERT	923,841	1,230,082
PS-I	334,785	N/A
PS-II	1,019,937	1,195,106
PS-III	669,921	845,538

TABLE I: Parameter Count of Models for different feature types

Model	Features	LCC ↑	SRCC ↑	MSE ↓
STOI-Net	wav2vec 2.0	$93.25\pm 0.99$	$94.74\pm 1.16$	$0.0075\pm 0.0011$
	HuBERT	$90.87\pm 1.42$	$93.71\pm 0.71$	$0.098\pm 0.0016$
	PS-II	$93.39\pm 0.52$	$94.54\pm 0.42$	$0.0071\pm 0.0005$
	PS-III	$78.78\pm 7.46$	$79.36\pm 7.27$	$0.0252\pm 0.0083$
Proposed	wav2vec 2.0	$94.38\pm 0.30$	$\mathbf{95.88\pm 0.29}$	$0.0061\pm 0.0003$
	HuBERT	$\mathbf{94.63\pm 0.52}$	$95.67\pm 0.40$	$\mathbf{0.0059\pm 0.0007}$

TABLE II: Baseline and proposed model performance on the seen test set

Fig. 2: Predicted STOI vs. Actual STOI for various SNR bins when using wav2vec 2.0 Features for seen test set using proposed model

Language

Librispeech English

RESPIN Bengali

RESPIN Bhojpuri

Bhashini Hindi

Features

LCC ↑

SRCC ↑

MSE ↓

LCC ↑

SRCC ↑

MSE ↓

LCC ↑

SRCC ↑

MSE ↓

LCC ↑

SRCC ↑

MSE ↓

wav2vec 2.0

87.24\pm 1.89

89.28\pm 1.79

0.016\pm 0.0027

80.07\pm 1.97

83.33\pm 2.17

0.021\pm 0.0020

79.14\pm 1.29

82.30\pm 1.29

0.023\pm 0.0018

80.49\pm 1.72

80.07\pm 2.31

0.018\pm 0.0013

HuBERT

87.33\pm 1.23

88.97\pm 1.11

0.018\pm 0.0025

80.56\pm 1.16

82.90\pm 1.28

0.022\pm 0.0021

79.50\pm 1.37

81.54\pm 1.65

0.024\pm 0.0020

84.01\pm 1.49

82.62\pm 1.78

0.016\pm 0.0020

PS-I

84.61\pm 2.72

85.42\pm 3.24

0.021\pm 0.0045

76.18\pm 3.42

77.64\pm 3.91

0.027\pm 0.0037

75.89\pm 3.16

77.14\pm 4.08

0.030\pm 0.0037

84.17\pm 1.10

81.74\pm 0.98

0.014\pm 0.0010

PS-II

84.72\pm 1.95

85.44\pm 2.45

0.021\pm 0.0039

80.06\pm 2.77

82.22\pm 3.60

0.021\pm 0.0021

78.59\pm 2.80

80.10\pm 3.82

0.025\pm 0.0025

84.98\pm 2.35

83.83\pm 2.12

0.014\pm 0.0018

PS-III

\mathbf{90.93\pm 1.24}

\mathbf{91.62\pm 1.60}

0.013\pm 0.0033

\mathbf{86.64\pm 2.49}

\mathbf{88.69\pm 2.99}

0.015\pm 0.0043

\mathbf{85.49\pm 3.17}

\mathbf{86.88\pm 3.93}

0.018\pm 0.0062

\mathbf{87.21\pm 0.97}

\mathbf{84.93\pm 0.88}

0.013\pm 0.0013

TABLE III: Performance of Proposed Bottleneck Transformer Model on unseen (English, Hindi, Bengali and Bhojpuri) test sets

Language

Librispeech English

RESPIN Bengali

RESPIN Bhojpuri

Bhashini Hindi

Features

LCC ↑

SRCC ↑

MSE ↓

LCC ↑

SRCC ↑

MSE ↓

LCC ↑

SRCC ↑

MSE ↓

LCC ↑

SRCC ↑

MSE ↓

wav2vec 2.0

86.22\pm 3.29

88.20\pm 3.58

0.020\pm 0.0079

79.66\pm 4.20

81.55\pm 4.47

0.022\pm 0.0066

77.66\pm 4.86

79.57\pm 4.80

0.026\pm 0.0071

83.26\pm 1.93

81.60\pm 1.96

0.015\pm 0.0016

HuBERT

86.54\pm 0.86

90.18\pm 0.81

0.017\pm 0.0015

81.62\pm 2.25

85.33\pm 2.12

0.019\pm 0.0022

78.19\pm 1.94

82.84\pm 1.19

0.022\pm 0.0013

78.19\pm 1.94

82.84\pm 1.19

0.022\pm 0.0013

PS-II

89.38\pm 0.68

90.34\pm 0.50

\mathbf{0.013\pm 0.0011}

85.60\pm 0.50

88.20\pm 0.50

\mathbf{0.015\pm 0.0005}

83.70\pm 0.80

85.60\pm 0.70

\mathbf{0.018\pm 0.0012}

86.50\pm 1.40

84.40\pm 1.10

\mathbf{0.013\pm 0.0012}

PS-III

68.73\pm 12.53

68.32\pm 12.11

0.034\pm 0.0171

77.80\pm 0.10

79.50\pm 0.00

0.024\pm 0.0032

61.90\pm 8.60

61.70\pm 10.70

0.036\pm 0.0080

80.30\pm 4.10

81.00\pm 2.30

0.024\pm 0.0049

TABLE IV: Performance of the baseline STOI-Net Model on unseen (English, Hindi, Bengali and Bhojpuri) test sets

Features	#Noise	LCC↑	SRCC↑	MSE↓
wav2vec 2.0	1	$97.27\pm 0.55$	$95.53\pm 0.60$	$0.0025\pm 0.0005$
	2	$94.83\pm 0.54$	$95.58\pm 0.66$	$0.0053\pm 0.0006$
	3	$86.47\pm 0.44$	$87.91\pm 0.49$	$0.0104\pm 0.0003$
HuBERT	1	$97.33\pm 0.37$	$93.95\pm 0.38$	$0.0027\pm 0.0005$
	2	$94.55\pm 0.42$	$94.86\pm 0.55$	$0.0056\pm 0.0005$
	3	$87.76\pm 1.66$	$89.35\pm 1.43$	$0.0094\pm 0.0013$

TABLE V: Impact on LCC, SRCC and MSE when different number of noises were added in seen test set using proposed model

SNR	Features	LCC↑	SRCC↑	MSE↓
<0	Wav2Vec2	$90.83\pm 1.75$	$90.63\pm 1.88$	$0.0090\pm 0.0002$
	HuBERT	$90.95\pm 1.07$	$90.87\pm 1.06$	$0.0090\pm 0.0008$
	PS-II	$83.83\pm 1.85$	$83.57\pm 1.78$	$0.0166\pm 0.0014$
	PS-III	$88.28\pm 0.41$	$88.39\pm 0.39$	$0.0116\pm 0.0004$
0-5	Wav2Vec2	$74.51\pm 4.46$	$81.01\pm 2.71$	$0.0044\pm 0.0012$
	HuBERT	$78.42\pm 3.65$	$76.79\pm 1.78$	$0.0043\pm 0.0009$

TABLE VI: Impact on LCC, SRCC, and MSE across different SNR levels in the seen test set using proposed model

Speech Quality Assessment

Speech quality assessment measures how well speech preserves perceptual quality, intelligibility, pronunciation, naturalness, and speaker characteristics.

See all 69 papers

LAST: Looped Audio Spectrogram Transformer
Oct 1, 2026Haider Al-Tahan, Sean O'Brien, Anastasia Razdaibiedina +1Audio Understanding Recurrent Transformers
Georgia Institute of Technology · University of California San Diego · Google DeepMind
STFT Frontend Mismatch in Pretrained Audio Spectrogram Transformers: Analysis and Adaptation
Oct 25, 2026Julia Wilkins, Wei-Ching Lin, Abinaya Kumar +2Audio Classification Transformer
New York University, USA · Bosch Research, USA
Benchmarking Speech-to-Speech Translation Models
Jun 2, 2026Alkis Koudounas, Hayato Futami, Quentin Jodelet +3Speech Translation Speech Quality Assessment
Sony Group Corporation, Japan · Carnegie Mellon University, USA

Oct 1, 2026J/K move · Enter open · S save

LAST: Looped Audio Spectrogram Transformer

Haider Al-Tahan, Sean O'Brien, Anastasia Razdaibiedina, N. Apurva Ratan Murty

Georgia Institute of Technology · University of California San Diego · Google DeepMind

Audio Understanding Recurrent Transformers Audio Classification

Open paper AskPDF arXiv

Increasing depth of transformer models improves recognition, but it comes at a substantial cost. Each additional layer requires more parameters, which makes the process computationally inefficient. We ask whether additional processing can focus on integrating features already computed. Looped Audio Spectrogram Transformer (LAST) first processes all tokens, then reuses the same blocks to refine only the class token over fixed audio features, thereby making later passes inexpensive. On AudioSet, ten-pass LAST achieves 0.345 mean average precision, exceeding a twelve-layer sequential transformer by 2.1% relative with 49.4% fewer parameters, 42% fewer multiply-accumulate operations, and 9.8% higher measured throughput. Across separately trained models, increasing the pass count from two to ten improves accuracy while adding only 1.2% computation. Further evaluations show improved robustness to temporal masking and various other auditory augmentations, with better generalization on classification tasks with music, environmental, and event sounds.

Abstract

Speech Quality Assessment

Explore similar work

LAST: Looped Audio Spectrogram Transformer

STFT Frontend Mismatch in Pretrained Audio Spectrogram Transformers: Analysis and Adaptation

Benchmarking Speech-to-Speech Translation Models