Speech Bandwidth Extension (BWE) reconstructs missing high frequencies from a band-limited signal. Please note that perceptually important information is often concentrated in short high-frequency (HF) events. Therefore, an effective BWE requires both accurate HF generation and explicit control over where, how often, and in what spectral form HF energy should be produced. To address the above HF supervision, we present HFSWO, a compact complex-spectral Swin-based dual-stream adversarial framework with two newly proposed specialized discriminators: 1. High-Frequency Safeguard Discriminator (HFSD) converts excessive framewise HF-energy ratios into a differentiable safeguard-constraint penalty optimized through a projected primal--dual augmented-Lagrangian update; 2. Weighted Objective Scoring Discriminator (WOSD) learns an interpretable convex objective scoring over spectral flatness, centroid, and rolloff. Our rigorous experiments on VCTK English and MLS French datasets across multiple frequency ranges, including noisy conditions, show significantly improved perceptual scores with 74.6% fewer parameters than the existing baseline AP-BWE, establishing a new baseline in the BWE task in multilingual and multispeaker setups.
Figures & tables
Figure 1: Architecture of (a,b) Swin-1D stacks, CSF, and newly proposed discriminators - (c) HFSD and (d) WOSD.
Method
Size
Data
NISQA-MOS ↑
STOI ↑
PESQ ↑
SI-SNR ↑
LSD ↓
WER % ↓
4–16
8–16
16–48
4–16
8–16
16–48
4–16
8–16
16–48
4–16
8–16
16–48
4–16
8–16
16–48
4–16
8–16
16–48
Unprocessed
-
VCTK
2.79
3.67
4.43
0.55
0.61
0.61
1.15
1.51
1.41
-10.53
-7.62
-5.63
3.27
2.27
2.85
90.0
6.1
2.1
MLS
2.29
3.12
-
0.51
0.57
-
1.05
1.34
-
-13.67
-17.71
-
3.48
2.48
-
91.1
6.8
-
EBEN
29.7M
VCTK
2.59
2.69
2.53
0.89
0.98
0.98
2.64
3.69
3.71
11.94
19.94
20.83
1.03
0.78
0.92
19.1
12.4
8.5
MLS
2.14
2.28
-
0.87
0.96
-
2.47
3.55
-
11.74
18.38
-
1.17
0.88
-
19.4
12.7
-
AERO
36.4M
VCTK
2.79
2.75
2.88
0.83
0.94
0.99
2.62
3.65
3.69
13.60
20.70
21.56
1.09
0.97
0.75
20.3
13.4
9.8
Table 1: Clean-condition comparison on English VCTK and French MLS datasets; MLS does not include 48 kHz speech.
Figure 2: Proof of recovered HF components for 4–16 kHz.
Band
EREi↓
DKLU,i↓
DP56,i↓
kHz
AP-BWE
HFSWO
AP-BWE
HFSWO
AP-BWE
HFSWO
0–4
0.002688
0.002536
0.058
0.035
0.002722
0.002557
4–8
0.001415
0.000623
0.277
0.199
0.001387
0.000578
8–12
0.002154
0.002077
2.232
2.246
0.001994
0.001956
Table 2: Sub-band analysis using ITU-derived evaluation metrics as evidence of HF reconstruction improvement.
Model
SNR
NISQA-MOS ↑
SI-SNR ↑
4–16
8–16
16–48
4–16
8–16
16–48
AP-BWE
-10
2.02
2.55
2.98
-0.60
-0.04
1.17
-5
2.52
2.86
3.54
2.86
3.48
4.44
0
2.83
3.21
3.80
5.06
5.83
6.80
5
3.05
3.42
4.07
6.53
7.42
8.64
10
3.22
3.70
4.22
7.62
9.02
9.77
Table 3: Expanded SNR-wise results on noisy VCTK dataset.
Model
Freq.
Param.
MAC(M)
FLOP(M)
RTF(M)
Latency
AP-BWE
4–16
72.07 M
14236.65
28473.31
0.0023 ×
14.93ms
HFSWO
4–16
18.30 M
5177.89
10355.77
0.0027 ×
15.54ms
HFSD
4–16
7.6 k
0.141
0.282
x
0.823
WOSD
4–16
7.4 k
0.139
0.278
x
0.889
Table 4: Model- and discriminator-wise complexity on GPU. Parameters, MAC, and FLOP are reported in millions (M).
Agg.
Method
N
MOS ↑
SD
SEM
95% CI
Utt.
AP-BWE
1000
4.138
0.593
0.019
[4.101, 4.175]
Utt.
HFSWO
1000
4.206
0.648
0.021
[4.166, 4.247]
Spk.
AP-BWE
7
4.126
0.281
0.106
[3.867, 4.386]
Spk.
HFSWO
7
4.189
0.399
0.151
[3.821, 4.558]
Table 5: MOS descriptive statistics for the 1,000-utterance subset. Utt. = utterance-level; Spk. = speaker-level.
Agg.
Mean Δ
Med. Δ
W+
W−
p
Utt.
0.069
0.082
300,039
200,461
5.04×10−8
Spk.
0.063
0.094
19
9
0.469
Table 6: Paired Wilcoxon signed-rank test for NISQA-MOS differences. Mean Δ /Med. Δ = mean/median paired difference; W+ / W− = positive/negative rank sums.
MRAD
MRPD
HFSD
WOSD
LSD ↓
PESQ ↑
SI-SNR ↑
NISQA ↑
✓
✗
✓
✗
0.97
2.35
12.9
4.28
✓
✓
✓
✗
1.01
2.27
12.99
4.29
✓
✓
✗
✓
0.99
2.3
12.65
4.28
✓
✓
✓
✓
0.95
2.56
13.37
4.32
✓
✓
✓
✓
0.99
2.29
12.69
4.26
✓
✓
✓
✓
0.95
2.56
13.37
4.32
Table 7: Ablation study on discriminators for 4 → 16 kHz.
Speech enhancement improves intelligibility and listening quality by suppressing noise, reverberation, and competing speakers in recorded or live audio.
School of Computing, National University of Singapore · Department of Statistics & Data Science, National University of Singapore · College of Computing and Data Science, Nanyang Technological University