Speech enhancement (SE) models degrade under domain shifts and have to adapt to unseen target domains during deployment. Most existing test-time adaptation (TTA) methods for SE do so by adapting a subset of the model weights using a self-supervised loss, which requires backpropagation at test-time and permanently alters the model. We instead recalibrate the prediction itself and propose BLINC, a training-free TTA method that remaps the predicted time-frequency mask onto a bimodal target distribution by histogram matching. At test-time, the target distribution is parameterized from blind features of the noisy recording, so neither a reference distribution from a classical algorithm nor online metric optimization is involved. BLINC improves the overall quality of both evaluated SE models on almost every target condition and matches or exceeds the loss-based TTA baselines at minimal overhead.
Figures & tables
Figure 1: Loss-based TTA minimizes a loss L at test-time, adjusting either the model weights θ or the prediction X . BLINC instead recalibrates the prediction of the frozen model in a single step without a loss.
Figure 2
Figure 4: Signal-wise optimal midpoint b over the blind speech-activity fraction PSp .
Source
RemixIT
LaDen
MPol
RaR
BLINC
0.003
0.014
0.372
0.032
0.903
0.003
Table 1: RTF ↓ with the AM model on VBD. Factors ≤1 indicate real-time capability.
PESQ↑
CSIG↑
CBAK↑
COVL↑
SSNR↑
SI-SDR↑
AM
Source
2.05
3.07
2.78
2.52
7.42
12.28
RemixIT
2.06
3.10
2.80
2.54
7.48
12.47
LaDen
2.13
3.13
2.80
2.59
7.01
12.33
MPol
2.10
3.17
2.82
2.60
7.40
12.35
BLINC
2.17
3.21
2.78
2.66
6.19
11.50
CMGAN
Source
2.60
3.75
3.02
3.15
6.00
11.32
Table 2: Evaluation results averaged over the target datasets. SSNR and SI-SDR in dB.
Figure 5: Δ COVL per target domain relative to the source model.
Target
PESQ ↑
SI-SDR ↑
Source
2.42
11.49
Constant target
2.45
10.73
BLINC
2.55
11.24
BLINC, SI-SDR
2.29
12.21
BLINC, in-domain
2.65
11.02
Sigmoid oracle
2.75
11.02
Table 3: Ablation study with the AM model on VBD. The lower block uses information unavailable at test-time. SI-SDR in dB.
Speech enhancement improves intelligibility and listening quality by suppressing noise, reverberation, and competing speakers in recorded or live audio.
Test-time adaptation (TTA) offers a promising direction for improving speech enhancement models under mismatched acoustic conditions, without requiring access to labeled target data. In this work, we propose a single-utterance TTA method that regularizes a pretrained speech enhancement model using an autoregressive prior trained on clean speech latent representations extracted from a neural audio codec. Adaptation is performed by minimizing the Kullback-Leibler divergence between the enhanced speech distribution and the clean speech prior. Experiments across multiple noisy speech datasets show consistent improvements in speech quality, particularly under training-testing noise mismatch conditions. Code and audio examples are available online.
Sofiene Kammoun, Simon Leglaive, Xavier Alameda-Pineda +1
CentraleSup´elec, IETR (UMR CNRS 6164), France · Inria at Univ. Grenoble Alpes, CNRS, LJK, France · Signal Processing Group, University of Hamburg, Germany
Recent studies have shown that post-deployment adaptation can improve the robustness of speech enhancement models in unseen noise conditions. However, existing methods often incur prohibitive computational and memory costs, limiting their suitability for on-device deployment. In this work, we investigate model adaptation in realistic settings with dynamic acoustic scene changes and propose a lightweight framework that augments a frozen backbone with low-rank adapters updated via self-supervised training. Experiments on sequential scene evaluations spanning 111 environments across 37 noise types and three signal-to-noise ratio ranges, including the challenging [-8, 0] dB range, show that our method updates fewer than 1% of the base model's parameters while achieving an average 1.51 dB SI-SDR improvement within only 20 updates per scene. Compared to state-of-the-art approaches, our framework achieves competitive or superior perceptual quality with smoother and more stable convergence, demonstrating its practicality for lightweight on-device adaptation of speech enhancement models under real-world acoustic conditions.
Longbiao Cheng, Shih-Chii Liu
Institute of Neuroinformatics, University of Zurich and ETH Zurich
We propose a domain-incremental learning framework for generative speech enhancement (SE) that learns from a sequence of datasets or domains recorded under diverse acoustic conditions. Fine-tuning a pretrained model on continuously evolving domains leads to catastrophic forgetting of previously acquired knowledge, while zero-shot generalization often fails to adequately adapt to unseen domains. To address these challenges, we first develop a novel language model-based generative SE model that we then use as a pretrained backbone and incrementally adapt it to acoustically mismatched domains using lightweight domain-specific Low-Rank Adaptation. The proposed framework enables the model to acquire enhancement capabilities for new domains while preserving performance on previously learned domains. Evaluated on four heterogeneous speech datasets, our approach effectively adapts to new domains without forgetting previously learned domains.
Manjunath Mulimani, Annamaria Mesaros, Minje Kim +1
Aalborg University, Department of Electronic Systems, Denmark · Tampere University, Signal Processing Research Centre, Finland · University of Illinois Urbana-Champaign, Siebel School of Computing and Data Science, USA