Investigating the Impact of Speech Enhancement on Audio Deepfake Detection in Noisy Environments
Authors: Anacin, Angela, Shruti Kshirsagar, Anderson R. Avila
Organizations: School of Computing, Wichita State University, Wichita, Kansas, USA · Institut national de la recherche scientifique (INRS–EMT), Montreal, Canada
Logical Access (LA) attacks, also known as audio deepfake attacks, use Text-to-Speech (TTS) or Voice Conversion (VC) methods to generate spoofed speech data. This can represent a serious threat to Automatic Speaker Verification (ASV) systems, as intruders can use such attacks to bypass voice biometric security. In this study, we investigate the correlation between speech quality and the performance of audio spoofing detection systems (i.e., LA task). For that, the performance of two enhancement algorithms is evaluated based on two perceptual speech quality measures, namely Perceptual Evaluation of Speech Quality (PESQ) and Speech-to-Reverberation Modulation Ratio (SRMR), and in respect to their impact on the audio spoofing detection system. We adopted the LA dataset, provided in the ASVspoof 2019 Challenge, and corrupted its test set with different Signal-to-Noise Ratio (SNR) levels, while leaving the training data untouched. Enhancement was applied to attenuate the detrimental effects of noisy speech, and the performances of two models, Speech Enhancement Generative Adversarial Network (SEGAN) and Metric-Optimized Generative Adversarial Network Plus (MetricGAN+), were compared. Although we expect that speech quality will correlate well with speech applications' performance, it can also have as a side effect on downstream tasks if unwanted artifacts are introduced or relevant information is removed from the speech signal. Our results corroborate with this hypothesis, as we found that the enhancement algorithm leading to the highest speech quality scores, MetricGAN+, provided the lowest Equal Error Rate (EER) on the audio spoofing detection task, whereas the enhancement method with the lowest speech quality scores, SEGAN, led to the lowest EER, thus leading to better performance on the LA task.
Figures & tables
Fig. 1: Training and Testing Setup; x(n) refers to the raw audio, h(n) refers to convolutive noise being added, Gs and Gt refer to Graph Spectral and Graph Temporal, these combine and pass into the Heterogeneous Stacking Graph Attention Layer (HS-GAL).
Fig. 2: SNR for real and spoof audio signal at 0dB
Fig. 3: Perceptual quality performance of two enhancement algorithms based on PESQ and SRMR.
0 dB
5 dB
10 dB
15 dB
20 dB
EER
t-DCF
EER
t-DCF
EER
t-DCF
EER
t-DCF
EER
t-DCF
Cafeteria
SEGAN
14.03
0.33
11.91
0.29
8.99
0.22
6.58
0.15
5.89
0.16
MetricGAN+
40.40
0.90
23.20
0.65
13.47
0.42
8.21
0.25
5.47
0.17
Noisy
42.58
0.99
36.83
0.94
12.24
0.35
7.92
0.24
2.80
0.08
Babble
SEGAN
15.21
0.40
10.41
0.26
7.56
0.20
6.16
0.17
5.70
0.16
MetricGAN+
42.80
0.94
24.17
0.67
11.23
0.34
6.29
0.34
3.60
0.09
TABLE I: EER (%) and t-DCF for SEGAN, MetricGAN+, and Noisy conditions across different noise levels
Fig. 4: Correlation between PESQ/SRMR scores and EER, t-DCF for SEGAN and MetricGAN+. (a–d) show correlations between PESQ vs. SEGAN/MetricGAN+ for EER and t-DCF, (e–h) show correlations between SRMR vs. SEGAN/MetricGAN+ for EER and t-DCF.
Audio anti-spoofing systems are typically trained to assign one authenticity label to an entire speech utterance. This formulation becomes under-specified for transformations where the underlying speaker identity and linguistic content remain unchanged. We study this problem using benign, authenticity-preserving speech transformations, including voice quality conversion and speech restoration, applied to both bona fide and spoofed speech. Instead of treating all processed audio as spoofed, we factorise labels into source authenticity and processed status. Across SSL representations and DF-Arena fine-tuning experiments, we find that utterance processing status can transfer more reliably than source attribution: detectors can often identify that speech has been processed, while still confusing processed bona fide and processed spoofed speech. These results suggest that audio deepfake defences must move beyond the binary spoofed/authentic paradigm. Robust detection requires granular reporting on source authenticity, processing status, and precise processing localisation.
Maliciously-created fake speech, including deepfaked and spoofed audio, is proliferating at an alarming rate, and detection models are racing to stay ahead of the curve. Yet, most detection models are trained to make inference on frame-level audio features alone without leveraging valuable linguistic cues at larger timescales. To address this gap, we present Linguistically Augmented Audio Speech Data (LinguAS), a dataset of genuine and deepfaked audio samples annotated with five strategically-chosen, Expert-Defined Linguistic Features (EDLFs) that occur frequently in spoken English and are characteristic of natural human speech. LinguAS contains over 800 audio samples, each of which are annotated with EDLFs. The dataset has a balanced number of four spoofed audio attack types and a proportionate number of genuine speech samples. We also include metadata on speaker gender and the generator/source for each spoofed audio sample, offering more granularity for model training. We found that models trained on data augmented with EDLFs had improved model performance significantly beyond the ASVspoof 2021 deep learning baselines and SSL models like HuBert and XLSR. LinguAS's augmented linguistic, gender, and generator metadata provide audio deepfake researchers with a dataset that emphasizes real human language traits to improve model inference of faked speech. Data and code are publicly available.
Ashley R. Keaton, Zahra Khanjani, Christine Mallinson +1
University of Maryland, Baltimore County Baltimore, MD, USA
Recent advances in text-to-speech and voice cloning make high-quality spoofing inexpensive and scalable, threatening voice authentication systems, especially automatic speaker verification (ASV). Existing defenses mainly address this threat through binary countermeasures (CMs) for deepfake detection or spoofing-aware speaker verification (SASV), where current systems are dominated by modular ASV-CM fusion and cascaded pipelines. Although large audio language models (LALMs) have shown promise on related audio tasks, including CM and ASV, their use for SASV remains unexplored, despite their capacity to produce natural-language rationales for auditing and robustness beyond discriminative predictions. This work systematically evaluates LALMs for SASV against conventional pipelines under zero-shot prompting, supervised adaptation, reasoning-oriented training, and reinforcement-learning-based optimization. Our results show that pretrained LALMs are near chance in the zero-shot setting, confirming that they are not natively suited to SASV, but that task-specific adaptation closes this gap. We further find that competitive SASV performance can be achieved through several distinct routes. These findings position LALMs as a promising and auditable foundation for unified SASV, while clarifying where conventional cascade systems still lead.