cs.SDMar 9, 2026

Gender Fairness in Audio Deepfake Detection: Performance and Disparity Analysis

Authors: Aishwarya Fursule, Shruti Kshirsagar, Anderson R. Avila

Organizations: School of Computing, Wichita State University, Wichita, KS, USA · Institut national de la recherche scientifique (INRS–EMT), Montreal, QC, Canada · INRS-UQO Mixed Research Unit on Cybersecurity, Gatineau, Canada

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Audio deepfake detection aims to detect real human voices from those generated by Artificial Intelligence (AI) and has emerged as a significant problem in the field of voice biometrics systems. With the ever-improving quality of synthetic voice, the probability of such a voice being exploited for illicit practices like identity thest and impersonation increases. Although significant progress has been made in the field of Audio Deepfake Detection in recent times, the issue of gender bias remains underexplored and in its nascent stage In this paper, we have attempted a thorough analysis of gender dependent performance and fairness in audio deepfake detection models. We have used the ASVspoof 5 dataset and train a ResNet-18 classifier and evaluate detection performance across four different audio features, and compared the performance with baseline AASIST model. Beyond conventional metrics such as Equal Error Rate (EER %), we incorporated five established fairness metrics to quantify gender disparities in the model. Our results show that even when the overall EER difference between genders appears low, fairness-aware evaluation reveals disparities in error distribution that are obscured by aggregate performance measures. These findings demonstrate that reliance on standard metrics is unreliable, whereas fairness metrics provide critical insights into demographic-specific failure modes. This work highlights the importance of fairness-aware evaluation for developing a more equitable, robust, and trustworthy audio deepfake detection system.

Figures & tables

Fig. 1: The five group fairness metrics adopted in this study.
Fig. 2: Balanced gender distribution for the ASVspoof 5 dataset across train, development, and evaluation splits.
Fig. 3: Illustration of the procedure adopted to perform group fairness assessment. Model performance is based on 5 group fairness measures.
ModelFemaleMaleDiff (F-M)p-value (Holm)
AASIST0.1900.210-0.016<1×10−16<1\times 10^{-16}
CQT0.3800.2900.090<1×10−16<1\times 10^{-16}
LogSpec0.3240.334-0.009<1×10−16<1\times 10^{-16}
Wav2vec0.3490.3160.033<1×10−16<1\times 10^{-16}
WavLM0.1950.1820.011<1×10−16<1\times 10^{-16}
TABLE I: Fairness assessment based on the statistical parity metric.
ModelFemaleMaleDiff (F-M)p-value (Holm)
AASIST0.1720.197-0.007<1×10−16<1\times 10^{-16}
CQT0.4740.3600.114<1×10−16<1\times 10^{-16}
LogSpec0.4980.501-0.0030.2171
Wav2vec0.6600.6310.029<1×10−16<1\times 10^{-16}
WavLM0.5430.5110.032<1×10−16<1\times 10^{-16}
TABLE II: Fairness assessment based on the equal opportunity metric.
ModelFemaleMaleDiff (F-M)p-value (Holm)
AASIST0.2790.358-0.078<1×10−16<1\times 10^{-16}
CQT0.3570.2710.086<1×10−16<1\times 10^{-16}
LogSpec0.2820.289-0.007<1×10−16<1\times 10^{-16}
Wav2vec0.2730.2310.042<1×10−16<1\times 10^{-16}
WavLM0.1100.0960.014<1×10−16<1\times 10^{-16}
TABLE III: Fairness assessment based on the equality of odds metric.
ModelFemaleMaleDiff (F-M)p-value (Holm)
AASIST0.6790.687-0.0070.003
CQT0.2440.263-0.019<1×10−16<1\times 10^{-16}
LogSpec0.3010.318-0.017<1×10−16<1\times 10^{-16}
Wav2vec0.3710.424-0.053<1×10−16<1\times 10^{-16}
WavLM0.5450.5891-0.044<1×10−16<1\times 10^{-16}
TABLE IV: Fairness assessment based on the predictive parity metric.
ModelFemaleMaleDiff (F-M)p-value (Holm)
AASIST0.09770.0991-0.0014<1×10−16<1\times 10^{-16}
CQT2.79041.5741.216<1×10−16<1\times 10^{-16}
LogSpec0.7340.749-0.015<1.32×10−16<1.32\times 10^{-16}
Wav2vec3.3112.3310.979<1×10−16<1\times 10^{-16}
WavLM0.9930.7290.263<1×10−16<1\times 10^{-16}
TABLE V: Fairness assessment based on the treatment equality metric.

Explore similar work

CardsList
May 9, 2026

Towards Trustworthy Audio Deepfake Detection: A Systematic Framework for Diagnosing and Mitigating Gender Bias

Audio deepfake detection systems are increasingly deployed in high-stakes security applications, yet their fairness across demographic groups remains critically underexamined. Prior work measures gender disparity but does not investigate where it comes from or how to fix it systematically. We present the first diagnosis-first framework that identifies bias source before applying targeted mitigation, evaluated on two models, AASIST and Wav2Vec2+ResNet18, on ASVSpoof5. Our diagnosis shows that bias does not stem from imbalanced training data but from acoustic representation differences, gender leakage in learned features, and structural evaluation asymmetry. We test mitigation strategies across in-processing, post-processing and combined families, including novel methods introduced in this work. Adjusting the decision threshold separately per gender reduces unfairness by 54% to 75% at no cost to detection accuracy, and our new epoch-level fairness regularisation method outperforms existing per-batch approaches. Adversarial debiasing succeeds only when gender leakage is localised, and fails when it is diffuse, an outcome correctly predicted by our diagnosis before training. No single method fully closes the fairness gap, confirming that bias sources must be identified before fixes are applied and that fairer benchmark design is equally important

Aishwarya Fursule, Shruti Kshirsagar, Anderson R. Avila

School of Computing, Wichita State University, Wichita, KS, USA · Institut national de la recherche scientifique (INRS-EMT), Montreal, QC, Canada · INRS-UQO Mixed Research Unit on Cybersecurity, Gatineau, Canada

Audio Deepfake DetectionAlgorithmic FairnessAlgorithmic Bias
AskPDF
Jul 10, 2026

What You Train Is What You Get: Gender Bias, Training Composition, and Post-Hoc Mitigation in Audio Deepfake Detection

Audio deepfake detection models determine whether speech is genuine or artificially generated, but high overall accuracy can mask substantial performance disparities across demographic groups. In this work, we investigate gender bias in audio deepfake detection using the ASVspoof5 dataset. We use ASVspoof5 under a controlled custom split designed to isolate gender-composition effects. We train attack-specific models on nine training sets with different gender compositions, ranging from female-only to male-only. We use a ResNet18 classifier with LogSpectrogram and WavLM-Base+ features, and we evaluated six post-hoc threshold calibration methods. Experimental results show that training data composition strongly predicts bias direction, with the underrepresented gender performing worse at test time. WavLM-Base+ features are shown to produce gender performance gaps 3.0 to 4.3 times larger than LogSpectrogram under identical training conditions, and balanced training is found to reduce LogSpectrogram bias but leave WavLM bias largely intact. Moreover, all six calibration strategies, including Oracle calibration with full test-set label access, leave the Equal Error Rate gap unchanged at 1.317 pp, confirming that threshold adjustment cannot correct underlying score distribution disparities. Overall, these findings suggest that gender fairness in audio deepfake detection must be addressed at training time, as post-hoc methods can only partially mitigate the resulting disparities

Aishwarya R. Fursule, Vamshi Nallaguntla, Shruti Kshirsagar +1

School of Computing, Wichita State University, Wichita, KS, USA · Institut national de la recherche scientifique (INRS–EMT), Montreal, QC, Canada · INRS-UQO Mixed Research Unit on Cybersecurity, Gatineau, QC, Canada

Audio Deepfake DetectionASVspoof
AskPDF
Apr 9, 2026

DeepFense: A Unified, Modular, and Extensible Framework for Robust Deepfake Audio Detection

Speech deepfake detection is a well-established research field with different models, datasets, and training strategies. However, the lack of standardized implementations and evaluation protocols limits reproducibility, benchmarking, and comparison across studies. In this work, we present DeepFense, a comprehensive, open-source PyTorch toolkit integrating the latest architectures, loss functions, and augmentation pipelines, alongside over 100 recipes. Using DeepFense, we conducted a large-scale evaluation of more than 400 models. Our findings reveal that while carefully curated training data improves cross-domain generalization, the choice of pre-trained front-end feature extractor dominates overall performance variance. Crucially, we show severe biases in high-performing models regarding audio quality, speaker gender, and language. DeepFense is expected to facilitate real-world deployment with the necessary tools to address equitable training data selection and front-end fine-tuning.

Yassine El Kheir, Arnab Das, Yixuan Xiao +6

German Research Center for Artificial Intelligence (DFKI), Germany · University of Stuttgart, Germany · National Institute of Informatics, Japan +1

Audio Deepfake Detection
AskPDF