A General Model for Deepfake Speech Detection: Diverse Bonafide Resources or Diverse AI-Based Generators

Lam Pham; Khoi Vu; Dat Tran; David Fischinger; Alexander Schindler; Martin Boyer; Ian McLoughlin

cs.SDMar 29, 2026

A General Model for Deepfake Speech Detection: Diverse Bonafide Resources or Diverse AI-Based Generators

Authors: Lam Pham, Khoi Vu, Dat Tran, David Fischinger, Alexander Schindler, Martin Boyer, Ian McLoughlin

Organizations: Austrian Institute of Technology, Austria. · FPT University, Vietnam. · Singapore Institute of Technology, Singapore.

Abstract

In this paper, we analyze two main factors of Bonafide Resource (BR) or AI-based Generator (AG) which affect the performance and the generality of a Deepfake Speech Detection (DSD) model. To this end, we first propose a deep-learning based model, referred to as the baseline. Then, we conducted experiments on the baseline by which we indicate how Bonafide Resource (BR) and AI-based Generator (AG) factors affect the threshold score used to detect fake or bonafide input audio in the inference process. Given the experimental results, a dataset, which re-uses public Deepfake Speech Detection (DSD) datasets and shows a balance between Bonafide Resource (BR) or AI-based Generator (AG), is proposed. We then train various deep-learning based models on the proposed dataset and conduct cross-dataset evaluation on different benchmark datasets. The cross-dataset evaluation results prove that the balance of Bonafide Resources (BR) and AI-based Generators (AG) is the key factor to train and achieve a general Deepfake Speech Detection (DSD) model.

Figures & tables

Fig. 1: The proposed baseline architecture

Datasets	Years	Speakers	Utt. No.	Fake speech	Real Speech
		(Male/Female)	(Bonafide/Fake)	Generators	Resources
ASVspoof 2015 [ 7 ] (audio)	2015	45/61	16,651/246,500	10 (7 VC, 3 TTS)	Speaker Volunteers
FoR [ 8 ] (audio)	2019	140	-/195541	7 TTS	Kaggle [ 9 ]
ASVspoof 2019 (LA task) [ 10 ] (audio)	2019	46/61	12,483/108,978	19 (8VC, 11 TTS)	Speaker Volunteers
DFDC [ 11 ] (video)	2020	3426	128,154/104,500	1 TTS	Speaker Volunteers
ASVspoof 2021 (LA task) [ 12 ] (audio)	2021	21/27	18,452/163,114	13 TTS/VC	Speaker Volunteers

TABLE I: Public and Benchmark Datasets Proposed for Deepfake Speech Detection (DSD) in English

Fig. 2: The statistics of DSD datasets in English

Datasets	Bonafide Utterances	Fake Utterances
AG dataset	53,552	861,304
BR dataset	50,131	273,176
BR-AG dataset	103,683	1,134,408

TABLE II: Proposed datasets to balance AI-Based Generator and Bonafide Resource factors

Fig. 3: Distribution of fake probabilities from In-The-Wild dataset using AG dataset to train the baseline model

Fig. 4: Distribution of Logarit scores from In-The-Wild dataset using AG dataset to train the baseline model

Fig. 5: Distribution of fake probabilities from In-The-Wild dataset using BR dataset to train the baseline model

Fig. 6: Distribution of Logarit scores from In-The-Wild dataset using BR dataset to train the baseline model

Fig. 7: Confusion matrix (%) for In-The-Wild dataset using the AG dataset to train the baseline model (The boundary probability of 0.5)

Fig. 8: Confusion matrix (%) for In-The-Wild dataset using the BR dataset to train the baseline model (The boundary probability of 0.5)

Fig. 9: Our proposed model with three-stage training strategy for DSD task

Model	Train Data	Test Data	Acc.	F1	AuC
WavLM-Large frozen + MLP (baseline)	AG dataset	In-The-Wild	0.71	0.72	0.74
WavLM-Large frozen + MLP (baseline)	BR dataset	In-The-Wild	0.75	0.58	0.69
WavLM-Large frozen + MLP (baseline)	BR-AG dataset	In-The-Wild	0.80	0.72	0.82
WavLM-Large finetune + MLP	BR-AG dataset	In-The-Wild	0.87	0.79	0.83
Whisper-base finetune + MLP	BR-AG dataset	In-The-Wild	0.73	0.60	0.65
Wav2Vec2-XLSR-53 finetune + MLP	BR-AG dataset	In-The-Wild	0.79	0.64	0.73

TABLE III: Evaluation of the BR and AG factors on In-The-Wild dataset (Using Probability Boundary of 0.5)

Test Dataset	Accuracy	F1 Score	AuC
In-The-Wild	0.87	0.79	0.83
ASVspoof5 Eva Subset	0.91	0.91	0.92
FakeAVCeleb	0.99	0.99	0.99

TABLE IV: Cross-Dataset Evaluation of the best model (WavLM-Large finetune + MLP) using the boundary probability of 0.5

Fig. 10: Distribution of fake probabilities from In-The-Wild dataset using BR-AG dataset to train the WavLM-Large-Finetune+MLP model

Fig. 11: Distribution of Logarithmic scores from In-The-Wild dataset using BR-AG dataset to train the WavLM-Large-Finetune+MLP model

Explore similar work

DeepFense: A Unified, Modular, and Extensible Framework for Robust Deepfake Audio Detection
Apr 9, 2026Yassine El Kheir, Arnab Das, Yixuan Xiao +6Audio Deepfake Detection
German Research Center for Artificial Intelligence (DFKI), Germany · University of Stuttgart, Germany · National Institute of Informatics, Japan +1
Towards Robust Speech Deepfake Detection via Human-Inspired Reasoning
Mar 11, 2026Artem Dvirniak, Evgeny Kushnir, Dmitrii Tarasov +5Audio Deepfake Detection Explainable Deepfake Detection
MIRAI · AXXX · HSE +5
Ethical and Technical Limits of Deepfake Speech Datasets
Jun 9, 2026Vojtěch Staněk, Eva Trnovská, Kamil Malinka +1Data Leakage Audio Deepfake Detection

Apr 9, 2026J/K move · Enter open · S save

DeepFense: A Unified, Modular, and Extensible Framework for Robust Deepfake Audio Detection

Yassine El Kheir, Arnab Das, Yixuan Xiao, Xin Wang+5

German Research Center for Artificial Intelligence (DFKI), Germany · University of Stuttgart, Germany · National Institute of Informatics, Japan+1

Audio Deepfake Detection

Open paper AskPDF arXiv

Speech deepfake detection is a well-established research field with different models, datasets, and training strategies. However, the lack of standardized implementations and evaluation protocols limits reproducibility, benchmarking, and comparison across studies. In this work, we present DeepFense, a comprehensive, open-source PyTorch toolkit integrating the latest architectures, loss functions, and augmentation pipelines, alongside over 100 recipes. Using DeepFense, we conducted a large-scale evaluation of more than 400 models. Our findings reveal that while carefully curated training data improves cross-domain generalization, the choice of pre-trained front-end feature extractor dominates overall performance variance. Crucially, we show severe biases in high-performing models regarding audio quality, speaker gender, and language. DeepFense is expected to facilitate real-world deployment with the necessary tools to address equitable training data selection and front-end fine-tuning.

Abstract

Explore similar work

DeepFense: A Unified, Modular, and Extensible Framework for Robust Deepfake Audio Detection

Towards Robust Speech Deepfake Detection via Human-Inspired Reasoning

Ethical and Technical Limits of Deepfake Speech Datasets