When Normality Shifts: Multi-Level WavLM Modeling for Open-World Audio Deepfake Detection
Authors: Daniel Niekurzak, Daniel Kozuch, Weronika Folwarska, Khan Muhammad, Rafał Scherer
Organizations: Department of Artificial Intelligence, Czestochowa University of Technology, Poland · Department of Computer Science, Czestochowa University of Technology, Poland · School of Convergence, Sungkyunkwan University, Seoul 03063, Republic of Korea · Faculty of Computer Science, AGH University of Krakow, Poland
Audio deepfake detectors trained with known synthesis attacks can learn generator-specific shortcuts that fail as synthesis systems and acoustic conditions evolve. We investigate an attack-free alternative: a normality model trained exclusively on bona fide speech, which scores synthetic audio by its deviation from authentic representations. We propose MLN-WavLM, a method that applies shrinkage-Mahalanobis modeling to pooled hidden states from a frozen large-scale, pre-trained speech processing WavLM Base+ architecture. Evaluated on ASVspoof 2019 LA, our best attack-free variant achieves an 8.45% Equal Error Rate (EER), closely tracking a fully supervised WavLM-HM-Conformer baseline (7.86%). However, when subjected to the ASVspoof 2021 DF domain shift, performance drops to 22.12% EER, exposing vulnerabilities in the bona-fide-calibrated operating point. Layer-wise analysis reveals that early WavLM representations dominate the ensemble. To test real-world viability, we simulated deployment streams using unlabeled threshold adaptation. While robust median/MAD recalibration easily handles in-domain contamination up to 10%, recalibrating under the 2021 DF domain shift triggers a total collapse of low-FPR recall. We conclude that while frozen source-domain normality can effectively rank deepfakes, it does not yield a stable open-world decision rule, and downstream threshold adaptation cannot rescue upstream ranking degradation.
Figures & tables
Artifact
Shape / dtype
Size
Train bona fide cache
2580×4×768 f32
30.5 MB
Dev cache
24844×4×768 f32
294 MB
2019 LA eval cache
71237×4×768 f32
843 MB
2021 DF eval cache
611828×4×768 f32
7.07 GB
2021 DF score cache
611828×6×5 f32
103.5 MB
Frozen WavLM Base+
∼ 94.7 M params (not trained)
TABLE I: Frozen embedding caches used for all one-class scoring. Shapes are rows × layers × embedding dimension.
System
EER (%)
AUC
TPR@1
%FPR
Centroid, PCA-128
19.05
0.866
7.95
MLN-WavLM-PCA
12.40
0.938
43.69
Centroid, no PCA
16.52
0.895
12.86
MLN-WavLM-Full
8.45
0.966
69.17
k NN ( k=5 ), PCA-128
21.35
0.871
20.92
TABLE II: ASVspoof 2019 LA evaluation. All rows except the supervised reference fit bona fide only. Ensemble: equal mean over layers 3/6/9/12. MLN-WavLM-PCA and MLN-WavLM-Full are the pre-specified Mahalanobis variants.
Fig. 1: ROC curves on ASVspoof 2019 LA evaluation for selected bona-fide-only systems. Shrinkage Mahalanobis without PCA dominates the other one-class controls.
Fig. 2: ASVspoof 2019 LA: layer-wise and ensemble metrics for the Mahalanobis configurations. Early layers are strong; equal fusion helps most relative to later layers.
PCA-128
No PCA
Layer
2019
2021
2019
2021
3
14.52
25.48
9.22
22.10
6
19.01
29.14
11.70
23.68
9
17.46
28.32
12.77
25.43
12
17.50
27.07
14.11
25.88
Ensemble
12.40
25.06
8.45
22.12
TABLE III: Per-layer EER (%) for shrinkage Mahalanobis. Ensemble uses equal weights fixed before evaluation.
2019 LA
2021 DF
Score
EER
Δ EER
EER
Δ EER
Full ensemble
8.45
—
22.12
—
Drop layer 3
11.64
+3.19
24.19
+2.07
Drop layer 6
8.17
−0.28
22.20
+0.08
Drop layer 9
8.40
−0.05
21.84
−0.28
Drop layer 12
8.28
−0.16
21.52
−0.60
TABLE IV: Leave-one-layer-out for the MLN-WavLM-Full ensemble. Positive Δ EER means removing the layer hurts the ensemble.
System
EER (%)
AUC
TPR@5
%FPR (%)
Centroid, PCA-128
29.36
0.764
6.57
MLN-WavLM-PCA
25.06
0.807
8.58
Centroid, no PCA
26.80
0.792
7.63
MLN-WavLM-Full
22.12
0.845
15.20
KNN ( k=5 ), PCA-128
28.65
0.771
9.82
TABLE V: Frozen ASVspoof 2021 DF results. The supervised reference uses spoof labels during training; all other rows are bona-fide-only.
Fig. 3: ROC curves on frozen ASVspoof 2021 DF. Ranking remains informative, but the operating region near low FPR is weak.
System
2019EER
2021EER
Δ EER
Supervised WavLM-HM-Conformer
7.86
14.94
+7.08
MLN-WavLM-Full
8.45
22.12
+13.67
MLN-WavLM-PCA
12.40
25.06
+12.66
TABLE VI: Relative cross-domain degradation for the same supervised checkpoint and the two pre-specified Mahalanobis variants.
Threshold source
2019 LA
2021 DF
FPR
TPR
FPR
TPR
Bona fide q99 (attack-free)
0.61%
60.07%
20.23%
74.81%
Mixed-dev EER (diagnostic)
9.10%
91.92%
49.86%
98.44%
TABLE VII: Fixed-threshold transfer for the MLN-WavLM-Full ensemble. The mixed-dev EER threshold is a post-hoc diagnostic only.
Fig. 4: Score distributions on 2021 DF dataset. Overlap between bona fide and spoof scores explains why a source-calibrated threshold transfers poorly.
Fig. 5: 2019 LA dataset: unlabeled threshold adaptation versus stream contamination (MLN-WavLM-Full, mean ± std over 10 seeds). Robust MAD transfer keeps recall up to about 10% spoof prevalence; the naive stream quantile does not.
Spoof
Stream quantile
Robust transfer
prevalence
FPR
TPR
FPR
TPR
0%
1.18%
70.7%
0.89%
66.7%
1%
0.52%
53.0%
0.85%
65.9%
5%
0.13%
17.2%
0.72%
62.3%
10%
0.07%
9.2%
0.58%
57.1%
25%
0.05%
3.6%
0.17%
34.9%
TABLE VIII: Unlabeled threshold adaptation on 2019 LA streams (MLN-WavLM-Full, mean over 10 seeds). Frozen threshold: 0.65% FPR / 60.1% TPR. Oracle: 1.18% FPR / 70.7% TPR at every prevalence.
Fig. 6: 2021 DF dataset: threshold adaptation versus stream contamination (Mahalanobis MLN-WavLM-Full, mean ± std over 10 seeds; 96%∗ is the natural eval prevalence). Adaptive thresholds restore the nominal FPR (top) but cannot restore recall (bottom): at a 1% false-alarm budget the shifted-domain ranking supports only ≈ 1.5% TPR even with oracle calibration.
Audio deepfake detection systems trained on one dataset often fail when deployed on data from different sources due to distributional shifts in recording conditions, synthesis methods, and acoustic environments. We present a modular pipeline for unsupervised domain adaptation that combines pre-trained Wav2Vec 2.0 embeddings with statistical transformations to improve cross-domain generalization without requiring labeled target data. Our approach applies power transformation for feature normalization, ANOVA-based feature selection, joint PCA for domain-agnostic dimensionality reduction, and CORAL alignment to match source and target covariance structures before classification via logistic regression. We evaluate on two cross-domain transfer scenarios: ASVspoof 2019 LA to Fake-or-Real (FoR) and FoR to ASVspoof, achieving 62.7--63.6% accuracy with balanced performance across real and fake classes. Systematic ablation experiments reveal that feature selection (+3.5%) and CORAL alignment (+3.2%) provide the largest individual contributions, with the complete pipeline improving accuracy by 10.7% over baseline. While performance is modest compared to within-domain detection (94-96%), our pipeline offers transparency and modularity, making it suitable for deployment scenarios requiring interpretable decisions.
A speech deepfake detector estimates whether a recording is synthetic, but a forensic analyst also asks which system produced it, whether recordings share a source, and what evidence supports each conclusion. We present VeriSpoof, an audio language model that unifies five tasks with shared weights: deepfake detection, generator attribution (generator, acoustic model and vocoder), recording-condition analysis, synthesized-span localisation, and source verification. Each decision is supported by grounded rationales whose statements can be independently verified. Answers are scored by sentence likelihood, allowing detection and source verification to be thresholded without separate heads. Rationale statements can be checked against waveform measurements, span labels, or generator metadata. Counterfactual examples train decisions to follow the stated evidence. Across the 14 Speech-DF-Arena test sets, VeriSpoof-Plus achieves 2.14% pooled EER, ranking third of 28 systems and first among open-weight systems. VeriSpoof-Base, trained only on official training splits of eight corpora, achieves 4.43% EER, less than half that of the next open-weight detector. With the same weights, VeriSpoof-Plus verifies whether recordings from generators unseen in training share a source at 6.2% EER. Controlled studies show that the audio representation governs detection, the LLM governs recording comparison, and source supervision is necessary for neural-codec fakes. Statement-checkable rationales catch partial fakes missed by one-step answers (99.6% vs. 78.4% on PartialEdit) and determine decisions under edits unseen in training (99.5%). Training updates only 29.7M of 2.85B parameters, fitting a single 24 GB GPU with gradient checkpointing. We release both models, inference code, a demo, and VeriSpoof-Bench, the evaluation suite covering these questions.
Audio deepfakes generated by neural text-to-speech and voice-cloning systems threaten speaker verification and public discourse at scale. The core challenge is cross-dataset generalization: detectors trained on one synthesis pipeline collapse on unseen forgeries. We argue that this failure is primarily because of structural synthetic speech artifacts which are multi-timescale trajectory anomalies. Though every existing detector aggregates a fixed-window frame statistics, this misaligns the architecture with the signal. We propose FlowFake, a Liquid Time-Constant (LTC) architecture whose hidden state evolves via a learned ODE, with per-neuron adaptive time constants simultaneously resolving spectral (10ms) and prosodic (2s) cues. At only 34K parameters FlowFake achieves formal BIBO stability and O(dt^4) integration error. On a four-dataset cross domain benchmark (ASVspoof2019-LA, FakeOrReal, InTheWild, MLAAD), FlowFake reaches 75.29% on ASVspoof2019 trained only on FakeOrReal and 79.97% trained only on MLAAD. It outperforms RawGAT-ST and Whisper-DF on every evaluated pair and matching SSL Wav2vec2 (300x larger) at 0.01% of its parameter count. The source code is available on : https://github.com/GhostRider2023/FlowFake