When Normality Shifts: Multi-Level WavLM Modeling for Open-World Audio Deepfake Detection
Authors: Daniel Niekurzak, Daniel Kozuch, Weronika Folwarska, Khan Muhammad, Rafał Scherer
Organizations: Department of Artificial Intelligence, Czestochowa University of Technology, Poland · Department of Computer Science, Czestochowa University of Technology, Poland · School of Convergence, Sungkyunkwan University, Seoul 03063, Republic of Korea · Faculty of Computer Science, AGH University of Krakow, Poland
Audio deepfake detectors trained with known synthesis attacks can learn generator-specific shortcuts that fail as synthesis systems and acoustic conditions evolve. We investigate an attack-free alternative: a normality model trained exclusively on bona fide speech, which scores synthetic audio by its deviation from authentic representations. We propose MLN-WavLM, a method that applies shrinkage-Mahalanobis modeling to pooled hidden states from a frozen large-scale, pre-trained speech processing WavLM Base+ architecture. Evaluated on ASVspoof 2019 LA, our best attack-free variant achieves an 8.45% Equal Error Rate (EER), closely tracking a fully supervised WavLM-HM-Conformer baseline (7.86%). However, when subjected to the ASVspoof 2021 DF domain shift, performance drops to 22.12% EER, exposing vulnerabilities in the bona-fide-calibrated operating point. Layer-wise analysis reveals that early WavLM representations dominate the ensemble. To test real-world viability, we simulated deployment streams using unlabeled threshold adaptation. While robust median/MAD recalibration easily handles in-domain contamination up to 10%, recalibrating under the 2021 DF domain shift triggers a total collapse of low-FPR recall. We conclude that while frozen source-domain normality can effectively rank deepfakes, it does not yield a stable open-world decision rule, and downstream threshold adaptation cannot rescue upstream ranking degradation.
Figures & tables
Artifact
Shape / dtype
Size
Train bona fide cache
2580×4×768 f32
30.5 MB
Dev cache
24844×4×768 f32
294 MB
2019 LA eval cache
71237×4×768 f32
843 MB
2021 DF eval cache
611828×4×768 f32
7.07 GB
2021 DF score cache
611828×6×5 f32
103.5 MB
Frozen WavLM Base+
∼ 94.7 M params (not trained)
TABLE I: Frozen embedding caches used for all one-class scoring. Shapes are rows × layers × embedding dimension.
System
EER (%)
AUC
TPR@1
%FPR
Centroid, PCA-128
19.05
0.866
7.95
MLN-WavLM-PCA
12.40
0.938
43.69
Centroid, no PCA
16.52
0.895
12.86
MLN-WavLM-Full
8.45
0.966
69.17
k NN ( k=5 ), PCA-128
21.35
0.871
20.92
TABLE II: ASVspoof 2019 LA evaluation. All rows except the supervised reference fit bona fide only. Ensemble: equal mean over layers 3/6/9/12. MLN-WavLM-PCA and MLN-WavLM-Full are the pre-specified Mahalanobis variants.
Fig. 1: ROC curves on ASVspoof 2019 LA evaluation for selected bona-fide-only systems. Shrinkage Mahalanobis without PCA dominates the other one-class controls.
Fig. 2: ASVspoof 2019 LA: layer-wise and ensemble metrics for the Mahalanobis configurations. Early layers are strong; equal fusion helps most relative to later layers.
PCA-128
No PCA
Layer
2019
2021
2019
2021
3
14.52
25.48
9.22
22.10
6
19.01
29.14
11.70
23.68
9
17.46
28.32
12.77
25.43
12
17.50
27.07
14.11
25.88
Ensemble
12.40
25.06
8.45
22.12
TABLE III: Per-layer EER (%) for shrinkage Mahalanobis. Ensemble uses equal weights fixed before evaluation.
2019 LA
2021 DF
Score
EER
Δ EER
EER
Δ EER
Full ensemble
8.45
—
22.12
—
Drop layer 3
11.64
+3.19
24.19
+2.07
Drop layer 6
8.17
−0.28
22.20
+0.08
Drop layer 9
8.40
−0.05
21.84
−0.28
Drop layer 12
8.28
−0.16
21.52
−0.60
TABLE IV: Leave-one-layer-out for the MLN-WavLM-Full ensemble. Positive Δ EER means removing the layer hurts the ensemble.
System
EER (%)
AUC
TPR@5
%FPR (%)
Centroid, PCA-128
29.36
0.764
6.57
MLN-WavLM-PCA
25.06
0.807
8.58
Centroid, no PCA
26.80
0.792
7.63
MLN-WavLM-Full
22.12
0.845
15.20
KNN ( k=5 ), PCA-128
28.65
0.771
9.82
TABLE V: Frozen ASVspoof 2021 DF results. The supervised reference uses spoof labels during training; all other rows are bona-fide-only.
Fig. 3: ROC curves on frozen ASVspoof 2021 DF. Ranking remains informative, but the operating region near low FPR is weak.
System
2019EER
2021EER
Δ EER
Supervised WavLM-HM-Conformer
7.86
14.94
+7.08
MLN-WavLM-Full
8.45
22.12
+13.67
MLN-WavLM-PCA
12.40
25.06
+12.66
TABLE VI: Relative cross-domain degradation for the same supervised checkpoint and the two pre-specified Mahalanobis variants.
Threshold source
2019 LA
2021 DF
FPR
TPR
FPR
TPR
Bona fide q99 (attack-free)
0.61%
60.07%
20.23%
74.81%
Mixed-dev EER (diagnostic)
9.10%
91.92%
49.86%
98.44%
TABLE VII: Fixed-threshold transfer for the MLN-WavLM-Full ensemble. The mixed-dev EER threshold is a post-hoc diagnostic only.
Fig. 4: Score distributions on 2021 DF dataset. Overlap between bona fide and spoof scores explains why a source-calibrated threshold transfers poorly.
Fig. 5: 2019 LA dataset: unlabeled threshold adaptation versus stream contamination (MLN-WavLM-Full, mean ± std over 10 seeds). Robust MAD transfer keeps recall up to about 10% spoof prevalence; the naive stream quantile does not.
Spoof
Stream quantile
Robust transfer
prevalence
FPR
TPR
FPR
TPR
0%
1.18%
70.7%
0.89%
66.7%
1%
0.52%
53.0%
0.85%
65.9%
5%
0.13%
17.2%
0.72%
62.3%
10%
0.07%
9.2%
0.58%
57.1%
25%
0.05%
3.6%
0.17%
34.9%
TABLE VIII: Unlabeled threshold adaptation on 2019 LA streams (MLN-WavLM-Full, mean over 10 seeds). Frozen threshold: 0.65% FPR / 60.1% TPR. Oracle: 1.18% FPR / 70.7% TPR at every prevalence.
Fig. 6: 2021 DF dataset: threshold adaptation versus stream contamination (Mahalanobis MLN-WavLM-Full, mean ± std over 10 seeds; 96%∗ is the natural eval prevalence). Adaptive thresholds restore the nominal FPR (top) but cannot restore recall (bottom): at a 1% false-alarm budget the shifted-domain ranking supports only ≈ 1.5% TPR even with oracle calibration.