Organizations: The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Ant Group · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security
Advances in synthesis techniques have made deepfake speech increasingly realistic, amplifying voice security threats, including unauthorized voice cloning and impersonation. Although existing deepfake detectors perform well on digitally generated speech, their performance often degrades on physically replayed deepfakes, which constitute a low-cost and practical threat to real-world systems. Existing research focuses primarily on dataset construction and evaluation, while available dataset diversity and replay-aware detector design remain limited. We propose ReplayFactor, a dual-attribute framework that enhances replay evidence by fusing self-supervised learning (SSL) representations with an acoustic prior, while leveraging multi-task learning and supervised contrastive learning to promote robust, generalizable representations. We also construct ReplayFake-CN, a large-scale, multi-device Chinese dataset comprising approximately 185k replayed synthetic utterances to complement public benchmarks. Our framework improves macro-F1 from 60.99% to 76.44% and reduces EER from 11.86% to 3.05% on the EchoFake open set over the baseline, demonstrating strong potential for practical replayed deepfake detection.
Figures & tables
Figure 1: Overview of the proposed framework, comprising (I) Representation extraction with SSL and acoustic-prior fusion; and (II) multi-task and supervised contrastive learning.
6 consumer and 3 high-fidelity loudspeakers, e.g., JBL, HP-speaker, Edifier-MR4, etc.
Table 1: Statistics of ReplayFake-CN.
Model
Test
EER ↓
F1-score ↑
macro.
B
RB
F
RF
XLSR-Linear † Reported in [ 21 ]
Closed
0.27
98.81
99.63
98.16
99.43
98.02
Open
11.86
60.99
78.83
40.13
75.64
49.34
ReplayDF
–
–
–
7.72
–
19.02
XLSR-AASIST
Closed
0.47
98.33
98.68
97.97
98.66
98.02
Open
8.17
56.99
86.69
19.96
71.91
49.40
Table 2: Results (%) on EchoFake and ReplayDF. Closed and Open denote the EchoFake evaluation sets. ReplayDF reports RB and RF F1 scores from EchoFake-trained models.
Method
Overall ACC
Unseen Synthesis
Unseen Devices
Unseen Synth.–Devices
Binary public checkpoint: AntiDeepfake [ 7 ]
MMS-300M
55.43
64.04
46.63
62.98
MMS-AASIST
84.22
93.97
82.59
73.30
Ours
85.44
96.95
83.00
73.75
Table 3: Accuracy (%) on ReplayFake-CN under unseen conditions. Binary and four-class models are reported separately.
Configuration
EER ↓
F1-score ↑
macro-F1
B
RB
F
RF
4-class baseline
7.26
62.27
90.14
25.93
77.99
55.02
Ours (full)
3.05
76.44
93.91
61.12
82.01
68.72
w/o acoustic prior
8.11
73.54
89.07
61.44
84.28
59.37
w/o SupCon
2.70
73.89
93.82
50.77
82.85
68.14
w/o 4-class head
4.31
69.22
94.84
32.97
84.03
65.03
Table 4: Ablation results on the EchoFake open set (%).
Figure 2: t-SNE visualizations on the EchoFake closed set: (a) joint, (b) source, and (c) replay representations.
Figure 3: Spectrograms of the original non-replay utterance (top) and its replays through JBL (middle) and Xiaoai (bottom) loudspeakers, both recorded with a Huawei Mate30Pro phone.
Audio deepfakes generated by neural text-to-speech and voice-cloning systems threaten speaker verification and public discourse at scale. The core challenge is cross-dataset generalization: detectors trained on one synthesis pipeline collapse on unseen forgeries. We argue that this failure is primarily because of structural synthetic speech artifacts which are multi-timescale trajectory anomalies. Though every existing detector aggregates a fixed-window frame statistics, this misaligns the architecture with the signal. We propose FlowFake, a Liquid Time-Constant (LTC) architecture whose hidden state evolves via a learned ODE, with per-neuron adaptive time constants simultaneously resolving spectral (10ms) and prosodic (2s) cues. At only 34K parameters FlowFake achieves formal BIBO stability and O(dt^4) integration error. On a four-dataset cross domain benchmark (ASVspoof2019-LA, FakeOrReal, InTheWild, MLAAD), FlowFake reaches 75.29% on ASVspoof2019 trained only on FakeOrReal and 79.97% trained only on MLAAD. It outperforms RawGAT-ST and Whisper-DF on every evaluated pair and matching SSL Wav2vec2 (300x larger) at 0.01% of its parameter count. The source code is available on : https://github.com/GhostRider2023/FlowFake
Advances in speech synthesis have made deepfake audio highly realistic. Earlier studies reported 70-80% human detection accuracy, but relied primarily on older synthesizers. We compare human detection for three selected voice synthesis tools released in 2019, 2022, and 2024 with 82 IT professionals, and benchmark humans against six pretrained detectors on the same material. For fully synthetic speech (full spoofs), the F1 score drops from about 90% for RTVC and YourTTS to 48% for ElevenLabs, although listeners were explicitly warned that deepfakes were present. For partial spoofing, where only one sentence of an utterance is altered, strict accuracy falls to 9%, and listeners classify the synthetic sentence as bona fide 77% of the time. Humans and detectors fail in complementary ways, and neither reliably localizes short manipulations. Additionally, listeners increasingly mislabel bona fide speech as fake, eroding trust in unmanipulated audio. These findings show that human perception alone is unreliable for the selected modern and partial-spoof conditions and motivate procedural verification, provenance, watermarking, and segment-level detection.
The rapid advancement of generative models has enabled highly realistic audio deepfakes, yet current detectors suffer from a critical bias problem, leading to poor generalization across unseen datasets. This paper proposes Artifact-Focused Self-Synthesis (AFSS), a method designed to mitigate this bias by generating pseudo-fake samples from real audio via two mechanisms: self-conversion and self-reconstruction. The core insight of AFSS lies in enforcing same-speaker constraints, ensuring that real and pseudo-fake samples share identical speaker identity and semantic content. This forces the detector to focus exclusively on generation artifacts rather than irrelevant confounding factors. Furthermore, we introduce a learnable reweighting loss to dynamically emphasize synthetic samples during training. Extensive experiments across 7 datasets demonstrate that AFSS achieves state-of-the-art performance with an average EER of 5.45%, including a significant reduction to 1.23% on WaveFake and 2.70% on In-the-Wild, all while eliminating the dependency on pre-collected fake datasets. Our code is publicly available at https://github.com/NguyenLeHaiSonGit/AFSS.