Recent progress in generative models has made talking-head deepfakes highly realistic and easy to create, jeopardizing public trust. However, most audio-visual deepfake detectors rely on lip-speech synchronization and are designed for static, frontal videos, limiting their reliability in dynamic, in-the-wild recordings where the speaker moves and lip cues are degraded or missing. As an alternative, we propose a physics-aware detector for dynamic speaking videos that leverages an acoustic constraint: in real recordings, speech energy varies predictably with speaker-to-camera distance, whereas deepfake generation can break this coupling. Our method combines distance estimates from video with distance-dependent acoustic measures from speech to detect deepfakes. Experiments on in-the-wild benchmarks show that the proposed physical cue is effective in dynamic scenarios, and that it complements lip-sync-based detectors, with a simple ensemble achieving consistent gains across datasets. Codes are available at https://byulharang.github.io/PADD/.
Figures & tables
Figure 1: Physics-aware deepfake detection for dynamic speaking videos. Existing detectors rely on lip–speech consistency and may fail when lip regions are partially visible or missing. Our method instead exploits the physical relationship between speaker distance and speech energy, enabling reliable detection even when visual lip cues are unreliable.
Figure 2: Physics-aware deepfake detection framework. Our method extracts speaker distance and acoustic measures from video and audio streams. For speaker distance, we calculated dspk by averaging monocular depth values (MiDaS [ 12 ] ) within facial landmarks (RetinaFace [ 13 ] ). For acoustic measures, SNR and C50 were calculated using speech and noise separated by SAM-Audio [ 14 ] with a ’speech’ prompt. Distance-wise binning is then applied and the variance and entropy of the acoustic measures are aggregated to form statistical features v . Finally, v is fed into a lightweight MLP detector to predict whether it is an authentic or a deepfake video.
Method
ROC-AUC
SpeechForensics [ 18 ]
0.5096
AuViRe [ 19 ]
0.5517
Ours
0.7449
Table 1: Results on the dynamic subset DF-Dynamic This subset contains videos with strong dynamics, such as substantial speaker motion. Under these conditions, our method substantially outperforms existing lip-sync-based detectors.
Setting
In-domain
Cross-dataset
Train: DF24
Train: DF24
Test: DF24 (mixed)
Test: RW (static-heavy)
\Block [l]1-3 (A) Lip-sync-based detectors
SpeechForensics [ 18 ]
0.6957
0.7819
AuViRe [ 19 ]
0.6890
0.6430
\Block [l]1-3 (B) Physics-aware detector
Table 2: Results on the in-the-wild benchmarks. Models are trained on the DF24 training split and evaluated under two settings: (i) in-domain evaluation on the DF24 test set (mixed; static and dynamic videos) and (ii) cross-dataset evaluation on the RW dataset (largely static videos). (C) reports score-averaging ensembles between lip-sync-based detectors and ours.
Figure 3: Qualitative examples of authentic (left) and deepfake (right) videos. We visualize the temporal trends of distance , SNR , and C50 . In authentic videos, the acoustic measures increase as the speaker moves closer to the camera, following the expected physical relationship between distance and speech energy. In contrast, deepfake videos often exhibit inconsistent trends between distance and acoustic measures.
Configuration
DF24
RW
ROC-AUC
Δ
ROC-AUC
Δ
Ours (Full)
0.7329
–
0.6510
–
(A) Acoustic-measure ablation
w/o SNR stats ( VarSNR,HSNR )
0.7187
-0.0262
0.6528
+0.0018
w/o C50 stats ( VarC50,HC50 )
0.6160
-0.1169
0.6090
-0.0420
(B) Statistical-descriptor ablation
Table 3: Ablation of physical features. Each row removes a subset of features from our model; Δ is relative score to ours.
Configuration
DF24
RW
Ours (with SAM Audio)
0.7329
0.6510
w/o SAM Audio
0.5945
0.5671
Table 4: Ablation of the sound separation module. Removing the sound separation module degrades ROC-AUC scores.
The rapid advancement of generative models has enabled highly realistic audio deepfakes, yet current detectors suffer from a critical bias problem, leading to poor generalization across unseen datasets. This paper proposes Artifact-Focused Self-Synthesis (AFSS), a method designed to mitigate this bias by generating pseudo-fake samples from real audio via two mechanisms: self-conversion and self-reconstruction. The core insight of AFSS lies in enforcing same-speaker constraints, ensuring that real and pseudo-fake samples share identical speaker identity and semantic content. This forces the detector to focus exclusively on generation artifacts rather than irrelevant confounding factors. Furthermore, we introduce a learnable reweighting loss to dynamically emphasize synthetic samples during training. Extensive experiments across 7 datasets demonstrate that AFSS achieves state-of-the-art performance with an average EER of 5.45%, including a significant reduction to 1.23% on WaveFake and 2.70% on In-the-Wild, all while eliminating the dependency on pre-collected fake datasets. Our code is publicly available at https://github.com/NguyenLeHaiSonGit/AFSS.
Audio deepfakes generated by neural text-to-speech and voice-cloning systems threaten speaker verification and public discourse at scale. The core challenge is cross-dataset generalization: detectors trained on one synthesis pipeline collapse on unseen forgeries. We argue that this failure is primarily because of structural synthetic speech artifacts which are multi-timescale trajectory anomalies. Though every existing detector aggregates a fixed-window frame statistics, this misaligns the architecture with the signal. We propose FlowFake, a Liquid Time-Constant (LTC) architecture whose hidden state evolves via a learned ODE, with per-neuron adaptive time constants simultaneously resolving spectral (10ms) and prosodic (2s) cues. At only 34K parameters FlowFake achieves formal BIBO stability and O(dt^4) integration error. On a four-dataset cross domain benchmark (ASVspoof2019-LA, FakeOrReal, InTheWild, MLAAD), FlowFake reaches 75.29% on ASVspoof2019 trained only on FakeOrReal and 79.97% trained only on MLAAD. It outperforms RawGAT-ST and Whisper-DF on every evaluated pair and matching SSL Wav2vec2 (300x larger) at 0.01% of its parameter count. The source code is available on : https://github.com/GhostRider2023/FlowFake
Audio deepfakes have improved rapidly recently, yet their effect on human trust in real speech remains unstudied. We present the largest listening study on audio deepfake perception to date, collecting 35,532 judgments from 1,768 participants across 138 text-to-speech and voice conversion systems. Our central finding is a skepticism shift: compared to a 2021 baseline, human accuracy on fake samples barely changed (72.9% to 71.2%), but accuracy on real samples dropped from 72.7% to 64.1%. Participants are not worse at detecting synthesis artifacts; rather, they increasingly distrust authentic speech. Samples generated by commercial and autoregressive language model systems proved hardest to detect (61.3 - 65.9%), while those from traditional seq2seq and flow-matching models remain easier to spot (75.4 - 76.8%). An ML detector that served as a reference point maintained over 94.5% accuracy across all conditions. Our results suggest that the primary threat posed by modern deepfakes may not be mere deception, but the erosion of trust in genuine audio.
Nicolas M. Müller, Wei Herng Choong
Fraunhofer AISEC & Resemble AI · Germany · Fraunhofer AISEC