Recent progress in generative models has made talking-head deepfakes highly realistic and easy to create, jeopardizing public trust. However, most audio-visual deepfake detectors rely on lip-speech synchronization and are designed for static, frontal videos, limiting their reliability in dynamic, in-the-wild recordings where the speaker moves and lip cues are degraded or missing. As an alternative, we propose a physics-aware detector for dynamic speaking videos that leverages an acoustic constraint: in real recordings, speech energy varies predictably with speaker-to-camera distance, whereas deepfake generation can break this coupling. Our method combines distance estimates from video with distance-dependent acoustic measures from speech to detect deepfakes. Experiments on in-the-wild benchmarks show that the proposed physical cue is effective in dynamic scenarios, and that it complements lip-sync-based detectors, with a simple ensemble achieving consistent gains across datasets. Codes are available at https://byulharang.github.io/PADD/.
Figures & tables
Figure 1: Physics-aware deepfake detection for dynamic speaking videos. Existing detectors rely on lip–speech consistency and may fail when lip regions are partially visible or missing. Our method instead exploits the physical relationship between speaker distance and speech energy, enabling reliable detection even when visual lip cues are unreliable.
Figure 2: Physics-aware deepfake detection framework. Our method extracts speaker distance and acoustic measures from video and audio streams. For speaker distance, we calculated dspk by averaging monocular depth values (MiDaS [ 12 ] ) within facial landmarks (RetinaFace [ 13 ] ). For acoustic measures, SNR and C50 were calculated using speech and noise separated by SAM-Audio [ 14 ] with a ’speech’ prompt. Distance-wise binning is then applied and the variance and entropy of the acoustic measures are aggregated to form statistical features v . Finally, v is fed into a lightweight MLP detector to predict whether it is an authentic or a deepfake video.
Method
ROC-AUC
SpeechForensics [ 18 ]
0.5096
AuViRe [ 19 ]
0.5517
Ours
0.7449
Table 1: Results on the dynamic subset DF-Dynamic This subset contains videos with strong dynamics, such as substantial speaker motion. Under these conditions, our method substantially outperforms existing lip-sync-based detectors.
Setting
In-domain
Cross-dataset
Train: DF24
Train: DF24
Test: DF24 (mixed)
Test: RW (static-heavy)
\Block [l]1-3 (A) Lip-sync-based detectors
SpeechForensics [ 18 ]
0.6957
0.7819
AuViRe [ 19 ]
0.6890
0.6430
\Block [l]1-3 (B) Physics-aware detector
Table 2: Results on the in-the-wild benchmarks. Models are trained on the DF24 training split and evaluated under two settings: (i) in-domain evaluation on the DF24 test set (mixed; static and dynamic videos) and (ii) cross-dataset evaluation on the RW dataset (largely static videos). (C) reports score-averaging ensembles between lip-sync-based detectors and ours.
Figure 3: Qualitative examples of authentic (left) and deepfake (right) videos. We visualize the temporal trends of distance , SNR , and C50 . In authentic videos, the acoustic measures increase as the speaker moves closer to the camera, following the expected physical relationship between distance and speech energy. In contrast, deepfake videos often exhibit inconsistent trends between distance and acoustic measures.
Configuration
DF24
RW
ROC-AUC
Δ
ROC-AUC
Δ
Ours (Full)
0.7329
–
0.6510
–
(A) Acoustic-measure ablation
w/o SNR stats ( VarSNR,HSNR )
0.7187
-0.0262
0.6528
+0.0018
w/o C50 stats ( VarC50,HC50 )
0.6160
-0.1169
0.6090
-0.0420
(B) Statistical-descriptor ablation
Table 3: Ablation of physical features. Each row removes a subset of features from our model; Δ is relative score to ours.
Configuration
DF24
RW
Ours (with SAM Audio)
0.7329
0.6510
w/o SAM Audio
0.5945
0.5671
Table 4: Ablation of the sound separation module. Removing the sound separation module degrades ROC-AUC scores.