Complementary rPPG-Derived and Lip-Region Frequency Cues for Talking-Face Deepfake Detection

Othmane Harraq; Tamer Aldwairi

cs.CVSep 13, 2026

Complementary rPPG-Derived and Lip-Region Frequency Cues for Talking-Face Deepfake Detection

Authors: Othmane Harraq, Tamer Aldwairi

Organizations: Temple University Philadelphia, PA, USA

Abstract

Talking-face (TF) deepfakes are detected unevenly by rPPG-based methods across generators. We study two lightweight visual-only cues, rPPG-derived waveforms extracted by RhythmFormer and lip-region discrete cosine transform (DCT) coefficients, on the seven TF methods of Celeb-DF++ under a subject-independent protocol. In-domain, lip-region DCT matches or exceeds the rPPG-derived 1D ResNet on every method except SadTalker, and Concat fusion reaches AUC 0.891 against 0.824 and 0.827 for the unimodal baselines. Under leave-one-generator-out evaluation the cues split: each transfers clearly better to three held-out methods, and IP-LAP is near chance for both. Concat averages 0.798 but falls below rPPG alone where DCT transfers poorly, so static fusion only partly exploits this complementarity. Lip-region DCT outperforms full-face DCT on six of seven methods. We treat the rPPG-derived signal as an empirical cue and do not claim it is cardiac in origin.

Figures & tables

Figure 1: Proposed rPPG $+$ DCT concat fusion pipeline. The rPPG branch (bottom) yields a z-scored waveform $\mathbf{x}\in\mathbb{R}^{160}$ ; the frequency branch (top) yields lip-region DCT coefficients $\mathbf{d}\in\mathbb{R}^{64}$ . Both are projected to 64-dim, concatenated, and classified. Bar heights indicate dimensionality.

Method	rPPG	rPPG ${}_{\text{lin}}$	DCT	Gated	Con.	CA
AniTalker	0.923	0.903	0.953	0.971	0.978	0.945
EchoMimic	0.878	0.824	0.990	0.992	0.993	0.984
EDTalk	0.956	0.946	0.965	0.985	0.990	0.956
FLOAT	0.824	0.769	0.984	0.984	0.987	0.979
IP-LAP	0.698	0.728	0.792	0.837	0.854	0.767
Real3DPortrait	0.988	0.990	0.989	0.998	0.998	0.987

Table 1: Per-method AUC. Rows: one model per method. Combined: one model trained on all seven methods, pooled AUC. rPPG: 1D ResNet; rPPG

{}_{\text{lin}}

: the linear encoder used inside fusion models. 5 seeds, std

\leq

0.010.

Held-out	rPPG	DCT	Concat
AniTalker	0.839	0.657	0.816
EchoMimic	0.650	0.906	0.908
EDTalk	0.848	0.671	0.768
FLOAT	0.771	0.896	0.910
IP-LAP	0.554	0.534	0.563
Real3DPortrait	0.736	0.815	0.893

Table 2: Leave-one-generator-out AUC. Train on six methods, test on the held-out one; identity-disjoint, 5 seeds. std

\leq

0.017, except rPPG on Real3DPortrait (0.035). Mean: unweighted over held-out methods.

	DCT only		Concat
Method	Lip	Face	Lip	Face
AniTalker	0.953	0.954	0.978	0.975
EchoMimic	0.990	0.979	0.993	0.983
EDTalk	0.965	0.945	0.990	0.985
FLOAT	0.984	0.969	0.987	0.974
IP-LAP	0.792	0.641	0.854	0.755

Table 3: Lip-Region vs Full-Face DCT. 5 seeds, std

\leq

0.010. Lip-region outperforms full-face on six of seven methods for DCT-only.

Figure 2: Representative rPPG-derived waveforms (left) and lip-region DCT log-magnitude maps (right, shared color scale) for a real video, an IP-LAP fake, and a SadTalker fake of one identity. Per-method AUC: rPPG 0.698 (IP-LAP) vs 0.946 (SadTalker); DCT 0.792 vs 0.712. Illustrative, not quantitative evidence.

Figure 3: Per-method AUC for the rPPG-derived branch (1D ResNet), lip-region DCT, and Concat. Left: per-method training. Right: leave-one-generator-out. Axis starts at chance.

Deepfake Detection

Deepfake detection identifies manipulated or AI-generated media to support trustworthy communication, identity verification, and forensic analysis.

See all 246 papers

Physiological Signals as a Forensic Modality for Talking-Face Deepfake Detection
Jul 23, 2026Othmane Harraq, Tamer AldwairiRemote Photoplethysmography Deepfake Detection
Temple University
Beyond Ambiguous Visual Cues: Studying Physiological Disruptions and Cross-Modal Inconsistencies in Deepfake Videos
Sep 11, 2026Chenxi Yang, Yassine Ouzar, Larbi BoubchirRemote Photoplethysmography Deepfake Detection
LIASD Laboratory Shanghai Jiaotong University Shanghai, China · LIASD Laboratory University of Paris 8 Saint-Denis, France
Physics-Aware Deepfake Detection via Distance-Speech Consistency
Oct 13, 2026Kyeongrae Kim, Kim Sung-Bin, Oh Hyun-Bin +1Deepfake Detection Audio Deepfake Detection
School of Electrical Engineering, KAIST, Korea · Dept. of Electrical Engineering, POSTECH, Korea · School of Computing, KAIST, Korea

Jul 23, 2026J/K move · Enter open · S save

Physiological Signals as a Forensic Modality for Talking-Face Deepfake Detection

Othmane Harraq, Tamer Aldwairi

Temple University

Remote Photoplethysmography Deepfake Detection Photoplethysmography

Open paper AskPDF arXiv

Talking-face (TF) deepfake generation synthesizes photore- alistic facial video from a static source image and an au- dio signal, producing forgeries that current image-based detectors consistently fail to identify. Unlike face-swap ma- nipulation, TF synthesis has no underlying real video from which to inherit physiological characteristics, making re- mote photoplethysmography (rPPG) a uniquely motivated detection modality for this forgery category. We propose a detection framework that extracts per-video rPPG wave- forms via RhythmFormer and trains a suite of lightweight classifiers to distinguish real from synthesized physiologi- cal signals. Evaluated on the TF subset of Celeb-DF++ un- der a strict subject-independent protocol, where test identi- ties are completely separated from training identities, our 1D ResNet achieves an AUC of 0.806 and EER of 27.8%, placing it within 2.4 points of the best published general- purpose detector (Effort, ICML 2025) while operating ex- clusively on the physiological channel. We document a con- trolled reproduction study of DeepFakesON-Phys, the rep- resentative prior rPPG detector, demonstrating degrada- tion from AUC 0.999 on legacy face-swap data to 0.622 on the TF subset of Celeb-DF++. We further show that detec- tion difficulty is strongly method-dependent: AUC ranges from 0.985 (Real3DPortrait) to 0.690 (IP-LAP) across the seven TF generators, with the ranking remaining perfectly stable across all evaluation protocols. This spread reflects an interpretable physiological property of each generator rather than evaluation noise, and constitutes the primary theoretical contribution of the work.

Abstract

Deepfake Detection

Explore similar work

Physiological Signals as a Forensic Modality for Talking-Face Deepfake Detection

Beyond Ambiguous Visual Cues: Studying Physiological Disruptions and Cross-Modal Inconsistencies in Deepfake Videos

Physics-Aware Deepfake Detection via Distance-Speech Consistency