MSCT: Differential Cross-Modal Attention for Deepfake Detection

Fangda Wei; Miao Liu; Yingxue Wang; Jing Wang; Shenghui Zhao; Nan Li

cs.CVApr 9, 2026

MSCT: Differential Cross-Modal Attention for Deepfake Detection

Authors: Fangda Wei, Miao Liu, Yingxue Wang, Jing Wang, Shenghui Zhao, Nan Li

Organizations: Beijing Institute of Technology, China · China Academy of Electronics and Information Technology, China

Abstract

Audio-visual deepfake detection typically employs a complementary multi-modal model to check the forgery traces in the video. These methods primarily extract forgery traces through audio-visual alignment, which results from the inconsistency between audio and video modalities. However, the traditional multi-modal forgery detection method has the problem of insufficient feature extraction and modal alignment deviation. To address this, we propose a multi-scale cross-modal transformer encoder (MSCT) for deepfake detection. Our approach includes a multi-scale self-attention to integrate the features of adjacent embeddings and a differential cross-modal attention to fuse multi-modal features. Our experiments demonstrate competitive performance on the FakeAVCeleb dataset, validating the effectiveness of the proposed structure.

Figures & tables

Figure 1: The video and audio parts corresponding to each frame contains incomplete text information

Figure 2: Our proposed approach consists of encoder and classification block (A_CLS, V_CLS, M_CLS). The encoder contains pre-encoder (A_E, V_E) and transformer module, which consists of self attention (SA) and cross attention (CA).

Figure 3: Differential cross-modal attention module

Figure 4: Multi-scale self-attention module

Method	ACC $\uparrow$	AUC $\uparrow$
VFD [ 2 ]	81.52	86.11
MDS [ 3 ]	82.80	86.50
AVOID-DF [ 16 ]	83.70	89.20
MRDF-CE [ 18 ]	94.05	92.43
BusterX [ 13 ]	96.30	-
Ours	98.75	98.83

Table 1: Comparison with other methods on FakeAVCeleb

Model	ACC $\uparrow$	AUC $\uparrow$
$CA+SA$	96.75	96.17
$CA+MSSA$	97.00	97.00
$DCA+SA$	98.00	98.00
$DCA+MSSA$	98.75	98.83

Table 2: Ablation study

Panel of Figure 5 — Figure 5: Comparison of T-SNE results of different models.

Explore similar work

Cross-modal Translation via Conditional Latent Denoising for Video Deepfake Detection
Sep 27, 2026Xinzhe Li, Youzhi Tu, Kong Aik LeeCross-Modal Learning Cross-Modal Alignment
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong SAR
Attribution-Guided Multimodal Deepfake Detection via Cross-Modal Forensic Fingerprints
Apr 29, 2026Wasim Ahmad, Wei Zhang, Xuerui MaoAudio Deepfake Detection Cross-Modal Representation Learning
School of Interdisciplinary Science, Beijing Institute of Technology, Beijing 100081, China
SAVe: Self-Supervised Audio-visual Deepfake Detection Exploiting Visual Artifacts and Audio-visual Misalignment
Mar 26, 2026Sahibzada Adil Shahzad, Ammarah Hashmi, Junichi Yamagishi +5Deepfake Detection Audio Deepfake Detection
Social Networks and Human-Centered Computing Program, Taiwan International Graduate Program, Institute of Information Science, Academia Sinica, Taipei 11529, Taiwan · Department of Computer Science, National Chengchi University, Taipei 11529, Taiwan · Institute of Information Systems and Applications, National Tsing Hua University, Hsinchu 300044, Taiwan +4

Sep 27, 2026J/K move · Enter open · S save

Cross-modal Translation via Conditional Latent Denoising for Video Deepfake Detection

Xinzhe Li, Youzhi Tu, Kong Aik Lee

Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong SAR

Cross-Modal Learning Cross-Modal Alignment Deepfake Detection

Open paper AskPDF arXiv

The growing threat of video deepfakes necessitates multimodal detection. Beyond serving as independent indicators of authenticity, audio and visual signals have intrinsic dependencies that also provide an essential criterion for detection. Previous methods often overlook the cross-modal correspondences, hindering information transfer between domains and leaving crucial detection cues unexplored. To address this challenge, we propose a framework called Cross-modal Translation via Conditional Latent Denoising (CTCLD) for video deepfake detection. It connects the distinct distributions of heterogeneous modalities in latent spaces, enabling smooth cross-domain information transfer to improve detection performance. We first establish a Bayesian foundation by decomposing the audio-visual joint distribution. Subsequently, CTCLD translates both modalities via bidirectional latent denoising conditioned on each other, effectively capturing subtle inconsistencies in the manipulated signals. Experimental results demonstrate that the proposed CTCLD enables comprehensive domain alignment, resulting in a robust video deepfake detection approach with competitive performance.

Abstract

Explore similar work

Cross-modal Translation via Conditional Latent Denoising for Video Deepfake Detection

Attribution-Guided Multimodal Deepfake Detection via Cross-Modal Forensic Fingerprints

SAVe: Self-Supervised Audio-visual Deepfake Detection Exploiting Visual Artifacts and Audio-visual Misalignment