cs.CROct 26, 2025

DeepfakeBench-MM: A Comprehensive Benchmark for Multimodal Deepfake Detection

Authors: Kangran Zhao, Xiaoyu Zhang, Yupeng Chen, Yize Chen, Weinan Guan, Bo Zheng, Yufan Liao, Chengzhe Sun, 

Organizations: The Chinese University of Hong Kong, Shenzhen · University at Buffalo, State University of New York · Nanjing University of Posts and Telecommunications

Abstract

Audio-visual deepfakes can spread misinformation and enable impersonation, undermining trust in digital media. However, existing datasets provide limited coverage of forgery operations and their combinations, constraining detector development and systematic evaluation. We introduce Mega-MMDF, containing 100,000 real and 1.2 million forged clips across 22 pipelines spanning modality-specific manipulations, stacked forgeries, and joint audio-video generation. To assess its training value and enable fair comparisons, we develop DeepfakeBench-MM, which standardizes training and evaluation across 8 datasets and 10 detectors. Training-based detectors achieve their highest mean AUC when trained on Mega-MMDF, with gains of up to 0.0784 over the best alternative training dataset for each detector. Experiments on data scale and pipeline coverage further support the value of large, diverse training data. Cross-pipeline evaluations reveal bidirectional and asymmetric generalization. Training on composite forgeries does not necessarily enable detection of their constituent operations in isolation. Our analysis of altered inputs shows that modality masking can increase false alarms and missed detections, and temporal misalignment can raise mean predicted forgery probabilities. Together, these resources provide a foundation for understanding and improving multimodal deepfake detection, with publicly available code and dataset access subject to application and usage terms.

Figures & tables

Explore similar work

CardsList
  1. Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection

    May 2, 2026Tianxiao Li, Zhenglin Huang, Haiquan Wen +10Multimodal Deepfake DetectionExplainable Deepfake Detection

  2. Leveraging large multimodal models for audio-video deepfake detection: a pilot study

    Feb 25, 2026Songjun Cao, Yuqi Li, Yunpeng Luo +2Multimodal Deepfake DetectionDeepfake Detection

  3. Attribution-Guided Multimodal Deepfake Detection via Cross-Modal Forensic Fingerprints

    Apr 29, 2026Wasim Ahmad, Wei Zhang, Xuerui MaoAudio Deepfake DetectionCross-Modal Representation Learning