Paper ID: 2501.01164 • Published Jan 2, 2025
Towards Interactive Deepfake Analysis
Lixiong Qin, Ning Jiang, Yang Zhang, Yuhan Qiu, Dingheng Zeng, Jiani Hu, Weihong Deng
TL;DR
Get AI-generated summaries with premium
Get AI-generated summaries with premium
Existing deepfake analysis methods are primarily based on discriminative
models, which significantly limit their application scenarios. This paper aims
to explore interactive deepfake analysis by performing instruction tuning on
multi-modal large language models (MLLMs). This will face challenges such as
the lack of datasets and benchmarks, and low training efficiency. To address
these issues, we introduce (1) a GPT-assisted data construction process
resulting in an instruction-following dataset called DFA-Instruct, (2) a
benchmark named DFA-Bench, designed to comprehensively evaluate the
capabilities of MLLMs in deepfake detection, deepfake classification, and
artifact description, and (3) construct an interactive deepfake analysis system
called DFA-GPT, as a strong baseline for the community, with the Low-Rank
Adaptation (LoRA) module. The dataset and code will be made available at
this https URL to facilitate further research.