In Deepfake Detection (DFD) tasks, researchers proposed two types of MLLM-based methods: complementary combination with small DFD detectors, or static forgery knowledge injection. The lack of professional forgery knowledge hinders the performance of these DFD-MLLMs. To solve this, we deeply considered two insightful issues: How to provide high-quality associated forgery knowledge for MLLMs? AND How to endow MLLMs with critical reasoning abilities given noisy reference information? Notably, we attempted to address above two questions with preliminary answers by leveraging the combination of Retrieval-Augmented Generation (RAG) and Reinforcement Learning (RL). Through RAG and RL techniques, we propose the VRAG-DFD framework with accurate dynamic forgery knowledge retrieval and powerful critical reasoning capabilities. Specifically, in terms of data, we constructed two datasets with RAG: Forensic Knowledge Database (FKD) for DFD knowledge annotation, and Forensic Chain-of-Thought Dataset (F-CoT), for critical CoT construction. In terms of model training, we adopt a three-stage training method (Alignment->SFT->GRPO) to gradually cultivate the critical reasoning ability of the MLLM. In terms of performance, VRAG-DFD achieved SOTA and competitive performance on DFD generalization testing.
Figures & tables
Figure 1: Comparison between the previous method and the proposed VRAG-DFD framework. Previous methods predicted based on a single image and prompt with severe hallucinations. The proposed VRAG-DFD effectively reduce hallucinations by constructing highly relevant references through RAG and making decisions through critical reasoning. (Faces are selected from publicly available FF++ [ 40 ] dataset.)
Figure 2: The anotation phases of the Forensic Knowledge Database (FKD) and Forensic Chain-of-Thought (F-CoT) Dataset. (Faces are selected from FF++ [ 40 ] dataset.)
Figure 3: Framework of the proposed VRAG-DFD. Firstly, the dynamic forensic retriever captures top- k samples from the multimodal FKD database. Afterwards, MLLM conducts a critical reasoning process ( e.g. , ①-②-③) based on the text information retrieved by RAG , and ultimately outputs the final conclusion ( e.g. , ④). (Faces are selected from FF++ [ 40 ] dataset.)
Figure 4: Training stages of VRAG-DFD. Stage-1 aims to equip the MLLM with basic forgery artifacts recognition capabilities. Stage-2 aims to guide the model to predict in the form of F-CoT. Stage-3 adopts GRPO with detailed reward function designs to encourage the MLLM to have the capabilities to critical reasoning given top- k RAG evidence. (Faces are selected from FF++ [ 40 ] dataset.)
Method
Venue
Explainable
Research Topic
CDF-v1
CDF-v2
DFDC
FFIW
WDF
F 3 -Net [ 36 ]
ECCV 2020
✗
Frequency + CNN
81.11
77.92
67.35
70.11
72.80
LTW [ 43 ]
AAAI 2021
✗
Meta Training
-
77.14
69.00
76.63
-
SPSL [ 28 ]
CVPR 2021
✗
Phase Spectrum
-
79.90
77.00
79.40
70.20
SRM [ 30 ]
CVPR 2021
✗
Attention + FPN
-
84.00
69.50
80.60
72.20
PCL+I2G [ 54 ]
ICCV 2021
✗
Blending
-
90.03
67.52
-
-
SBI [ 41 ]
CVPR 2022
✗
Blending
93.44
93.18
72.42
84.83
-
Table 1: Cross-dataset performance comparison with SOTA face forgery detection methods on CDF-v1 [ 25 ] , CDF-v2 [ 25 ] , DFDC [ 8 ] , FFIW [ 58 ] , and WDF [ 60 ] . All methods are trained on FF++ [ 40 ] . The best results are shown in bold , and the second results are underlined . The Video-level AUC metric is adopted for comparison.
Stage
Objective
Epoch
lr
Batch
Param.
Stage-1
Alignment
3
5×10−5
512
-
Stage-2
SFT
2
3×10−5
64
-
Stage-3
GRPO
1
1×10−6
32
β=0.001
Table 2: Training configurations of three training stages.
Metrics
Test Set AUC
CDF1
CDF2
DFDC
FFIW
WDF
w/o RAG Module
91.13
87.85
72.42
78.62
71.62
VRAG-DFD
99.60
95.97
81.82
93.49
88.96
Table 3: Ablation study of RAG (AUC %). w/o RAG removes the forensic retriever and all associated critical reasoning components.
Metrics
Test Set AUC
CDF1
CDF2
DFDC
FFIW
WDF
Stage-1
96.77
94.03
78.54
90.89
87.63
Stage-1&2
91.53
95.8
81.58
92.37
89.13
VRAG-DFD
99.60
95.97
81.82
93.49
88.96
Table 4: Ablation study of the three-stage training pipeline. We report AUC (%) on five generalization benchmarks.
Retrievers
Test Set AUC
Avg.
CDF1
CDF2
DFDC
FFIW
WDF
Effort [ 49 ]
99.60
95.97
81.82
93.49
88.96
91.62
CLIP-LoRA [ 17 ]
100.00
95.77
82.47
94.00
88.42
92.13
Table 5: Ablation study of retriever (AUC %). ‘CLIP-LoRA’ replaces Effort [ 49 ] with a standard CLIP model fine-tuned by LoRA.
Model
GPT-4o
Gemini2.5 Pro
Avg.
GPT-4o
4.60
3.25
3.93
Gemini2.5 Pro
7.31
7.02
7.16
VRAG-DFD
7.55
7.78
7.66
Table 6: Explanation quality evaluation. GPT-4o and Gemini 2.5 Pro serve as evaluators. VRAG-DFD achieves the highest average score.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Metrics
Test Set AUC
Avg.
CDF1
CDF2
DFDC
FFIW
WDF
DD-VQA [ 53 ]
96.17
94.83
79.48
92.50
85.88
89.77
FKD (Ours)
99.60
95.97
81.82
93.49
88.96
91.62
Appendix
Table 7: Data Quality Comparison between FKD and DDVQA. Replacing FKD with DDVQA leads to a significant performance drop, indicating that forensic-grade knowledge is essential for effective reasoning.
Stage
Config
Test Set AUC
Δ
CDF1
CDF2
DFDC
FFIW
WDF
Stage-1
w/o RAG
89.72
86.30
73.31
78.59
78.73
+8.16
w/ RAG
96.77
94.03
78.54
90.89
87.63
Stage-2
w/o RAG
91.33
86.96
71.57
76.43
70.05
+10.66
w/ RAG
91.53
95.80
81.58
92.37
89.13
Stage-3
w/o RAG
91.53
86.98
72.05
76.51
70.04
+12.2
Appendix
Table 8: Ablation study on the impact of the RAG module across three training stages. We report the AUC (%) on five generalization benchmarks. The column ‘ Δ Avg.’ highlights the performance gain attributable to the RAG module within each stage. Crucially, we observe that RAG provides substantial improvements consistently across all stages , demonstrating that dynamic retrieval of expert knowledge is fundamental to bridging the generalization gap, even in the early visual alignment phase.
Metrics
Test Set AUC
CDF1
CDF2
DFDC
FFIW
WDF
Qwen2.5VL-7B
66.00
73.35
56.82
71.91
66.82
Qwen2.5VL-7B + ours
99.60
95.97
81.82
93.49
88.96
Intern3VL-8B
66.33
72.06
57.19
70.86
65.75
Intern3VL-8B + ours
99.19
95.09
80.16
92.45
90.00
Appendix
Table 9: Effectiveness across different base MLLMs (AUC %). We show the performance of each model before (zero-shot) and after (’+ VRAG-DFD (Ours)’) applying our three-stage VRAG-DFD pipeline.
Retrieval
Inference
Total
Metric
GFLOPs
Ratio
GFLOPs
Ratio
GFLOPs
Ratio
Value
∼ 81
0.35%
∼ 22,760
99.65%
∼ 22,841
100.00%
Appendix
Table 10: Computational complexity analysis of VRAG-DFD.
Samp. Strategy
Total
Test Set AUC
Avg.
CDF1
CDF2
DFDC
FFIW
WDF
Sparse (8 fms)
∼ 28k
92.14
76.85
70.69
70.40
77.95
77.61
Dense (All fms)
∼ 80k
89.72
86.30
73.71
78.59
78.73
81.41
Gain ( Δ )
-
-2.42
+9.45
+3.02
+8.19
+0.78
+3.80
Appendix
Table 11: Ablation study on frame sampling density in Stage-1. We compare sparse sampling (8 frames/video) against dense sampling (all frames). The results demonstrate a substantial performance leap in the average AUC (+3.80%), confirming that dense visual supervision is indispensable for learning robust forgery features across diverse datasets.
Metric
Test Set AUC
Weigh. Avg.
CDF-v1
CDF-v2
DFDC
FFIW
WDF
# Adv. Samp.
34
429
534
1,374
5,226
7,597
# VRAG_DFD Cor.
32
417
520
1,345
5,048
7,362
Robust. Rate (%)
94.12
97.20
97.38
97.89
96.59
96.91
Appendix
Table 12: Robustness against misleading retrieval evidence. We evaluate a subset of adversarial samples where the initial visual perception is correct ( s1 =Correct), but the RAG evidence is misleading ( RAG =False). The results demonstrate that VRAG-DFD maintains an exceptionally high Robustness Rate ( >96% on avg.), effectively rejecting misleading external noise.
Figure 5: Qualitative analysis of VRAG-DFD (real case 1).
Figure 7: Qualitative analysis of VRAG-DFD (fake case 1).
Multimodal large language models (MLLMs) offer a promising path toward interpretable deepfake detection by generating textual explanations. However, the reasoning process of current MLLM-based methods combines evidence generation and manipulation localization into a unified step. This combination blurs the boundary between faithful observations and hallucinated explanations, leading to unreliable conclusions. Building on this, we present VIGIL, a part-centric structured forensic framework inspired by expert forensic practice through a plan-then-examine pipeline: the model first plans which facial parts warrant inspection based on global visual cues, then examines each part with independently sourced forensic evidence. A stage-gated injection mechanism delivers part-level forensic evidence only during examination, ensuring that part selection remains driven by the model's own perception rather than biased by external signals. We further propose a progressive three-stage training paradigm whose reinforcement learning stage employs part-aware rewards to enforce anatomical validity and evidence--conclusion coherence. To enable rigorous generalizability evaluation, we construct OmniFake, a hierarchical 5-Level benchmark where the model, trained on only three foundational generators, is progressively tested up to in-the-wild social-media data. Extensive experiments on OmniFake and cross-dataset evaluations demonstrate that VIGIL consistently outperforms both expert detectors and concurrent MLLM-based methods across all generalizability levels.
Xinghan Li, Junhao Xu, Jingjing Chen
College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China.
Existing explainable deepfake forensic methods typically rely on task-adapted MLLM to jointly address detection, localization, and explanation. Inspired by agent-style tool use, we instead introduce a Perception-as-Tool paradigm and instantiate it as PATE-Forensics, which architecturally decouples detection and localization from explanation generation while coupling detection and localization as tightly as possible within a forensic perception tool. The DINOv3-based tool couples a multi-granularity detection module that integrates global, patch-level, and segment-level evidence with a cue-guided localization module by spatializing the patch-level and segment-level evidence into forgery score maps that guide dense mask prediction. The original image and forensic perception outputs produced by the tool form structured forensic context for a general-purpose MLLM, which is guided by prompt constraints to generate explanations without task-specific fine-tuning. On DDL-X Track 3, PATE-Forensics achieves the best official score of 0.89, outperforming the second-ranked team by 0.19 points. Our code is available at https://github.com/yqli00000/PATE-Forensics.
The rapid advancement of deepfake generation techniques poses significant threats to public safety and causes societal harm through the creation of highly realistic synthetic facial media. While existing detection methods demonstrate limitations in generalizing to emerging forgery patterns, this paper presents Deepfake Forensics Adapter (DFA), a novel dual-stream framework that synergizes vision-language foundation models with targeted forensics analysis. Our approach integrates a pre-trained CLIP model with three core components to achieve specialized deepfake detection by leveraging the powerful general capabilities of CLIP without changing CLIP parameters: 1) A Global Feature Adapter is used to identify global inconsistencies in image content that may indicate forgery, 2) A Local Anomaly Stream enhances the model's ability to perceive local facial forgery cues by explicitly leveraging facial structure priors, and 3) An Interactive Fusion Classifier promotes deep interaction and fusion between global and local features using a transformer encoder. Extensive evaluations of frame-level and video-level benchmarks demonstrate the superior generalization capabilities of DFA, particularly achieving state-of-the-art performance in the challenging DFDC dataset with frame-level AUC/EER of 0.816/0.256 and video-level AUC/EER of 0.836/0.251, representing a 4.8% video AUC improvement over previous methods. Our framework not only demonstrates state-of-the-art performance, but also points out a feasible and effective direction for developing a robust deepfake detection system with enhanced generalization capabilities against the evolving deepfake threats. Our code is available at https://github.com/Liao330/DFA.git
Jianfeng Liao, Yichen Wei, Raymond Chan Ching Bon +3
Shenzhen Technology University, Guangdong, China · Singapore Institute of Technology, Singapore, Singapore · The University of Hong Kong, Hong Kong, China +1