Beyond Static Artifacts: A Forensic Benchmark for Video Deepfake Reasoning in Vision Language Models

Zheyuan Gu; Qingsong Zhao; Yusong Wang; Zhaohong Huang; Xinqi Li; Cheng Yuan; Jiaowei Shao; Chi Zhang

cs.CVFeb 25, 2026

Beyond Static Artifacts: A Forensic Benchmark for Video Deepfake Reasoning in Vision Language Models

Authors: Zheyuan Gu, Qingsong Zhao, Yusong Wang, Zhaohong Huang, Xinqi Li, Cheng Yuan, Jiaowei Shao, Chi Zhang, +1 more

Organizations: Institute of Artificial Intelligence, China Telecom (TeleAI) · Peking University · Fudan University

Abstract

Current Vision-Language Models (VLMs) for deepfake detection excel at identifying spatial artifacts but overlook a critical dimension: temporal inconsistencies in video forgeries. Adapting VLMs to reason about these dynamic cues remains a distinct challenge. To bridge this gap, we propose Forensic Answer-Questioning (FAQ), a large-scale benchmark that formulates temporal deepfake analysis as a multiple-choice task. FAQ introduces a three-level hierarchy to progressively evaluate and equip VLMs with forensic capabilities: (1) Facial Perception, testing the ability to identify static visual artifacts; (2) Temporal Deepfake Grounding, requiring the localization of dynamic forgery artifacts across frames; and (3) Forensic Reasoning, challenging models to synthesize evidence for final authenticity verdicts. We evaluate a range of VLMs on FAQ and generate a corresponding instruction-tuning set, FAQ-IT. Extensive experiments show that models fine-tuned on FAQ-IT achieve advanced performance on both in-domain and cross-dataset detection benchmarks. Ablation studies further validate the impact of our key design choices, confirming that FAQ is the driving force behind the temporal reasoning capabilities of these VLMs.

Figures & tables

Figure 1: Illustration of our hierarchical benchmark.

Benchmark	MCQ	#Sample	Annotation	#Task Types	Trace
Image-based Benchmarks
GenImage [ 47 ]	-	2,681,167	A	1	-
DDVQA [ 46 ]	-	14,784	H	1	-
VLFFD [ 36 ]	-	25,663	A+H	1	-
MMTD-Set [ 43 ]	-	34,470	A	2	-
SID-Set desc [ 14 ]	-	3,000	A	2	-

Table 1: Comparison between FAQ and other forgery detection benchmarks. H and A respectively indicate human and automatic annotation.

Figure 2: An overview of FAQ’s data construction pipeline.

Panel of Figure 3 — Figure 3: Visualization of data distribution in FAQ across three levels. (a), (b) and (c) shows QA distribution under different facial region, duration and artifact, respectively.

Proprietary Large Vision Language Models
Model	Level 1	Level 2	Level 3	Average
GPT-4o	$26.9$	$27.1$	$13.2$	$22.8$
Gemini-2.5-Flash	$40.0$	$25.6$	$15.3$	$27.8$
Open-sourced Large Vision Language Models
InternVL-Chat-V1-2	$29.4$	$21.4$	$13.7$	$21.5$
LLaVA-1.5-13b	$36.6$	$27.2$	$17.3$	$27.3$

Table 2: Average and per-level accuracy (%) of various VLMs on our proposed FAQ benchmark. Our benchmark evaluates fine-grained reasoning across three levels.

Model	Condition	Level 1	Level 2	Level 3	Average
Qwen2.5-VL	${\phi}$	24.1	23.8	16.8	21.6
	${FAQ-IT^{\spadesuit}}$	31.3 ↑7.2	21.9 ↓1.9	17.9 ↑1.1	23.9 ↑2.3
	${FAQ-IT}$	89.9 ↑65.8	41.4 ↑17.6	25.8 ↑9.0	52.4 ↑30.8
LLaVA-NeXT	${\phi}$	40.0	29.0	21.2	30.3
	${FAQ-IT^{\spadesuit}}$	49.2 ↑9.2	28.8 ↓0.2	23.3 ↑2.1	33.8 ↑3.5
	${FAQ-IT}$	88.8 ↑48.8	45.8 ↑16.8	26.5 ↑5.3	53.7 ↑23.4

Table 3: Ablation study of BenchIT fine-tuning on LLaVA-NeXT and Qwen2.5-VL models across our benchmark levels. VE denotes Vision Encoder. The symbol

{\phi}

denotes a zero-shot scenario where the model has not been trained. Symbol

{FAQ-IT^{\spadesuit}}

and

{FAQ-IT}

means training models on static-only QA from FAQ and training on full FAQ, respectively.

In-domain MCQ ACC (%)
Model	Condition	FS	NT	F2F	DF	FSh	Avg
Qwen2.5-VL	${\phi}$	9.7	7.4	13.5	9.6	7.7	9.6
	${FAQ-IT^{\spadesuit}}$	10.5 ↑0.8	13.3 ↑5.8	12.8 ↓0.7	10.4 ↑0.8	10.7 ↑3.0	11.5 ↑1.9
	${FAQ-IT}$	45.9 ↑36.2	46.7 ↑39.3	24.4 ↑10.9	45.3 ↑35.7	45.3 ↑37.6	41.5 ↑31.9
LLaVA-NeXT	${\phi}$	24.7	23.0	19.7	21.6	24.2	22.6
	${FAQ-IT^{\spadesuit}}$	27.2 ↑2.5	20.8 ↓2.2	25.0 ↑5.3	22.4 ↑0.8	27.9 ↑3.7	24.7 ↑2.1

Table 4: In-domain MCQ accuracy & In-domain Detection accuracy. Symbol definitions follow Table 3 .

Model	Com.	FS	NT	F2F	DF	FSh
Qwen2.5-VL FAQ-IT	c23	$45.9$	$46.7$	$24.4$	$45.3$	$45.3$
Qwen2.5-VL FAQ-IT	c40	$41.8$	$42.2$	$20.9$	$40.1$	$40.4$
	original	$46.1$	$46.5$	27.2	$47.3$	$46.0$
LLaVA-NeXT FAQ-IT	c23	49.3	$50.6$	$26.0$	$49.2$	46.9
LLaVA-NeXT FAQ-IT	c40	$44.1$	$46.2$	$21.2$	$46.1$	$42.9$
	original	$49.1$	51.9	$26.9$	50.9	$43.7$

Table 5: In-domain MCQ accuracy under different compressions on FF++. Symbol definitions follow Table 3 .

Model	Condition	Edges	Color	Light	Texture	Pixel	Shapes
Qwen2.5-VL	${\phi}$	11.3	8.4	4.5	8.0	13.2	9.0
	${FAQ-IT^{\spadesuit}}$	13.5	9.1	7.9	10.9	14.3	12.1
	${FAQ-IT}$	52.9	45.9	46.2	43.1	50.2	44.3
LLaVA-NeXT	${\phi}$	9.8	11.1	14.9	16.6	10.7	11.4
	${FAQ-IT^{\spadesuit}}$	12.8	13.9	18.8	16.4	14.3	13.4
	${FAQ-IT}$	45.7	42.1	47.9	42.1	43.9	46.8

Table 6: MCQ performance for different artifact types in deepfake videos. Symbol definitions follow Table 3 .

Training Setup		CDF		DFo		WDF
Model	Training Data	Acc	F1	Acc	F1	Acc	F1
Qwen2.5-VL	${\phi}$	17.9	13.8	14.5	11.1	10.8	8.9
	${FAQ-IT^{\spadesuit}}$	49.1	59.1	54.7	57.9	40.9	54.8
	${FAQ-IT}$	73.3	71.0	78.6	70.1	65.9	59.3
LLaVA-NeXT	${\phi}$	12.3	10.2	10.5	9.5	8.7	7.6
	${FAQ-IT}^{\spadesuit}$	47.2	49.0	52.3	54.8	39.8	49.6

Table 7: Cross-dataset generalization accuracy (%) and F1-score (%) on CDF [ 22 ] , DFo [ 15 ] , and WDF [ 48 ] with different training settings. Symbol definitions follow Table 3 .

Figure 4: Effect of frame sampling strategies.

Exp 1: Robustness		Exp 2: Strategy		Exp 3: Open-Ended QA
Cond.	Acc	Strategy	Acc	Model	Score
Original	54.4%	Baseline	30.3%	Random	$\approx$ 0.0%
Bright.	$52.8\%_{\textbf{{\color[rgb]{1,0,0}-1.6}}}$	Multi-Stage	$35.6\%_{\text{{{\color[rgb]{0,0.3906,0}+5.3}}}}$	GPT-4o	13.2%
Compr.	$49.8\%_{\textbf{{\color[rgb]{1,0,0}-4.6}}}$	Lvl 3 Only	$37.8\%_{\text{{{\color[rgb]{0,0.3906,0}+7.5}}}}$	Baseline	18.5%
Blur	$46.2\%_{\textbf{{{\color[rgb]{1,0,0}-8.2}}}}$	Mixed	$\mathbf{54.4\%_{\text{{{\color[rgb]{0,0.3906,0}+24.1}}}}}$	Ours	$\mathbf{49.1\%}$

Table R1 : New Exp on LLaVA-NeXT. Left: Robustness. Mid: Training Strategy. Right: Open-Ended QA.

Figure 5: Distribution of Videos across Face Detection Confidence Metrics and the Filtering Boundary. Each point represents a video from the original set. The filtering boundary (dashed light blue lines) separates the retained high-quality samples (blue points) from the removed low-quality samples (light purple points), demonstrating the rigorous exclusion of videos with unstable or weak face detections.

Overall Benchmark Accuracy (%)
Model	Data Ratio	Level 1	Level 2	Level 3	Average
Qwen2.5-VL	100% (Raw)	90.3	40.1	23.5	51.3
	90% (Ours)	89.9	41.4	25.8	52.4
	80%	89.7	41.0	24.9	51.9
	70%	88.1	39.5	23.2	50.3
	60%	85.4	36.8	20.1	47.4

Table 9: Ablation study on data filtering ratios. The table reports (Top) Overall benchmark accuracy including real cases; and (Bottom) In-domain and Cross-domain tasks. 90% corresponds to our proposed setting.

Video Input Preprocessing
Configuration	Value
Max Pixels per Video	$50176$
Max Frames per Video	$36$
Input Video Frame Resolution	$448\times 448$
Training / Fine-tuning Configuration
Total Training Epochs	$1$

Table 10: Hyperparameters for Training and Inference

Figure 6: Overview of our custom-developed video annotation interface. The platform is designed to capture fine-grained spatiotemporal forgery traces. The central player allows annotators to mark dynamic artifacts (indicated by red dots) in real-time, recording precise $(x,y,t)$ coordinates. The right panel displays the sequential list of annotated clicks with corresponding frame previews, while the bottom text field captures a detailed natural language description of the observed manipulation anomalies.

Figure 7: User interface of our verification platform for Level 1 (Facial Perception). To ensure high data quality, validators execute a dual-check protocol: (1) Answer Verification, ensuring the selected option (e.g., “soft or blurred edges”) accurately reflects the visual artifacts observed in the video; and (2) Text Quality Assessment, confirming that the phrasing is unambiguous. The “Exception” panel at the bottom allows validators to flag and discard samples with severe hallucinations or low-quality generation.

Figure 8: The verification interface for Level 2 (Temporal Deepfake Grounding). In contrast to Level 1, this stage requires rigorous temporal verification. Validators must manually use the “Mark Start” and “Mark End” buttons to define the precise boundaries of the artifact, ensuring the timestamps in the generated QA match the actual video content. Additional checks for Artifact Presence, Region Correctness, and Distractor Validity are implemented to prevent ambiguous or incorrectly grounded samples.

Figure 9: The verification interface for Level 3 (Forensic Reasoning). Unlike lower-level tasks, this stage requires validators to perform a holistic assessment of the entire video sequence. Validators are tasked with: (1) independently selecting the most accurate description among complex options to confirm the ground truth’s validity; and (2) evaluating Distractor Quality, ensuring that incorrect options are “plausible and non-trivial” to guarantee the task effectively challenges the model’s reasoning capabilities.

Explore similar work

Unleashing Vision-Language Semantics for Deepfake Video Detection
Mar 25, 2026Jiawen Zhu, Yunqi Miao, Xueyi Zhang +2Deepfake Detection Vision-Language Alignment
Singapore Management University, Singapore · The University of Warwick, UK · Nanyang Technological University, Singapore +1
Multi-Agent Forensic Reasoning for Generalizable Deepfake Video Detection
Aug 7, 2026Xuechao Zou, Shun Zhang, Kai Li +6Explainable Deepfake Detection Deepfake Detection
Beijing Jiaotong University · Tsinghua University · Ant Group
Interpretable Deepfake Detection in Videos via Explicit Forensic Features and Temporal Modeling
Oct 2, 2026Chahira Benhama, Mohand Saïd Allili, Assia HamadeneDeepfake Detection Explainable Deepfake Detection
LARIVIA laboratory Dep. of Computer science and Eng. University of Quebec in Outaouais Gatineau, QC, Canada · LARIVIA laboratory and Mixed Research Unit INRS-UQO University of Quebec in Outaouais Gatineau, QC, Canada

Mar 25, 2026J/K move · Enter open · S save

Unleashing Vision-Language Semantics for Deepfake Video Detection

Jiawen Zhu, Yunqi Miao, Xueyi Zhang, Jiankang Deng+1

Singapore Management University, Singapore · The University of Warwick, UK · Nanyang Technological University, Singapore+1

Deepfake Detection Vision-Language Alignment

Open paper AskPDF arXiv

Recent Deepfake Video Detection (DFD) studies have demonstrated that pre-trained Vision-Language Models (VLMs) such as CLIP exhibit strong generalization capabilities in detecting artifacts across different identities. However, existing approaches focus on leveraging visual features only, overlooking their most distinctive strength -- the rich vision-language semantics embedded in the latent space. We propose VLAForge, a novel DFD framework that unleashes the potential of such cross-modal semantics to enhance model's discriminability in deepfake detection. This work i) enhances the visual perception of VLM through a ForgePerceiver, which acts as an independent learner to capture diverse, subtle forgery cues both granularly and holistically, while preserving the pretrained Vision-Language Alignment (VLA) knowledge, and ii) provides a complementary discriminative cue -- Identity-Aware VLA score, derived by coupling cross-modal semantics with the forgery cues learned by ForgePerceiver. Notably, the VLA score is augmented by an identity prior-informed text prompting to capture authenticity cues tailored to each identity, thereby enabling more discriminative cross-modal semantics. Comprehensive experiments on video DFD benchmarks, including classical face-swapping forgeries and recent full-face generation forgeries, demonstrate that our VLAForge substantially outperforms state-of-the-art methods at both frame and video levels. Code is available at https://github.com/mala-lab/VLAForge.

Abstract

Explore similar work

Unleashing Vision-Language Semantics for Deepfake Video Detection

Multi-Agent Forensic Reasoning for Generalizable Deepfake Video Detection

Interpretable Deepfake Detection in Videos via Explicit Forensic Features and Temporal Modeling