GenVideoLens: Where LVLMs Fall Short in AI-Generated Video Detection?

Yueying Zou; Pei Pei Li; Zekun Li; Xinyu Guo; Xing Cui; Huaibo Huang; Ran He

cs.CVMar 19, 2026

GenVideoLens: Where LVLMs Fall Short in AI-Generated Video Detection?

Authors: Yueying Zou, Pei Pei Li, Zekun Li, Xinyu Guo, Xing Cui, Huaibo Huang, Ran He

Organizations: Beijing University of Posts and Telecommunications, Beijing, China · University of California, Santa Barbara, CA, USA · Center for Research on Intelligent Perception and Computing, NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China

Abstract

In recent years, AI-generated videos have become increasingly realistic and sophisticated. Meanwhile, Large Vision-Language Models (LVLMs) have shown strong potential for detecting such content. However, existing evaluation protocols largely treat the task as a binary classification problem and rely on coarse-grained metrics such as overall accuracy, providing limited insight into where LVLMs succeed or fail. To address this limitation, we introduce GenVideoLens, a fine-grained benchmark that enables dimension-wise evaluation of LVLM capabilities in AI-generated video detection. The benchmark contains 400 highly deceptive AI-generated videos and 100 real videos, annotated by experts across 15 authenticity dimensions covering perceptual, optical, physical, and temporal cues. We evaluate eleven representative LVLMs on this benchmark. Our analysis reveals a pronounced dimensional imbalance. While LVLMs perform relatively well on perceptual cues, they struggle with optical consistency, physical interactions, and temporal-causal reasoning. Model performance also varies substantially across dimensions, with smaller open-source models sometimes outperforming stronger proprietary models on specific authenticity cues. Temporal perturbation experiments further show that current LVLMs make limited use of temporal information. Overall, GenVideoLens provides diagnostic insights into LVLM behavior, revealing key capability gaps and offering guidance for improving future AI-generated video detection systems.

Figures & tables

Figure 1: Overview of GenVideoLens. Existing evaluations treat AI-generated video detection as a binary classification task, providing limited insight into the capabilities of LVLMs. GenVideoLens addresses this limitation by decomposing video authenticity into 15 fine-grained dimensions across frame-level and video-level analyses, enabling diagnostic evaluation that reveals where LVLMs fail and guides for future improvement.

Figure 2: Visualization of the 15 authenticity dimensions in GenVideoLens. The dimensions are organized into frame-level and video-level categories, each illustrated with representative visual examples from the dataset to highlight the corresponding authenticity cues.

Figure 3: Dataset statistics of GenVideoLens. (a) Distribution of annotated artifacts across frame-level and video-level authenticity dimensions. (b) Correlation heatmap of the 15 authenticity dimensions computed from ground-truth annotations. (c) Word cloud of senmantic diversity in GenVideoLens

Figure 4: Overview of the GenVideoLens dataset construction and annotation pipeline. The process consists of data collection, dataset filtering, and human annotation.

Model	Tex	Edg	Mat	Loc	Dep	Txt	Com	Chr	Lig	Ref	Int	Mov	Tmp	Intx	ComS
Qwen3-VL-8B	0.34	0.29	0.28	0.22	0.25	0.30	0.32	0.07	0.15	0.23	0.27	0.12	0.13	0.19	0.42
Qwen2.5-VL-7B	0.48	0.29	0.31	0.50	0.18	0.09	0.19	0.21	0.09	0.10	0.04	0.03	0.10	0.26	0.35
InternVL3.5-8B	0.43	0.11	0.25	0.05	0.19	0.08	0.32	0.13	0.19	0.16	0.07	0.14	0.08	0.36	0.43
InternVL3-8B	0.41	0.26	0.31	0.02	0.24	0.28	0.29	0.00	0.08	0.08	0.27	0.17	0.15	0.21	0.42
Video-LLaVA	0.09	0.39	0.11	0.43	0.21	0.00	0.08	0.11	0.06	0.14	0.04	0.00	0.13	0.27	0.20
LLaVA-NeXT-Video	0.01	0.03	0.02	0.58	0.00	0.09	0.07	0.06	0.00	0.13	0.00	0.00	0.02	0.04	0.22

Table 1: Performance of LVLMs across 15 dimensions on the GenVideoLens (

\text{F}_{1}^{\text{forged}}

). Higher values indicate better alignment with human judgments. The best and second-best results are highlighted in yellow and light yellow, respectively. Abbreviations: Tex (texture authenticity), Edg (edge and contour clarity), Mat (material and physical consistency), Loc (local forgery artifacts), Dep (depth-of-field and background reasonableness), Txt (text and symbol readability), Int (inter-frame consistency and repetition), Com (naturalness of composition), Chr (chromatic consistency), Lig (lighting and shadow consistency), Mov (facial and movement continuity), Tmp (temporal logic), Intx (physical plausibility of interactions), Ref (correctness of reflection, refraction, and parallax), and ComS (real-world logic consistency).

Inter-frame Consistency
$\text{F}_{1}^{\text{forged}}$	8 Frames			All Frames
	ordered	random	repeated	ordered	random	repeated
Qwen3-VL-8B	0.27	0.26	0.27	0.07	0.07	0.08
InternVL3.5-8B	0.07	0.07	0.07	0.17	0.17	0.17
Facial and Movement Continuity
Qwen3-VL-8B	0.12	0.12	0.10	0.05	0.06	0.04

Table 2: Temporal sensitivity under frame-order perturbations. Model performance (

\text{F}_{1}^{\text{forged}}

) under different frame-order configurations. The results of using sparse 8-frame inputs and dense all-frame inputs are shown.

Figure 5: Visualization of the inputs for our physical-causal reasoning experiment. (a)(b) Consecutive frames showing a basketball approaching the hoop. (c) Frame difference map reveals the motion trajectory, and (d) optical-flow map captures directional velocity fields. Results shown with Intern3.5VL-8B.

Model	Acc ${}^{\text{all}}$	$\text{F}_{1}^{\text{real}}$	$\text{F}_{1}^{\text{forged}}$	Prec
Qwen3-VL-8B	0.51	0.66	0.11	0.51
InternVL3.5-8B	0.49	0.63	0.19	0.50

Table 3: Performance of Qwen3-VL-8B and InternVL3.5-8B on physical and causal reasoning. Metrics include overall accuracy (Acc

{}^{\text{all}}

), F

{}_{1}^{\text{real}}

, F

{}_{1}^{\text{forged}}

, and precision (Prec).

Figure 6: Cross-Dimension Prompting Results. Analysis of model behavior when multiple authenticity dimensions are evaluated within a single prompt. (a),(c) Heatmaps of $\Delta F_{1}^{\text{forged}}$ across dimensions, computed as the performance difference between multi-dimensional prompting and the default single-dimension evaluation. (b),(d) Collapse rate, measuring how often the model outputs identical binary decisions across all evaluated dimensions. The top row shows results for Qwen3-VL-8B, and the bottom row shows results for InternVL3.5-8B.

Model	Zero-shot			Dimension-Guided
Model	Acc ${}^{\text{all}}$	Acc ${}^{\text{real}}$	Acc ${}^{\text{forged}}$	Acc ${}^{\text{all}}$	Acc ${}^{\text{real}}$	Acc ${}^{\text{forged}}$
Qwen3-VL-8B	$0.64$	$0.79$	$0.60$	0.65	$0.89$	$0.60$
Qwen2.5-VL-7B	$0.58$	$0.91$	$0.49$	0.80	$0.00$	$1.00$
Intern3.5-VL-8B	$0.42$	$0.96$	$0.28$	$0.36$	$0.94$	$0.22$
Video-LLaVA-7B	$0.29$	$0.96$	$0.12$	$0.26$	$0.92$	$0.10$
LLaVA-Next-Video-7B	$0.30$	$0.88$	$0.16$	0.79	$0.09$	$0.96$

Table 4: Detection accuracy on GenVideoLens . Comparison between Zero-shot prompting and Dimension-Guided Prompting. Acc

{}^{\text{all}}

, Acc

{}^{\text{real}}

, and Acc

{}^{\text{forged}}

denote overall, real-video, and AI-generated video accuracy.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Scene category distribution in the GenVideoLens dataset. (a) Real videos. (b) AI-generated videos.

Dimension	Yes	No	N/A
Texture Authenticity	215	185	0
Edge and Contour Clarity	272	128	0
Material Consistency	293	104	3
Local Forgery Artifacts	198	202	0
Depth-of-field Reasonableness	364	34	2
Text and Symbol Readability	38	22	340

Appendix

Table 5: Label distribution across the 15 authenticity dimensions.

Figure 8: Examples of authenticity violations in GenVideoLens dimensions. (a) A synthetic frame exhibiting abnormal texture patterns and inconsistent edge structures, corresponding to violations of Texture Authenticity and Edge & Contour Clarity . (b) A video sequence showing unnatural eye motion and physically implausible interactions, violating Facial & Movement Continuity and Physical Plausibility of Interactions .

Panel of Figure 9 — Figure 9: Annotation interface of the GenVideoLens platform. Top: The main annotation website used for dimension-wise labeling. Bottom: The expert review interface for conflict-bucket samples in Round-2, where reviewers can inspect previous annotation results before making the final decision.

Figure 10: Examples used in annotator training to illustrate dimension-specific authenticity violations. Different authenticity dimensions capture distinct types of inconsistencies, enabling annotators to evaluate each dimension independently.

Panel of Figure 11 — Figure 11: Full prompt used in GenVideoLens 15-dimensional authenticity evaluation.

Figure 12: An illustrative example of Texture Authenticity, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 13: An illustrative example of Edge and Contour Clarity, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 14: An illustrative example of Material and Physical Consistency, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 15: An illustrative example of Local Forgery Artifacts, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 16: An illustrative example of Depth of Field and Background Reasonableness, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 17: An illustrative example of Text and Symbol Readability, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 18: An illustrative example of Inter-frame Consistency/Repetition, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 19: An illustrative example of Naturalness of Composition, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 20: An illustrative example of Chromatic Consistency, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 21: An illustrative example of Lighting and Shadow Consistency, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 22: An illustrative example of Facial and Movement Continuity, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 23: An illustrative example of Temporal and Scene Logic, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 24: An illustrative example of Physical Plausibility of Interactions, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 25: An illustrative example of Correctness of Reflection, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Figure 26: An illustrative example of Real-world Consistency, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.

Synthetic Video Detection Benchmarks

See all 14 papers

Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videos
May 18, 2026Yuqi Tang, Yang Shi, Zhuoran Zhang +21VLM Evaluation Video Generation Evaluation
HKUST(GZ) · PKU · Kling Team +10
REVEAL: Robust Evolution of Vision-Language Models for Explainable AI-Video Detection
Feb 20, 2025Yun-Yun Tsai, Qingyuan Liu, Ruijian Zha +4Tool Use in VLMs Vision-Language Models
Columbia University · Rutgers University
Detecting AI-Generated Video: A Vision-Language Dual-View Survey
Jul 12, 2026Dylan Xinming Hou, Juntian Zhang, Xu Gu +5Factual Consistency Evaluation Deepfake Detection
MBZUAI · Gaoling School of Artificial Intelligence, Renmin University of China · Harvard University

May 18, 2026J/K move · Enter open · S save

Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videos

Yuqi Tang, Yang Shi, Zhuoran Zhang, Qixun Wang+20

HKUST(GZ) · PKU · Kling Team+10

VLM Evaluation Video Generation Evaluation

Open paper AskPDF arXiv

Recent video generative models have greatly improved the realism of AI-generated videos, yet their outputs still exhibit artifacts such as temporal inconsistencies, structural distortions, and semantic incoherence. While Multimodal Large Language Models (MLLMs) show strong visual understanding capabilities, their ability to perceive and reason about such artifacts remains unclear. Existing benchmarks often lack systematic evaluation of artifact-aware perception and fine-grained diagnostic reasoning, especially across diverse AI-generated video domains beyond photorealistic content. To address this gap, we introduce Artifact-Bench, a comprehensive benchmark for evaluating MLLMs on AI-generated video artifact detection and analysis. We first establish a three-level hierarchical taxonomy of realism artifacts, covering photorealistic, animated, and CG-style videos. Based on this taxonomy, Artifact-Bench defines three complementary tasks: real vs. AI-generated video classification, pairwise realism comparison, and fine-grained artifact identification. Experiments on 19 leading MLLMs reveal substantial limitations in artifact perception and reasoning, with many models approaching random or even below-random performance in challenging settings. We further observe significant misalignment between MLLM judgments and human perceptual preferences, highlighting their limited reliability as general evaluators for AI-generated video realism.

Abstract

Synthetic Video Detection Benchmarks

Explore similar work

Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videos

REVEAL: Robust Evolution of Vision-Language Models for Explainable AI-Video Detection

Detecting AI-Generated Video: A Vision-Language Dual-View Survey