GenVideoLens: Where LVLMs Fall Short in AI-Generated Video Detection?
Authors: Yueying Zou, Pei Pei Li, Zekun Li, Xinyu Guo, Xing Cui, Huaibo Huang, Ran He
Organizations: Beijing University of Posts and Telecommunications, Beijing, China · University of California, Santa Barbara, CA, USA · Center for Research on Intelligent Perception and Computing, NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China
In recent years, AI-generated videos have become increasingly realistic and sophisticated. Meanwhile, Large Vision-Language Models (LVLMs) have shown strong potential for detecting such content. However, existing evaluation protocols largely treat the task as a binary classification problem and rely on coarse-grained metrics such as overall accuracy, providing limited insight into where LVLMs succeed or fail. To address this limitation, we introduce GenVideoLens, a fine-grained benchmark that enables dimension-wise evaluation of LVLM capabilities in AI-generated video detection. The benchmark contains 400 highly deceptive AI-generated videos and 100 real videos, annotated by experts across 15 authenticity dimensions covering perceptual, optical, physical, and temporal cues. We evaluate eleven representative LVLMs on this benchmark. Our analysis reveals a pronounced dimensional imbalance. While LVLMs perform relatively well on perceptual cues, they struggle with optical consistency, physical interactions, and temporal-causal reasoning. Model performance also varies substantially across dimensions, with smaller open-source models sometimes outperforming stronger proprietary models on specific authenticity cues. Temporal perturbation experiments further show that current LVLMs make limited use of temporal information. Overall, GenVideoLens provides diagnostic insights into LVLM behavior, revealing key capability gaps and offering guidance for improving future AI-generated video detection systems.
Figures & tables
Figure 1: Overview of GenVideoLens. Existing evaluations treat AI-generated video detection as a binary classification task, providing limited insight into the capabilities of LVLMs. GenVideoLens addresses this limitation by decomposing video authenticity into 15 fine-grained dimensions across frame-level and video-level analyses, enabling diagnostic evaluation that reveals where LVLMs fail and guides for future improvement.
Figure 2: Visualization of the 15 authenticity dimensions in GenVideoLens. The dimensions are organized into frame-level and video-level categories, each illustrated with representative visual examples from the dataset to highlight the corresponding authenticity cues.
Figure 3: Dataset statistics of GenVideoLens. (a) Distribution of annotated artifacts across frame-level and video-level authenticity dimensions. (b) Correlation heatmap of the 15 authenticity dimensions computed from ground-truth annotations. (c) Word cloud of senmantic diversity in GenVideoLens
Figure 4: Overview of the GenVideoLens dataset construction and annotation pipeline. The process consists of data collection, dataset filtering, and human annotation.
Model
Tex
Edg
Mat
Loc
Dep
Txt
Com
Chr
Lig
Ref
Int
Mov
Tmp
Intx
ComS
Qwen3-VL-8B
0.34
0.29
0.28
0.22
0.25
0.30
0.32
0.07
0.15
0.23
0.27
0.12
0.13
0.19
0.42
Qwen2.5-VL-7B
0.48
0.29
0.31
0.50
0.18
0.09
0.19
0.21
0.09
0.10
0.04
0.03
0.10
0.26
0.35
InternVL3.5-8B
0.43
0.11
0.25
0.05
0.19
0.08
0.32
0.13
0.19
0.16
0.07
0.14
0.08
0.36
0.43
InternVL3-8B
0.41
0.26
0.31
0.02
0.24
0.28
0.29
0.00
0.08
0.08
0.27
0.17
0.15
0.21
0.42
Video-LLaVA
0.09
0.39
0.11
0.43
0.21
0.00
0.08
0.11
0.06
0.14
0.04
0.00
0.13
0.27
0.20
LLaVA-NeXT-Video
0.01
0.03
0.02
0.58
0.00
0.09
0.07
0.06
0.00
0.13
0.00
0.00
0.02
0.04
0.22
Table 1: Performance of LVLMs across 15 dimensions on the GenVideoLens ( F1forged ). Higher values indicate better alignment with human judgments. The best and second-best results are highlighted in yellow and light yellow, respectively. Abbreviations: Tex (texture authenticity), Edg (edge and contour clarity), Mat (material and physical consistency), Loc (local forgery artifacts), Dep (depth-of-field and background reasonableness), Txt (text and symbol readability), Int (inter-frame consistency and repetition), Com (naturalness of composition), Chr (chromatic consistency), Lig (lighting and shadow consistency), Mov (facial and movement continuity), Tmp (temporal logic), Intx (physical plausibility of interactions), Ref (correctness of reflection, refraction, and parallax), and ComS (real-world logic consistency).
F1forged
8 Frames
All Frames
ordered
random
repeated
ordered
random
repeated
Inter-frame Consistency
Qwen3-VL-8B
0.27
0.26
0.27
0.07
0.07
0.08
InternVL3.5-8B
0.07
0.07
0.07
0.17
0.17
0.17
Facial and Movement Continuity
Qwen3-VL-8B
0.12
0.12
0.10
0.05
0.06
0.04
Table 2: Temporal sensitivity under frame-order perturbations. Model performance ( F1forged ) under different frame-order configurations. The results of using sparse 8-frame inputs and dense all-frame inputs are shown.
Figure 5: Visualization of the inputs for our physical-causal reasoning experiment. (a)(b) Consecutive frames showing a basketball approaching the hoop. (c) Frame difference map reveals the motion trajectory, and (d) optical-flow map captures directional velocity fields. Results shown with Intern3.5VL-8B.
Model
Acc all
F1real
F1forged
Prec
Qwen3-VL-8B
0.51
0.66
0.11
0.51
InternVL3.5-8B
0.49
0.63
0.19
0.50
Table 3: Performance of Qwen3-VL-8B and InternVL3.5-8B on physical and causal reasoning. Metrics include overall accuracy (Acc all ), F 1real , F 1forged , and precision (Prec).
Figure 6: Cross-Dimension Prompting Results. Analysis of model behavior when multiple authenticity dimensions are evaluated within a single prompt. (a),(c) Heatmaps of ΔF1forged across dimensions, computed as the performance difference between multi-dimensional prompting and the default single-dimension evaluation. (b),(d) Collapse rate, measuring how often the model outputs identical binary decisions across all evaluated dimensions. The top row shows results for Qwen3-VL-8B, and the bottom row shows results for InternVL3.5-8B.
Model
Zero-shot
Dimension-Guided
Acc all
Acc real
Acc forged
Acc all
Acc real
Acc forged
Qwen3-VL-8B
0.64
0.79
0.60
0.65
0.89
0.60
Qwen2.5-VL-7B
0.58
0.91
0.49
0.80
0.00
1.00
Intern3.5-VL-8B
0.42
0.96
0.28
0.36
0.94
0.22
Video-LLaVA-7B
0.29
0.96
0.12
0.26
0.92
0.10
LLaVA-Next-Video-7B
0.30
0.88
0.16
0.79
0.09
0.96
Table 4: Detection accuracy on GenVideoLens . Comparison between Zero-shot prompting and Dimension-Guided Prompting. Acc all , Acc real , and Acc forged denote overall, real-video, and AI-generated video accuracy.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Scene category distribution in the GenVideoLens dataset. (a) Real videos. (b) AI-generated videos.
Dimension
Yes
No
N/A
Texture Authenticity
215
185
0
Edge and Contour Clarity
272
128
0
Material Consistency
293
104
3
Local Forgery Artifacts
198
202
0
Depth-of-field Reasonableness
364
34
2
Text and Symbol Readability
38
22
340
Appendix
Table 5: Label distribution across the 15 authenticity dimensions.
Figure 8: Examples of authenticity violations in GenVideoLens dimensions. (a) A synthetic frame exhibiting abnormal texture patterns and inconsistent edge structures, corresponding to violations of Texture Authenticity and Edge & Contour Clarity . (b) A video sequence showing unnatural eye motion and physically implausible interactions, violating Facial & Movement Continuity and Physical Plausibility of Interactions .
Figure 9: Annotation interface of the GenVideoLens platform. Top: The main annotation website used for dimension-wise labeling. Bottom: The expert review interface for conflict-bucket samples in Round-2, where reviewers can inspect previous annotation results before making the final decision.
Figure 10: Examples used in annotator training to illustrate dimension-specific authenticity violations. Different authenticity dimensions capture distinct types of inconsistencies, enabling annotators to evaluate each dimension independently.
Figure 11: Full prompt used in GenVideoLens 15-dimensional authenticity evaluation.
Figure 12: An illustrative example of Texture Authenticity, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 13: An illustrative example of Edge and Contour Clarity, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 14: An illustrative example of Material and Physical Consistency, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 15: An illustrative example of Local Forgery Artifacts, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 16: An illustrative example of Depth of Field and Background Reasonableness, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 17: An illustrative example of Text and Symbol Readability, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 18: An illustrative example of Inter-frame Consistency/Repetition, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 19: An illustrative example of Naturalness of Composition, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 20: An illustrative example of Chromatic Consistency, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 21: An illustrative example of Lighting and Shadow Consistency, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 22: An illustrative example of Facial and Movement Continuity, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 23: An illustrative example of Temporal and Scene Logic, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 24: An illustrative example of Physical Plausibility of Interactions, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 25: An illustrative example of Correctness of Reflection, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Figure 26: An illustrative example of Real-world Consistency, comparing the reasoning outputs of Qwen3VL-8B and GPT-5.
Recent video generative models have greatly improved the realism of AI-generated videos, yet their outputs still exhibit artifacts such as temporal inconsistencies, structural distortions, and semantic incoherence. While Multimodal Large Language Models (MLLMs) show strong visual understanding capabilities, their ability to perceive and reason about such artifacts remains unclear. Existing benchmarks often lack systematic evaluation of artifact-aware perception and fine-grained diagnostic reasoning, especially across diverse AI-generated video domains beyond photorealistic content. To address this gap, we introduce Artifact-Bench, a comprehensive benchmark for evaluating MLLMs on AI-generated video artifact detection and analysis. We first establish a three-level hierarchical taxonomy of realism artifacts, covering photorealistic, animated, and CG-style videos. Based on this taxonomy, Artifact-Bench defines three complementary tasks: real vs. AI-generated video classification, pairwise realism comparison, and fine-grained artifact identification. Experiments on 19 leading MLLMs reveal substantial limitations in artifact perception and reasoning, with many models approaching random or even below-random performance in challenging settings. We further observe significant misalignment between MLLM judgments and human perceptual preferences, highlighting their limited reliability as general evaluators for AI-generated video realism.
The rapid advancement of AI-generated video poses challenges to digital authenticity and security. Current detection methods, often trained on specific datasets, struggle with the ever-evolving landscape of generative techniques and unseen manipulations. We introduce a framework leveraging Vision Language Models (VLMs) for robust AI-generated video detection. Our approach equips the VLM with the ability to reason about video content and use external tools to identify subtle inconsistencies, mirroring human system 2 thinking. Our self-evolving VLM dynamically selects and composes appropriate tools, enhancing its ability to generalize to novel video generation techniques. The modular design promotes interpretability, allowing for a clearer understanding of VLM's decision-making process. To evaluate, we establish the first benchmark VidForensic containing 1.4k+ high-quality AI-generated videos across eight generative models. Experiments show that REVEAL improves F1 scores by 9.1% to 30.2% over top baselines across our datasets for VLMs, notably for GPT-4o, Gemini 1.5 pro, and QWen-VL-Max, and Llava-One-Vision-7B. While open-world AI-video detection remains an open challenge, our results indicate that existing methods fail primarily because they lack tool-enabled, higher-order reasoning.
The evolving realism of AI-generated Videos (AIGC-V) is rapidly rendering traditional artifact-centric detection insufficient, necessitating a paradigm shift from low-level inspection to high-level semantic verification. This paper presents a comprehensive survey of AIGC-V detection, reframing the task as Factual Fidelity Verification, which asks whether the events, entities, and physical processes depicted in a video are consistent with real-world facts. To systematize this rapidly evolving field, we propose a Vision-Language Dual-View taxonomy that organizes existing methods into a hierarchical, four-layer landscape, spanning intrinsic cue analysis, spatiotemporal consistency modeling, cross-modal consistency reasoning, and language-guided world-level reasoning. This dual-view framing highlights a fundamental transition from artifact matching in traditional deepfake detection to evidence-based semantic verification enabled by vision-language models and agentic reasoning pipelines. Based on a systematic review of 221 works, we synthesize AIGC-V generation paradigms, survey the landscape of detection methods, and review evaluation metrics and benchmarks in line with proposed views. Finally, we discuss current challenges and identify promising directions toward robust, explainable, and trustworthy detection.
Dylan Xinming Hou, Juntian Zhang, Xu Gu +5
MBZUAI · Gaoling School of Artificial Intelligence, Renmin University of China · Harvard University