Video Generation Evaluation

Overview

The field measures whether generated videos look good and, crucially, whether they follow prompts and depict coherent events over time. It combines perceptual metrics such as FVD and human preference ratings with benchmarks for motion, physics, geometry, long-form continuity, and audio-video alignment. Recent approaches use task-specific rubrics, vision-language model judges, and interpretable diagnostics to assess failures that aggregate scores can hide. A central challenge is that automated evaluators may disagree with people, while visual realism often fails to reflect physical reasoning, causal consistency, or successful task completion.

Latest papers

CardsList

192 papers34 papers in the last four weeks, up 113% on the four weeks before. 0.2% of all new papers.

  1. Evaluating Visual Realism of AI-Generated Videos with Eye Tracking

    Oct 28, 2026Danqing Shi, Lan Jiang, Jake Lyons +1Video Generation EvaluationEye Tracking

  2. More Memory, Better Videos? A Controlled Study of Visual Memory in Long-Video Generation

    Oct 26, 2026Yuzhuo Chen, Huafeng Shi, Xinyu Wang +6Memory-Augmented Video GenerationRetrieval-Augmented Video Generation

  3. RGraph-Eval: Task-Grounded Scene Graphs for Evaluating Robot Video Generation

    Oct 22, 2026Aleksandr Lazarev, Sofia Zimina, Roman Kuznetsov +3Video Generation EvaluationVideo SGG

  4. RoboJudge: Multimodal Language Models as Judges for Embodied Video Generation

    Oct 20, 2026Siyuan Ma, Yu Qi, Cheng Ye +8Vision-Language Model-as-a-JudgeVideo Generation Evaluation

  5. PVBench: Quantifying Physical Plausibility in Video Generation with Object-Level Physical Measurements

    Oct 19, 2026Kaiwei Chen, Moran Li, Zhenye Gan +5Physical Consistency in Video GenerationVideo Generation Evaluation

  6. AVC-Reward: Event-Centric Audio-Video Consistency Evaluation

    Oct 15, 2026Yuruo Wu, Kaihang Pan, Kai Shen +6Video Generation EvaluationAudio-Video Generation

  7. Beyond Source Prompts: Towards Visually Grounded AI-Generated Video Quality Assessment

    Oct 15, 2026Kehao Zhou, Yixuan Li, Yuxuan Jiang +1Video Quality AssessmentVideo Generation Evaluation

  8. Orbia: Do Generated Videos Form a Consistent 3D World?

    Oct 14, 2026Dongyue Lu, Guian Fang, Hur Lim +20Geometric Consistency in Video GenerationWorld Model Evaluation

  9. PhyMetric: Diagnosing Physical Plausibility in Text-to-Video Generation via Scene-Level QA

    Oct 14, 2026Siwei Meng, Xuerou Cai, Rui Hu +1Video Generation EvaluationPhysical Consistency in Video Generation

  10. IsoVRBench: Video Reward Modeling Without Human Feedback

    Oct 12, 2026Shuyue Stella Li, Jacqueline He, Jiahui Chen +5Reward ModelingVideo Generation Evaluation

  11. A Plausible Future Is Not a Causal Future: Can Video Models Generate How the World Unfolds?

    Oct 11, 2026Jialiang Li, Zhengning Xu, Zongyang Ma +1Causal Video GenerationVideo Generation

  12. EmoTrace-Bench: Diagnosing Emotional Expression in Text-to-Video Generation through Controlled Cues and Intensity

    Oct 11, 2026Yeheng Wang, Liudan Shi, Yi Zheng +5T2V GenerationVideo Generation Evaluation

  13. TextVBench: Evaluating Text Rendering and Preservation in Video Generation

    Oct 9, 2026Ruiqiang Zhang, Hengyi Wang, Chang Liu +3Video Generation EvaluationT2V Generation

  14. Towards Unified Evaluation of Prompt Enhancers for Video Generation

    Oct 8, 2026Yawen Shao, Yubo Zhu, Ziyun Dai +21Video Generation EvaluationVideo-Reference Video Generation

  15. Expression-Diverse References for Identity-Preserving Video Generation

    Oct 8, 2026Tianwen Fu, Wenbin Teng, Gonglin Chen +3Identity-Preserving Video GenerationVideo Generation Evaluation

  16. What Does Fréchet Distance Measure? A Directional Decomposition

    Oct 4, 2026Yunghee Lee, Jaeyeon KimImage Generation EvaluationOptimal Transport

  17. How Does Geometry Enter Generated Motion?

    Oct 4, 2026Weihan Li, Junhao Wu, Yuhan Song +2Physical Consistency in Video GenerationImage-to-Video Generation

  18. World Embedding Benchmark

    Oct 2, 2026Yiqi Liu, Ruifeng Yuan, Yang Wang +7Physical Reasoning in VideoVideo Representation Learning

  19. DiVid: Diagnosing Dimension-Specific Diversity Collapse in Video Generation Models

    Oct 1, 2026Huanran Hu, Zihui Ren, Dingyi Yang +4Video GenerationControllable Video Generation

  20. VTR-Bench: A Systematic Benchmark for Evaluating Visual Text Rendering in Video Generation

    Oct 1, 2026Yu Huang, Jungang Li, Zhiyuan Wang +8Agentic Video GenerationVideo Generation

  21. PhysicsLENS: Diagnosing Physical Property Blindness in Video Generation Models

    Oct 1, 2026Isaiah Milkey, Som Sagar, Aditya Taparia +3Physical Consistency in Video GenerationPhysical Reasoning in Video

  22. Ego2Act: Evaluating Goal-Directed Manipulation in Egocentric Video Generation

    Oct 1, 2026Patrick Amadeus Irawan, Iskandar Muda Rizky Parlambang, Rava Maulana +10Physical Consistency in Video GenerationRobot Manipulation Benchmarks