Video Generation Evaluation

Overview

The field measures whether generated videos look good and, crucially, whether they follow prompts and depict coherent events over time. It combines perceptual metrics such as FVD and human preference ratings with benchmarks for motion, physics, geometry, long-form continuity, and audio-video alignment. Recent approaches use task-specific rubrics, vision-language model judges, and interpretable diagnostics to assess failures that aggregate scores can hide. A central challenge is that automated evaluators may disagree with people, while visual realism often fails to reflect physical reasoning, causal consistency, or successful task completion.

Latest papers

CardsList

205 papers15 papers in the last four weeks, down 53% on the four weeks before. 0.1% of all new papers.

  1. EVF-Bench: A Comprehensive Benchmark for E-commerce Visual Effects in Video Generation

    Nov 18, 2026Run Ling, Ke Cao, Runze He +7Video Generation EvaluationVideo Generation

  2. EVA-RSI: Recursive Self-Improvement of Video Evaluators for Judging Temporal Quality

    Nov 18, 2026Yu Yuan, Ruizhou Ding, Shaoheng Fang +2Video Generation EvaluationRecursive Self-Improvement

  3. When a Larger Guidance Scale Repairs Text-to-Video Failures and When It Does Not

    Nov 17, 2026Taewon Kang, Matthias ZwickerT2V GenerationDiffusion Model Guidance

  4. FVScore: Specification-Driven Neuro-Symbolic Video Evaluation

    Nov 17, 2026Do Xuan Long, Long T. Le, Yiwen Song +3Video Generation EvaluationTemporal Consistency in Video Generation

  5. SciWorldBench: Do Video World Models Really Understand Scientific Experiments?

    Nov 16, 2026Xue Yang, Haonan Cai, Xinyu Hang +4Video World ModelsVideo Generation Evaluation

  6. VAHBench: A Harness Benchmark for Agentic Long-Video Generation

    Nov 16, 2026Hongbo Zhang, Liuyang Song, Hang Zhou +4Agentic Video GenerationVideo Generation Evaluation

  7. PhyRubric: Rubric-Grounded Diagnostic Evaluation of Physical Commonsense in Text-to-Video Generation

    Nov 10, 2026Huiyi Chen, Weitai Kang, Shicheng Fan +5Physical Consistency in Video GenerationVideo Generation Evaluation

  8. ArtiVidRM: Verifiable Artifact-as-Rubric Reward Modeling for Explicit Quality Articulation in Video Generation

    Nov 5, 2026Caorui Li, Yangguang Ji, Shuxia Lin +12Video Reward ModelingReward Modeling

  9. Synthetic Realities: Evaluating Human Ability to Distinguish AIGenerated Videos from Real Footage

    Nov 1, 2026Danyal Sarfraz, Raja Mubashar Karim, KwanMyung KimVideo Generation EvaluationSynthetic Video Detection Benchmarks

  10. GeoFlip Preview: Benchmarking Geometry-Sensitive Physics in Video Generation

    Oct 30, 2026Yuqi Li, Junming Liu, Bowen Hu +5Physical Consistency in Video GenerationGeometric Consistency in Video Generation

  11. Evaluating Visual Realism of AI-Generated Videos with Eye Tracking

    Oct 28, 2026Danqing Shi, Lan Jiang, Jake Lyons +1Video Generation EvaluationEye Tracking

  12. More Memory, Better Videos? A Controlled Study of Visual Memory in Long-Video Generation

    Oct 26, 2026Yuzhuo Chen, Huafeng Shi, Xinyu Wang +6Memory-Augmented Video GenerationRetrieval-Augmented Video Generation

  13. RGraph-Eval: Task-Grounded Scene Graphs for Evaluating Robot Video Generation

    Oct 22, 2026Aleksandr Lazarev, Sofia Zimina, Roman Kuznetsov +3Video Generation EvaluationVideo SGG

  14. RoboJudge: Multimodal Language Models as Judges for Embodied Video Generation

    Oct 20, 2026Siyuan Ma, Yu Qi, Cheng Ye +8Vision-Language Model-as-a-JudgeVideo Generation Evaluation

  15. PVBench: Quantifying Physical Plausibility in Video Generation with Object-Level Physical Measurements

    Oct 19, 2026Kaiwei Chen, Moran Li, Zhenye Gan +5Physical Consistency in Video GenerationVideo Generation Evaluation

  16. AVC-Reward: Event-Centric Audio-Video Consistency Evaluation

    Oct 15, 2026Yuruo Wu, Kaihang Pan, Kai Shen +5Video Generation EvaluationAudio-Video Generation

  17. Beyond Source Prompts: Towards Visually Grounded AI-Generated Video Quality Assessment

    Oct 15, 2026Kehao Zhou, Yixuan Li, Yuxuan Jiang +1Video Quality AssessmentVideo Generation Evaluation

  18. Orbia: Do Generated Videos Form a Consistent 3D World?

    Oct 14, 2026Dongyue Lu, Guian Fang, Hur Lim +20Geometric Consistency in Video GenerationWorld Model Evaluation

  19. PhyMetric: Diagnosing Physical Plausibility in Text-to-Video Generation via Scene-Level QA

    Oct 14, 2026Siwei Meng, Xuerou Cai, Rui Hu +1Video Generation EvaluationPhysical Consistency in Video Generation

  20. IsoVRBench: Video Reward Modeling Without Human Feedback

    Oct 12, 2026Shuyue Stella Li, Jacqueline He, Jiahui Chen +5Video Reward ModelingReward Modeling

  21. A Plausible Future Is Not a Causal Future: Can Video Models Generate How the World Unfolds?

    Oct 11, 2026Jialiang Li, Zhengning Xu, Zongyang Ma +1Causal Video GenerationVideo Generation

  22. EmoTrace-Bench: Diagnosing Emotional Expression in Text-to-Video Generation through Controlled Cues and Intensity

    Oct 11, 2026Yeheng Wang, Liudan Shi, Yi Zheng +5T2V GenerationVideo Generation Evaluation

  23. TextVBench: Evaluating Text Rendering and Preservation in Video Generation

    Oct 9, 2026Ruiqiang Zhang, Hengyi Wang, Chang Liu +3Video Generation EvaluationT2V Generation