cs.AINov 16, 2026

VAHBench: A Harness Benchmark for Agentic Long-Video Generation

Authors: Hongbo Zhang, Liuyang Song, Hang Zhou, Xinchen Liu, Wei He, Kai Han, Yunhe Wang

Organizations: TokenRhythm Technologies · Peking University

Abstract

AI agents are commonly used for everyday information retrieval and coding. We ask whether they can also interpret human requests and turn them into complete videos. We introduce VAHBench, a benchmark that tests this ability through five short-drama briefs specifying story, characters, dialogue and duration. Agents plan and carry out production using shared generation tools. Evaluation combines video completeness and basic quality with checks of production reliability and reporting accuracy. Across 240 runs covering 48 configurations of general-purpose and video-specific agents, higher quality scores do not necessarily mean more complete delivery. Missing episodes, repeated footage and static filler reveal gaps between producing a video file and fulfilling the brief. Added composition tools help some agents but do not consistently improve performance. VAHBench measures how well agents translate user intent into finished video content and helps identify where that process breaks down.

Figures & tables

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. VideoGen-Agent: Reinforcing Video Generation Agents

    Sep 21, 2026Binxu Li, Haoyi Duan, Yuhui Zhang +10RL for Video GenerationAgentic Video Generation