A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilation
Organizations: School of Artificial Intelligence, Beijing University of Posts and Telecommunications · AI Technology Center, Online Video Business Unit, Tencent PCG · Tsinghua University · Beijing Academy of Artificial Intelligence ·
Abstract
The surging demand for adapting long-form cinematic content into short videos has motivated the need for versatile automatic video compilation systems. However, existing compilation methods are limited to predefined tasks, and the community lacks a comprehensive benchmark to evaluate the cinematic compilation. To address this, we introduce CineBench, the first benchmark for instruction-driven cinematic video compilation, featuring diverse user instructions and high-quality ground-truth compilations annotated by professional editors. To overcome contextual collapse and temporal fragmentation, we present CineAgents, a multi-agent system that reformulates cinematic video compilation into ``design-and-compose'' paradigm. CineAgents performs script reverse-engineering to construct a hierarchical narrative memory to provide multi-level context and employs an iterative narrative planning process that refines a creative blueprint into a final compiled script. Extensive experiments demonstrate that CineAgents significantly outperforms existing methods, generating compilations with superior narrative coherence and logical coherence.
Figures & tables
Instruction-Based Video Editing
Instruction-based video editing changes video content from natural-language directions while preserving everything the instruction does not target.
See all 63 papers