cs.CVMay 27, 2026

SmartDirector: Keyframe-Conditioned Cinematic Video Generation with Narrative Pacing Control

Authors: Zhida ZhangJie MaZhan PengHaoxue WuYang HanJun LiangJie CaoJing Li

Organizations: 1NLPR, CISIA · 2Youku Moku-Lab · 3HUST

Abstract

The narrative quality of a video fundamentally determines its perceptual value. Although existing video generation methods can produce visually appealing content, they predominantly rely on sparse conditioning signals such as text prompts or first/last frames, which limits precise control over narrative structure and temporal pacing. In this paper, we propose SmartDirector, a framework that enhances the narrative capacity of video generation models through multiple keyframes. SmartDirector supports flexible generation scenarios including single-shot generation, multi-shot narrative synthesis, and video extension. The framework operates in two stages: Director-Gen generates a low-resolution video conditioned on the provided keyframes, and Director-SR refines the output by exploiting high-resolution keyframes as semantic anchors to recover fine-grained details. To enable robust multi-keyframe training, we construct a data pipeline that curates single-shot and multi-shot sequences from movies. Extensive experiments demonstrate that SmartDirector substantially outperforms existing state-of-the-art approaches. We will release the code to facilitate further research.

Explore similar work

CardsList
  1. DramaDirector: Geometry-Guided Short Drama Generation

    Jun 23, 2026Hengji Zhou, Sijie Liu, Jianrun Chen +3Co-DirectorFilms

  2. DrawVideo: Generating Long Video from Storyboard Keyframe Sketches

    May 22, 2026Chuanzhi Xu, Huiqi Liang, Bang Shi +7Sketches