Designing with Cinematic Language: An Intent-Driven Approach to Human-AI Video Clip Co-Creation
Authors: Yi He, Zhengxi Yu, Feilin Han, Chuer Chen, Shengqi Dang, Nan Cao
Organizations: aiDVX Lab, College of Design and Innovation, Tongji University, Shanghai, China · Innovation Center for Intelligent Design, Shanghai Innovation Institute, Shanghai, China · Shanghai Research Institute for Intelligent Autonomous Systems, Shanghai, China
Generative video models have made video creation more accessible, yet creators still struggle to translate high-level narrative intentions into coherent cinematic expressions. Existing systems primarily rely on prompt refinement or expose isolated camera controls, leaving a gap between what to express and how to film it. We present CineAssist, a human-AI co-creation system that treats cinematic language as an interactive design medium connecting creative intentions with executable video controls. We construct a Cinematic Language Design Space and an annotated dataset from 107 filmmaking sources and 1,436 film clips, linking communicative intentions to cinematic parameter values. CineAssist combines a multi-agent workflow for intent alignment and strategy recommendation with an interactive cinematic editor and a context-sensitive preference learner for user-controlled personalization. We evaluate the system through three studies examining end-to-end usability, cinematic quality, and preference evolution. Results show that CineAssist facilitates efficient, controllable, and personalized intent-driven cinematic video creation.
Figures & tables
Figure 1: Overview of CineAssist. Guided by (a) the cinematic language design space, (b) the Intent Perceiver clarifies vague requests into explicit intentions, and (c) the Strategy Recommender translates them into executable Shot Scripts. (d) The editor instantiates and refines the resulting camera configurations in 3D, while (e) the Preference Learner learns reusable preferences from creators’ choices and edits. Together, these modules produce controllable cinematic rewrites that communicate the intended effect.
ID
Group
Age
Gender
Background
YoE
GenAI tools used
E1
Experienced
29
M
Director
5 years
Runway, Kling, Jimeng AI (Dreamina)
E2
Experienced
34
M
Writer-director
4 years
Vidu, Runway, Kling, Jimeng AI (Dreamina)
E3
Experienced
31
F
Director
5 years
Jimeng AI (Dreamina), ComfyUI, Kling
E4
Experienced
28
M
AI filmmaking practitioner
5 years
Vidu, Kling
E5
Experienced
36
M
Director
6 years
Jimeng AI (Dreamina), Runway, Pika, Kling
E6
Experienced
39
M
AI filmmaking practitioner
8 years
Stable Diffusion, ComfyUI, Runway, Kling
Table 1: Participant demographics and commonly used GenAI tools.
Figure 2: Current GenAI video-production workflow, the three challenges identified in the formative study, and their corresponding design goals.
Figure 3: Overview of CineAssist. The system consists of five interconnected modules: the Intent-Oriented Cinematic Language Design Space, Intent Perceiver, Strategy Recommender, Intent-Oriented Editor, and Preference Learner.
Figure 4: The CineAssist interface comprises (A) the Dialog Box for clarifying and confirming intentions, (B) the Strategy Explorer for comparing alternative strategies and Shot Scripts, (C) the Cinematic Editor for 3D refinement, and (D) the Video Generator for generating and previewing results. Insets A.1, B.1, and C.1 show representative interactions in greater detail.
Figure 5: Construction of the cinematic language design space. A literature review identifies what creators use cinematic language to communicate, while film-clip annotation characterizes how these intents are realized through cinematic techniques.
Figure 6: Frequency of the 16 initial intent categories identified from the literature.
Dimension
Representative Values
Shot scale
Extreme wide; wide; full; medium; medium close-up; close-up; extreme close-up
Composition
Centered; symmetry; rule of thirds; leading lines; negative space; foreground occlusion
Camera angle and viewpoint
Eye-level; low angle; high angle; overhead; subjective/POV; objective observation
Table 2: Cinematic parameter categories and representative values used in the annotation codebook.
Figure 7: The complete Intent-Oriented Cinematic Language Design Space. It organizes 53 secondary intents into 12 primary intent families and maps each intent to observed values across nine cinematic parameter dimensions. Colors distinguish the intent families, while cell saturation indicates how frequently each parameter value was observed for the corresponding intent.
Figure 8: Design Space.
Figure 9: Overview of the Preference Learner, which operates in three stages. (1) Preference Collection converts editing interactions into contextualized preference atoms. (2) Preference Consolidation integrates consistent and opposing evidence across sessions to update preference states. (3) Preference Ranking retrieves and ranks applicable preferences for a new task; user-confirmed preferences are then distilled into a workflow skill that guides strategy generation and refinement.
Figure 10: Qualitative comparison between CineAssist and direct Seedance 2.0 generation on four representative test cases. Five temporally ordered keyframes are shown for each condition and intent.
Figure 12: Four representative videos created with the complete CineAssist workflow during the end-to-end creation phase.
Figure 13: Ratings of intent alignment, strategy exploration, editing control, and the overall system across different criteria on a 5-point Likert scale, as provided by 6 expert users, where 5 is the best and 1 is the worst. Error bars represent standard deviations.
Figure 14: Clarifying an underspecified intent before cinematic strategy generation. Starting from the creator’s initial request, CineAssist progressively clarifies the intended experience through structured multi-turn interaction, captures the creator’s preferred visual expression, and synthesizes the resulting decisions into a confirmed Intent Brief.
Figure 15: Refining cinematic parameters and previewing their effects on the communicative intent and generated result.
Figure 16: Preference-informed video generation across tasks. Based on cinematic preferences learned from the creator’s prior selections and refinements, CineAssist adapts its strategies to diverse scenes while maintaining a consistent visual language characterized by environment-dominant framing, relatively small off-center characters, and substantial negative space.
We propose Camera Artist, a multi-agent framework that models a real-world filmmaking workflow to generate narrative videos with explicit cinematic language. While recent multi-agent systems have made substantial progress in automating filmmaking workflows from scripts to videos, they often lack explicit mechanisms to structure narrative progression across adjacent shots and deliberate use of cinematic language, resulting in fragmented storytelling and limited filmic quality. To address this, Camera Artist builds upon established agentic pipelines and introduces a dedicated Cinematography Shot Agent, which integrates recursive storyboard generation to strengthen shot-to-shot narrative continuity and cinematic language injection to produce more expressive, film-oriented shot designs. Extensive quantitative and qualitative results demonstrate that our approach consistently outperforms existing baselines in narrative consistency, dynamic expressiveness, and perceived film quality.
Haobo Hu, Qi Mao, Yuanhang Li +1
School of Information and Communication Engineering, Communication University of China, Beijing, China · State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing, China
Generative AI has made content creation increasingly accessible, but many AI-generated videos lack narrative coherence and creative direction, issues that become more substantial at longer durations. Unlike coding, where AI generation benefits from reliable feedback and techniques such as recurrent self-improvement, video generation requires subjective feedback about plot, scenes, and narrative, which naturally motivates approaches that incorporate human creative direction. We introduce CHIEF, a human-AI co-creation video generation framework that places the creator at the center of human-in-the-loop iterative video refinement, and supports them by providing automatic subjective feedback. The creator incorporates their creative direction by driving each iteration, while their revisions are incorporated by a specialized refiner agent. The feedback loop is generated by persona-conditioned multimodal LLMs that watch generated videos and produce subjective critique from the audience perspectives, providing feedback that self-evaluation alone cannot capture. To test the effectiveness of our proposed framework, we work with high school and college students with no prior filmmaking experience to create videos, from short 1-minute videos to a complete short 10-minute film with a complicated plot.
Denis Savytski, Aiden Lei, Heding Liu +4
Department of Computer Science, University of California, Davis, USA · The Harker School · Basis Independent Silicon Valley +1
Generative video models have achieved remarkable visual fidelity and temporal coherence, yet intentional camera control remains elusive. Existing frameworks treat camera motion as a byproduct of pixel synthesis, producing trajectories that are stochastic, spatially inconsistent, and indifferent to the human subject driving the scene. In this work, we present Auteur, a method for language-driven, human-centric camera framing in generative video. Our core insight is that professional filmmakers conceive shots not as world-space trajectories but as framings defined relative to the actor, encoding shot size, angle, and composition as functions of human pose and motion. We formalize this intuition as a human-centric camera parameterization and introduce a Domain-Specific Language (DSL) that is convertible to standard 6-DoF camera parameters. A fine-tuned multimodal large language model then acts as a virtual director, mapping natural language descriptions and coarse human motion to sparse DSL keyframes that are deterministically interpolated into continuous camera trajectories, which are then provided as input to video generators. We train and evaluate Auteur on a new dataset of 34K aligned text, human motion, and DSL-annotated camera trajectories drawn from procedural synthesis and real-world movie footage from the CondensedMovies dataset. Auteur enables cinematographic framing of human-centered scenes, a capability largely absent in prior generative models. To assess this behavior, we propose new framing-focused metrics, and our experiments show that Auteur consistently outperforms existing methods. Project page is https://cyberiada.github.io/Auteur/
Muhammed Burak Kizil, Enes Sanli, Niloy J. Mitra +4
Koç University · University College London, Adobe · Adobe +1