ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation

Songlin Yang; Zhe Wang; Xuyi Yang; Songchun Zhang; Xianghao Kong; Taiyi Wu; Xiaotong Zhao; Ran Zhang

cs.CVMar 12, 2026

ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation

Authors: Songlin Yang, Zhe Wang, Xuyi Yang, Songchun Zhang, Xianghao Kong, Taiyi Wu, Xiaotong Zhao, Ran Zhang,

Organizations: Tencent Video, China

Abstract

Text-driven video generation has democratized film creation, but camera control in cinematic multi-shot scenarios remains a significant block. Implicit textual prompts lack precision, while explicit trajectory conditioning imposes prohibitive manual overhead and often triggers execution failures in current models. To overcome this bottleneck, we propose a data-centric paradigm shift, positing that aligned (Caption, Trajectory, Video) triplets form an inherent joint distribution that can connect automated plotting and precise execution. Guided by this insight, we present ShotVerse, a ``Plan-then-Control'' framework that decouples generation into two collaborative agents: a VLM (Vision-Language Model)-based Planner that leverages spatial priors to obtain cinematic, globally aligned trajectories from text, and a Controller that renders these trajectories into multi-shot video content via a camera adapter. Central to our approach is the construction of a data foundation: we design an automated multi-shot camera calibration pipeline aligns disjoint single-shot trajectories into a unified global coordinate system. This facilitates the curation of ShotVerse-Bench, a high-fidelity cinematic dataset with a three-track evaluation protocol that serves as the bedrock for our framework. Extensive experiments demonstrate that ShotVerse effectively bridges the gap between unreliable textual control and labor-intensive manual plotting, achieving superior cinematic aesthetics and generating multi-shot videos that are both camera-accurate and cross-shot consistent.

Figures & tables

Figure 1. Cinematic, Camera-Controlled, Multi-Shot Video Creation via our ShotVerse Framework. Examples demonstrate high-fidelity and great camera-controlled generation across diverse genres. The inset 3D plots visualize the plotted explicit trajectories.

Figure 2. Method Overview. (i) Dataset Curation . We construct the ShotVerse-Bench by aligning multi-shot trajectories into a unified global coordinate system via camera calibration, paired with hierarchical global and per-shot captions. (ii) Trajectory Plotting : The Planner utilizes a VLM to process the hierarchical prompt interleaved with learnable trajectory query tokens. These inputs are encoded into context-aware embeddings and transformed into explicit camera poses via a Trajectory Decoder and a Pose De-Tokenizer. (iii) Trajectory Injection : The Controller synthesizes high-fidelity videos using a holistic DiT backbone. It precisely follows the trajectories via a Camera Adapter and a 4D Rotary Positional Embedding strategy.

Dataset	Caption Annotation				Traj Type	Domain	Statistics
Dataset	Traj	Scene	Intent	#Vocab	Traj Type	Domain	#Sample	#Frame
MVImgNet ( 60 )	$\times$	$\times$	$\times$	–	Object/Scene-Centric	Captured	22K	6.5M
RealEstate10k ( 68 )	$\times$	$\times$	$\times$	–	Object/Scene-Centric	Youtube	79K	11M
DL3DV-10K ( 37 )	$\times$	$\times$	$\times$	–	Object/Scene-Centric	Captured	10K	51M
CCD ( 29 )	$\checkmark$	$\times$	$\times$	48	Tracking	Synthetic	25K	4.5M
E.T. ( 10 )	$\checkmark$	$\times$	$\times$	1790	Tracking	Film	115K	11M

Table 1. Comparisons of Camera Trajectory Datasets. ShotVerse-Bench is the first large-scale dataset that provides multi-shot cinematic camera trajectories together with rich, multi-level caption annotations.

Method	Dataset	DataDoP ( 62 )		ShotVerse-Bench
Method	Dataset	F1-Score $\uparrow$	CLaTr-CLIP $\uparrow$	F1-Score $\uparrow$	CLaTr-CLIP $\uparrow$
CCD ( 29 )	Pre-Trained	0.315	4.247	0.323	7.032
E.T. ( 10 )	Pre-Trained	0.319	0.000	0.289	2.462
Director3D ( 35 )	Pre-Trained	0.126	0.000	0.162	1.237
GenDoP ( 62 )	Pre-Trained	0.399	32.408	0.326	23.089
GenDoP ( 62 )	ShotVerse-Bench	0.268	24.132	0.343	33.875

Table 2. Track A: Quantitative Results of Text-Trajectory Alignment.

Method	Trans. Error $\downarrow$	Rotation Error $\downarrow$	CAS $\uparrow$
MotionCtrl ( 54 )	0.0900	2.56	0.329
CameraCtrl ( 22 )	0.0571	1.28	0.343
ReCamMaster ( 3 )	0.0589	1.12	0.408
ShotVerse (Ours)	0.0163	0.73	0.500

Table 3. Track B: Quantitative Evaluation of Camera Control. All methods receive ground-truth trajectories.

Method	Sem. Consist. (Global) $\uparrow$	Sem. Consist. (Shot) $\uparrow$	Aesthetic Quality $\uparrow$	Shot Trans. Accuracy $\uparrow$	FVD $\downarrow$
HoloCine ( 42 )	0.297	0.254	4.981	0.645	407.54
MultiShotMaster ( 51 )	0.279	0.247	5.210	0.927	440.78
Sora2 ( 43 )	0.297	-	5.344	-	372.13
VEO3 ( 19 )	0.282	-	5.441	-	941.50
Kling3.0 ( 31 )	0.288	-	5.167	-	719.44
Seedance2.0 ( 17 )	0.285	-	5.381	-	605.17

Table 4. Track C: Quantitative Evaluation of Multi-Shot Quality. Without shot-splitting, shot metrics cannot be calculated for some baselines.

	Method	Motion Type Appropriateness $\uparrow$	Motion Duration Appropriateness $\uparrow$	Subject Emphasis & Saliency $\uparrow$	Cinematic Pacing $\uparrow$
	HoloCine ( 42 )	4.324	4.281	3.997	3.208
	VEO3 ( 19 )	4.402	4.189	4.252	3.288
VLM-Based	Sora2 ( 43 )	4.371	4.258	3.892	3.236
	Kling3.0 ( 31 )	4.302	4.153	3.872	3.108
	Seedance2.0 ( 17 )	4.402	4.279	4.328	3.279
	ShotVerse (Ours)	4.447	4.304	4.426	3.384

Table 5. Track C: Quantitative Evaluation of Cinematic Quality.

	w/o VLM encoder	w/ VLM decoder	w/o Query Tokens	Ours
F1-Score $\uparrow$	0.343	0.248	0.251	0.422
CLaTr-CLIP $\uparrow$	33.875	15.078	18.796	35.016

Table 6. Quantitative Evaluation of Ablation Study (Planner).

Method	Trans. Error $\downarrow$	Rotation Error $\downarrow$	Shot Trans. Acc. $\uparrow$	Sem. Consist. (Global) $\uparrow$	Sem. Consist. (Shot) $\uparrow$	Aesthetic Quality $\uparrow$
w/o Cam. Enc. (HoloCine)	0.0609	1.27	0.645	0.297	0.254	4.981
w/ Low&High Noise Enc.	0.0189	0.74	0.930	0.296	0.250	5.321
w/ 3D RoPE	0.0323	1.04	0.429	0.290	0.251	5.413
w/ Synthetic Data	0.0509	1.35	0.705	0.292	0.253	4.833
w/o Camera Calibration	0.0165	0.79	0.931	0.296	0.251	5.136
ShotVerse (Ours)	0.0163	0.73	0.933	0.299	0.255	5.465

Table 7. Quantitative Evaluation of Ablation Study (Controller).

Figure 3. Comparisons with the State-of-the-Art Baseline Methods. Early camera-controlled text-driven generation models ( e.g. , CameraCtrl, MotionCtrl) struggle to handle complex cinematic camera trajectories. ReCamMaster executes the trajectory but drifts away from the subject in Shot 1. HoloCine, MultiShotMaster, Sora2, VEO3, and Kling3.0, and Seedance2.0 fail to execute the complex “orbit” command, remaining nearly static. These failures demonstrate that for text-driven models, scaling up caption density is insufficient to achieve precise control without explicit geometric guidance.

Figure 5. Discussion on Generalizability .

Camera-Controlled Video Generation

Camera-controlled video generation creates videos that follow specified viewpoints and camera paths while preserving coherent scenes and motion.

See all 142 papers

Mise-en-Video: Fine-Grained 3D Control for Multi-Shot Video Generation
Oct 19, 2026Yingshuang Zou, Xiaoyu Shi, Yawen Luo +8Multi-Shot Video Generation Controllable Video Generation
The Hong Kong University of Science and Technology · KlingAI Research · The Chinese University of Hong Kong +2
ShotPlan: Cinematic Video Generation with Learnable Planning Token
Jul 20, 2026Su Guo, Guangce Liu, Haosen Yang +7Video Diffusion Models Video Generation
Institute of Artificial Intelligence (TeleAI), China Telecom · Harbin Institute of Technology
TriMotion: Modality-Agnostic Camera Control for Video Generation
Jun 18, 2026Seunghyun Shin, Jifei Song, Wooseok Jeon +2Camera-Controlled Video Generation Multimodal Learning
GIST · Huawei Noah’s Ark Lab · Yonsei University +1

Oct 19, 2026J/K move · Enter open · S save

Mise-en-Video: Fine-Grained 3D Control for Multi-Shot Video Generation

Yingshuang Zou, Xiaoyu Shi, Yawen Luo, Ziang Li+7

The Hong Kong University of Science and Technology · KlingAI Research · The Chinese University of Hong Kong+2

Multi-Shot Video Generation Controllable Video Generation 3D-Aware Video Generation Temporal Consistency in Video Generation

Open paper AskPDF arXiv

Recent advances in text-to-video generation have enabled high-fidelity single-shot synthesis, yet practical storytelling creation requires multi-shot videos with both strong inter-shot consistency and fine-grained controllability. Existing multi-shot video generation approaches achieve strong consistency but still fall short in controllability. In this paper, we propose Mise-en-Video (MeV), a highly controllable multi-shot video generation framework that allows video creators to conduct staging the multi-shot in the 3D space and then generate the corresponding multi-shot video. To achieve this, our framework combines a hierarchical attention mechanism for multi-level narrative consistency with 3D-aware layout guidance for precise, per-shot spatial control. To support training and evaluation, we develop standardized annotation pipelines and construct a 160K-video dataset with hierarchical multi-shot texts and oriented 3D bounding boxes. Extensive experiments demonstrate that MeV consistently outperforms strong baselines in video quality, inter-shot consistency, and subject-level controllability.

Abstract

Camera-Controlled Video Generation

Explore similar work

Mise-en-Video: Fine-Grained 3D Control for Multi-Shot Video Generation

ShotPlan: Cinematic Video Generation with Learnable Planning Token

TriMotion: Modality-Agnostic Camera Control for Video Generation