Unveiling the Value of Motion for Cinematic Camera Trajectories

Ziqi Zhou; Yujian Yuan; Laura Sevilla-Lara

cs.CVSep 30, 2026

Unveiling the Value of Motion for Cinematic Camera Trajectories

Authors: Ziqi Zhou, Yujian Yuan, Laura Sevilla-Lara

Organizations: University of Edinburgh · The Hong Kong University of Science and Technology

Abstract

Cinematic camera motion is a fundamental storytelling tool, defined not only by where the camera is positioned in the scene, but also by how it moves in terms of direction and speed. Recent work on camera trajectory generation and alignment to text relies on pose-centric representations. While in principle a network could derive direction of movement and speed, we find that in practice this might not happen. In fact, in this paper we discover that decomposing the camera trajectory representation from the traditional per-frame poses to direction and speed has surprising benefits across multiple tasks, including trajectory-to-text alignment as well as text-to-trajectory generation. To accurately evaluate the former, we introduce a simple and reliable protocol that overcomes the limitations of prior evaluation baselines. For the latter, building on this representational insight, we propose a novel generative model for camera trajectories, CineGEN, that achieves superior performance across a variety of metrics. We also propose a novel dataset, CineScript, containing movie clips that are enriched with scene descriptions as well as higher-level metadata. This novel data allows us to test models' ability to capture high-level cinematographic information. We show that, despite its simplicity, representing camera trajectories through direction and speed not only helps numerically to achieve better alignment and generation, but also inherently encodes complex directorial intent.

Figures & tables

Figure 1 : Overview of CineScript construction and trajectory representation. Movie clips are re-processed into camera trajectories, motion captions, screenplay-style loglines, and linked movie attributes. The right panel illustrates two kinds of representations mentioned in our paper.

Panel of Figure 2 — Figure 2 : Data statistics and examples in CineScript . Left: distribution of extracted translation (yellow) and rotation (green) primitives. Right: examples of clips with a similar coarse motion pattern (dolly-in) but different movie attributes, illustrating that camera style can vary substantially even under the same high-level motion.

Model	Rep.	R@1 $\uparrow$	R@5 $\uparrow$	R@10 $\uparrow$	MedR $\downarrow$	AlignScore $\uparrow$	#Params
Ours	DirSpeed	25.2	40.7	49.0	11	66.3	3.6 M
Ours	Pose9D	17.8	26.5	31.4	51	46.6	3.6 M
CLaTr [ 9 ]	DirSpeed	19.7	30.5	39.2	23	70.6	30 M
CLaTr [ 9 ]	Pose9D	6.9	12.0	14.7	361	37.4	30 M

Table 1: Alignment evaluator validation on motion captions. We compare CLaTr and our purely contrastive protocol under both trajectory representations.

Panel of Figure 3 — Figure 3 : Joint embedding spaces of the four alignment evaluators on the validation set, projected jointly with t-SNE. Trajectory embeddings ( $\bullet$ ) and matched text embeddings ( $\star$ ) are coloured by the same K-means cluster (computed on trajectory embeddings, $k=8$ ); thin lines connect $200$ random matched ( $\bullet$ , $\star$ ) pairs.

Attribute	Task	#Class	DirSpeed	Pose9D	$\Delta$
Era	single-label	3	49.8	47.3	$+2.6$
Genre	multi-label	3	62.1	60.7	$+1.4$
Director	single-label	3	68.2	51.8	$+16.4$

Table 2: Attribute prediction on real trajectories. All entries report macro-F1 (

\times 100

Figure 4 : Overview of the proposed generation and evaluation framework. Left: CineGEN integrates motion captions, loglines, and initial pose constraints to synthesize camera trajectories. The model progressively recovers masked trajectory tokens using a Transformer-based sequencer and a conditional denoiser through a variance-guided unmasking process. Right: Our evaluation protocol employs a purely contrastive InfoNCE objective to align motion captions and trajectories within a normalized joint embedding space, providing a rigorous measure of cross-modal consistency.

	Trajectory Quality			Text–Trajectory Alignment			Movie Attributes
Method / Setting	F1 $\uparrow$	FCD $\downarrow$	Coverage $\uparrow$	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$	Era $\uparrow$	Genre $\uparrow$	Director $\uparrow$
CCD ∗ [ 8 ]	0.117	52.45	0.298	4.90	0.08	1032.5	44.3	46.3	12.1
CCD [ 8 ]	0.128	49.19	0.270	18.82	0.43	397.0	47.2	56.3	32.7
E.T. ∗ [ 9 ]	0.015	135.74	0.034	0.00	0.08	960.0	40.7	56.7	25.0
E.T. [ 9 ]	0.000	179.88	0.014	0.36	0.16	745.0	40.7	45.8	33.6
GenDoP ∗ [ 10 ]	0.175	102.97	0.085	7.63	0.23	715.5	33.5	45.6	18.1

Table 3: Comparison with prior text-to-trajectory methods on CineScript .

*

denotes evaluation using released pretrained checkpoints, while other baseline rows are trained on CineScript . The light green row highlights our model. Bold indicates the best result.

Figure 5 : Qualitative comparison with prior generation methods. Starred columns denote released pretrained checkpoints, corresponding to $*$ in Table 3 . Compared with prior methods, CineGEN produces smoother and more coherent camera paths that better reflect compound motion instructions.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6 : Distribution of linked movie attributes in CineScript . Left: era distribution. Middle: genre distribution. Right: director distribution, highlighting only the top 10 named directors alongside the grouped long tail. This inherent imbalance motivates our use of targeted, frequent-class subsets for rigorous attribute evaluation.

Dataset	#Samples	#Frames	Avg(s)	motion	logline	movie attributes	Source
	Scale			Annotations
CCD [ 8 ]	25 K	4.5 M	7.2	✓	✗	✗	Synthetic
E.T. [ 9 ]	115 K	11 M	3.8	✓	✗	✗	Movie
DataDoP [ 10 ]	29 K	11 M	14.4	✓	✗	✗	Movie
CineScript (ours)	28 K	10 M	12.0	✓	✓	✓ ( $3.2$ K)	Movie

Appendix

Table 4: Comparison of datasets for text-to-camera-trajectory generation. CineScript is distinguished by the joint availability of motion captions, loglines, and linked real movie attributes.

Attribute	Classes	Type
Era	Film era ( $<2005$ ); Early digital ( $2005$ – $2012$ ); Digital mature ( $\geq 2013$ )	single-label
Genre	Drama/Romance; Comedy; Action/Thriller/Sci-Fi	multi-label
Director	Christopher Nolan; Wes Anderson; Steven Spielberg	single-label

Appendix

Table 5: Attribute task definitions. Clips outside the target classes are completely dropped rather than assigned to a generic “Other” class. This strictly prevents the classifiers from cheating the metric by defaulting to a trivial majority-class background prediction.

	Trajectory Quality			Text–Trajectory Alignment			Movie Attributes
Method	F1 $\uparrow$	FCD $\downarrow$	Coverage $\uparrow$	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$	Era $\uparrow$	Genre $\uparrow$	Director $\uparrow$
CCD ∗ [ 8 ]	0.117	52.45	0.298	4.90	0.08	1032.5	44.3	46.3	12.1
CCD [ 8 ]	0.128	49.19	0.270	18.82	0.43	397.0	47.2	56.3	32.7
CCD † [ 8 ]	0.168	25.96	0.576	11.71	0.35	690.0	42.5	48.5	39.8
E.T. ∗ [ 9 ]	0.015	135.74	0.034	0.00	0.08	960.0	40.7	56.7	25.0
E.T. [ 9 ]	0.194	16.39	0.596	29.78	0.81	251.0	42.0	57.5	28.1

Appendix

Table 6: Extended comparison against prior generation methods on CineScript . ∗ denotes released pretrained checkpoints; unmarked rows denote baselines retrained on CineScript using their native formulations; and † denotes diagnostic variants retrained with continuous DirSpeed prediction and motion-caption conditioning. Movie-attribute metrics report macro-F1 (

\times 100

Method	F1 $\uparrow$	FCD $\downarrow$	Coverage $\uparrow$	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$	Era $\uparrow$	Genre $\uparrow$	Director $\uparrow$
CineGEN	0.437	6.77	0.783	57.79	3.35	52	47.8	60.7	48.1
CineGEN caption-only	0.419	6.76	0.749	65.83	5.82	42	45.6	60.6	41.6
GenDoP [ 10 ]	0.234	22.10	0.586	33.11	0.93	195	33.2	52.5	29.9

Appendix

Table 7: Conditioning-matched comparison. The caption-only CineGEN variant removes both logline and first-pose conditioning.

Figure 7 : Blinded human evaluation across 30 clips and 24 participants. Multiple selections are allowed, so the selection-rate bars do not sum to

100\%

Method	FCD $\downarrow$	Coverage $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$
CCD ∗ [ 8 ]	195.23	0.293	0.39	1158.5
CCD [ 8 ]	567.67	0.253	0.43	965.5
E.T. ∗ [ 9 ]	1311.94	0.006	0.04	1175.0
E.T. [ 9 ]	111.71	0.593	0.39	558.0
GenDoP ∗ [ 10 ]	339.73	0.188	0.12	810.0
GenDoP [ 10 ]	59.85	0.656	0.27	467.5

Appendix

Table 8: Independent evaluation using the frozen official CLaTr encoder. ∗ denotes released pretrained checkpoints, while unmarked baselines are retrained on CineScript using their native formulations.

Evaluator	Matched cosine $\uparrow$	Mismatched cosine	R@1 $\uparrow$	MedR $\downarrow$
CLaTr- DirSpeed	70.6	$3.2\pm 31.4$	19.7	23
Ours- DirSpeed	66.3	$-0.5\pm 22.8$	25.2	11

Appendix

Table 9: Matched and mismatched cosine statistics for the two DirSpeed evaluators.

	Trajectory Quality			Text–Trajectory Alignment			Movie Attributes
Setting	F1 $\uparrow$	FCD $\downarrow$	Coverage $\uparrow$	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$	Era $\uparrow$	Genre $\uparrow$	Director $\uparrow$
CineGEN (ours)	0.437	6.77	0.783	57.79	3.35	52.0	47.8	60.7	48.1
Pose9D rep.	0.189	20.99	0.638	41.11	1.94	110.5	42.5	58.2	29.8
w/o $\mathbf{x}_{\mathrm{fp}}$	0.425	15.13	0.722	53.07	3.84	54.0	40.2	61.2	35.2
w/o $\mathbf{e}_{\mathrm{logline}}$	0.402	9.75	0.769	51.16	2.44	72.0	47.7	56.1	45.1
w/o variance	0.401	8.84	0.784	53.60	2.79	58.0	42.1	61.2	43.2

Appendix

Table 10: Ablation study of CineGEN on CineScript . Underlined entries indicate specific metrics where an ablated variant marginally exceeds the full model.

Feature	R@1 $\uparrow$	R@5 $\uparrow$	R@10 $\uparrow$	MedR $\downarrow$	AlignScore $\uparrow$
direction-only	19.7	33.0	40.1	22	61.2
speed-only	5.7	9.1	12.3	290	46.8
velocity	3.8	8.0	9.3	377	48.4
DirSpeed (ours)	25.2	40.7	49.0	11	66.3

Appendix

Table 11: Component ablation of DirSpeed for text–trajectory alignment. All variants utilize the identical contrastive architecture trained from scratch. DirSpeed dominates retrieval metrics, confirming the vital synergy between normalized direction and log-speed.

Rep.	Text	AlignScore $\uparrow$	R@1 $\uparrow$	R@5 $\uparrow$	R@10 $\uparrow$	MedR $\downarrow$
DirSpeed	motion	66.3	25.2	40.7	49.0	11
	motion + logline	50.6	4.6	14.6	23.7	41
	logline	31.1	1.0	2.9	4.8	227
Pose9D	motion	38.7	8.7	15.9	21.5	66
	motion + logline	33.3	2.3	7.7	12.3	102
	logline	26.2	0.7	1.9	3.0	428

Appendix

Table 12: Effect of text conditioning on alignment retrieval across

2{,}578

validation clips. We train and evaluate our contrastive protocol using varying text inputs. Motion captions provide the unambiguous signal necessary for geometric alignment, whereas narrative loglines fail to act as reliable singular targets. Note: Random motion-to-text R@1 is approximately

0.04\%

Representation	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$
First-frame-canonical Pose9D	46.4	15.2	51
+ Scale normalization Pose9D	49.0	18.9	40
DirSpeed	66.3	25.2	11

Appendix

Table 13: Matched coordinate and scale controls for Pose9D .

Magnitude parameterization	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$
Linear speed	63.6	21.5	14
Global z-score	64.7	24.4	11
$\log(1+\mathrm{speed})$	64.5	22.4	14
$\log(\mathrm{speed}+\varepsilon)$	66.3	25.2	11

Appendix

Table 14: Ablation of the speed parameterization with direction features held fixed.

Representation	Short F1 $\uparrow$	Long F1 $\uparrow$	Short FCD $\downarrow$	Long FCD $\downarrow$	Short Cov. $\uparrow$	Long Cov. $\uparrow$
Pose9D	0.240	0.187	10.14	33.08	0.794	0.635
DirSpeed	0.411	0.419	10.72	17.72	0.820	0.786

Appendix

Table 15: Generation quality on short and long trajectories.

Representation	Input dim.	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$
Pose9D	9	46.6	17.8	51
DirSpeed	8	66.3	25.2	11
Pose9D + DirSpeed	17	66.9	27.0	9

Appendix

Table 16: Joint use of pose and motion features.

Training data	Pose9D R@1 $\uparrow$	DirSpeed R@1 $\uparrow$
10%	11.3	15.1
25%	11.0	15.9
50%	12.2	20.2
100%	17.8	25.2

Appendix

Table 17: Data scaling for Pose9D and DirSpeed .

Model	Trainable params	Pose9D R@1 $\uparrow$	DirSpeed R@1 $\uparrow$
Small	0.53M	16.3	24.2
Base	3.49M	17.8	25.2
Large	26.14M	17.1	25.0

Appendix

Table 18: Trajectory-encoder scaling.

Trajectory Encoder (Trainable, ${\sim}3.6$ M params)
Hyperparameter	Value
Input dimension	8 ( DirSpeed ) or 9 ( Pose9D )
Linear projection	$256$
Positional encoding	Sinusoidal (max length $5000$ )
Transformer layers	$4$
Transformer dimensions	$d_{\mathrm{model}}=256$ , $4$ heads, $d_{\mathrm{ff}}=1024$

Appendix

Table 19: Architecture and training hyperparameters for the Contrastive Alignment Evaluator.

Conditioning and Architecture
Hyperparameter	Value
Text encoder	CLIP ViT-B/32 (frozen, $512$ dims)
Logline embedding dim	$64$
Sequencer	TransformerAdaLN ( $1$ block)
Sequencer dimensions	$d_{\mathrm{model}}=512$ , $8$ heads, $d_{\mathrm{ff}}=4096$ , Dropout $0.2$
Conditioning fusion	Text ( $512$ ) + First Pose ( $64$ ) + Logline ( $64$ ) = $640$ dims $\rightarrow$ $512$ dims

Appendix

Table 20: Architecture, sampling, and training hyperparameters for CineGEN .

Figure 8 : Impact of logline conditioning on trajectory generation. We generate camera paths using the same motion caption (Top row: “truck left”, Bottom row: “pedestal up”) but varying loglines. While the fundamental camera movement firmly aligns with the motion caption, the logline modulates the trajectory’s scale, pace, and subtle dynamics to suit the specific narrative context.

Explore similar work

Jul 29, 2026cs.CV

CinemaTraj: Composing Atomic Camera Trajectories for 3D Scenes with LLM Agents

Automatically generating cinematically expressive camera trajectories through 3D scenes from natural language descriptions is a challenging task of high practical value, with applications ranging from real-estate advertising to virtual tour creation. Existing methods either lack true 3D spatial awareness by relying on 2D image priors, or treat trajectory generation as a geometric path planning problem divorced from cinematographic semantics. We present CinemaTraj, a framework that reframes camera trajectory planning as a language-grounded spatial reasoning problem. Given a set of RGB-D images and a user prompt, CinemaTraj equips an LLM agent with a structured 3D scene graph: the agent decomposes the prompt into a sequence of atomic cinematographic movements (dolly, orbit, crane, pan, tilt, zoom, arc). Each movement is instantiated via a novel parametric trajectory representation that is both cinematographically expressive and optimizable for collision avoidance. The scene graph acts as a structured spatial prior, grounding the agent's reasoning in accurate geometric and semantic knowledge of the environment. CinemaTraj further generates synchronized voiceover and subtitles aligned with camera motion, producing narrated cinematic video outputs. We evaluate CinemaTraj on real-world ScanNet++ environments, and show that it produces prompt-faithful, collision-free trajectories with high cinematographic quality, outperforming existing approaches on prompt alignment, trajectory quality, and safety metrics.

Qianru Li, Xuyang Chen, Erkin Türköz +5

Technical University of Munich Munich, Germany · Huawei Dresden Research Center Munich, Germany

Camera Trajectories Cinematic Ideal Films Trajectory Generation Spatial Reasoning Large Language Model Agents

Ask PDF

Dec 3, 2025cs.CV

LAMP: Language-Assisted Motion Planning for Controllable Video Generation

Video generation has achieved remarkable progress in visual fidelity and controllability, enabling conditioning on text, layout, or motion. Among these, motion control - specifying object dynamics and camera trajectories - is essential for composing complex, cinematic scenes, yet existing interfaces remain limited. We introduce LAMP that leverages large language models (LLMs) as motion planners to translate natural language descriptions into explicit 3D trajectories for dynamic objects and (relatively defined) cameras. LAMP defines a motion domain-specific language (DSL), inspired by cinematography conventions. By harnessing program synthesis capabilities of LLMs, LAMP generates structured motion programs from natural language, which are deterministically mapped to 3D trajectories. We construct a large-scale procedural dataset pairing natural text descriptions with corresponding motion programs and 3D trajectories. Experiments demonstrate LAMP's improved performance in motion controllability and alignment with user intent compared to state-of-the-art alternatives establishing the first framework for generating both object and camera motions directly from natural language specifications. Code, models and data are available on our project page.

Muhammed Burak Kizil, Enes Sanli, Niloy J. Mitra +3

Koç University · University College London · Adobe Research +1

Camera Trajectories Motion Motion Planning Domain-Specific Language

Ask PDF

Jun 1, 2026cs.CV

Auteur: Language-Driven Cinematographic Framing for Human-Centric Video Generation

Generative video models have achieved remarkable visual fidelity and temporal coherence, yet intentional camera control remains elusive. Existing frameworks treat camera motion as a byproduct of pixel synthesis, producing trajectories that are stochastic, spatially inconsistent, and indifferent to the human subject driving the scene. In this work, we present Auteur, a method for language-driven, human-centric camera framing in generative video. Our core insight is that professional filmmakers conceive shots not as world-space trajectories but as framings defined relative to the actor, encoding shot size, angle, and composition as functions of human pose and motion. We formalize this intuition as a human-centric camera parameterization and introduce a Domain-Specific Language (DSL) that is convertible to standard 6-DoF camera parameters. A fine-tuned multimodal large language model then acts as a virtual director, mapping natural language descriptions and coarse human motion to sparse DSL keyframes that are deterministically interpolated into continuous camera trajectories, which are then provided as input to video generators. We train and evaluate Auteur on a new dataset of 34K aligned text, human motion, and DSL-annotated camera trajectories drawn from procedural synthesis and real-world movie footage from the CondensedMovies dataset. Auteur enables cinematographic framing of human-centered scenes, a capability largely absent in prior generative models. To assess this behavior, we propose new framing-focused metrics, and our experiments show that Auteur consistently outperforms existing methods. Project page is https://cyberiada.github.io/Auteur/

Muhammed Burak Kizil, Enes Sanli, Niloy J. Mitra +4

Koç University · University College London, Adobe · Adobe +1

Cinematic Ideal Dramadirector Camera Control Human Motion Generation Authorship

Ask PDF