Unveiling the Value of Motion for Cinematic Camera Trajectories

Ziqi Zhou; Yujian Yuan; Laura Sevilla-Lara

cs.CVSep 30, 2026

Unveiling the Value of Motion for Cinematic Camera Trajectories

Authors: Ziqi Zhou, Yujian Yuan, Laura Sevilla-Lara

Organizations: University of Edinburgh · The Hong Kong University of Science and Technology

Abstract

Cinematic camera motion is a fundamental storytelling tool, defined not only by where the camera is positioned in the scene, but also by how it moves in terms of direction and speed. Recent work on camera trajectory generation and alignment to text relies on pose-centric representations. While in principle a network could derive direction of movement and speed, we find that in practice this might not happen. In fact, in this paper we discover that decomposing the camera trajectory representation from the traditional per-frame poses to direction and speed has surprising benefits across multiple tasks, including trajectory-to-text alignment as well as text-to-trajectory generation. To accurately evaluate the former, we introduce a simple and reliable protocol that overcomes the limitations of prior evaluation baselines. For the latter, building on this representational insight, we propose a novel generative model for camera trajectories, CineGEN, that achieves superior performance across a variety of metrics. We also propose a novel dataset, CineScript, containing movie clips that are enriched with scene descriptions as well as higher-level metadata. This novel data allows us to test models' ability to capture high-level cinematographic information. We show that, despite its simplicity, representing camera trajectories through direction and speed not only helps numerically to achieve better alignment and generation, but also inherently encodes complex directorial intent.

Figures & tables

Figure 1 : Overview of CineScript construction and trajectory representation. Movie clips are re-processed into camera trajectories, motion captions, screenplay-style loglines, and linked movie attributes. The right panel illustrates two kinds of representations mentioned in our paper.

Panel of Figure 2 — Figure 2 : Data statistics and examples in CineScript . Left: distribution of extracted translation (yellow) and rotation (green) primitives. Right: examples of clips with a similar coarse motion pattern (dolly-in) but different movie attributes, illustrating that camera style can vary substantially even under the same high-level motion.

Model	Rep.	R@1 $\uparrow$	R@5 $\uparrow$	R@10 $\uparrow$	MedR $\downarrow$	AlignScore $\uparrow$	#Params
Ours	DirSpeed	25.2	40.7	49.0	11	66.3	3.6 M
Ours	Pose9D	17.8	26.5	31.4	51	46.6	3.6 M
CLaTr [ 9 ]	DirSpeed	19.7	30.5	39.2	23	70.6	30 M
CLaTr [ 9 ]	Pose9D	6.9	12.0	14.7	361	37.4	30 M

Table 1: Alignment evaluator validation on motion captions. We compare CLaTr and our purely contrastive protocol under both trajectory representations.

Panel of Figure 3 — Figure 3 : Joint embedding spaces of the four alignment evaluators on the validation set, projected jointly with t-SNE. Trajectory embeddings ( $\bullet$ ) and matched text embeddings ( $\star$ ) are coloured by the same K-means cluster (computed on trajectory embeddings, $k=8$ ); thin lines connect $200$ random matched ( $\bullet$ , $\star$ ) pairs.

Attribute	Task	#Class	DirSpeed	Pose9D	$\Delta$
Era	single-label	3	49.8	47.3	$+2.6$
Genre	multi-label	3	62.1	60.7	$+1.4$
Director	single-label	3	68.2	51.8	$+16.4$

Table 2: Attribute prediction on real trajectories. All entries report macro-F1 (

\times 100

Figure 4 : Overview of the proposed generation and evaluation framework. Left: CineGEN integrates motion captions, loglines, and initial pose constraints to synthesize camera trajectories. The model progressively recovers masked trajectory tokens using a Transformer-based sequencer and a conditional denoiser through a variance-guided unmasking process. Right: Our evaluation protocol employs a purely contrastive InfoNCE objective to align motion captions and trajectories within a normalized joint embedding space, providing a rigorous measure of cross-modal consistency.

	Trajectory Quality			Text–Trajectory Alignment			Movie Attributes
Method / Setting	F1 $\uparrow$	FCD $\downarrow$	Coverage $\uparrow$	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$	Era $\uparrow$	Genre $\uparrow$	Director $\uparrow$
CCD ∗ [ 8 ]	0.117	52.45	0.298	4.90	0.08	1032.5	44.3	46.3	12.1
CCD [ 8 ]	0.128	49.19	0.270	18.82	0.43	397.0	47.2	56.3	32.7
E.T. ∗ [ 9 ]	0.015	135.74	0.034	0.00	0.08	960.0	40.7	56.7	25.0
E.T. [ 9 ]	0.000	179.88	0.014	0.36	0.16	745.0	40.7	45.8	33.6
GenDoP ∗ [ 10 ]	0.175	102.97	0.085	7.63	0.23	715.5	33.5	45.6	18.1

Table 3: Comparison with prior text-to-trajectory methods on CineScript .

*

denotes evaluation using released pretrained checkpoints, while other baseline rows are trained on CineScript . The light green row highlights our model. Bold indicates the best result.

Figure 5 : Qualitative comparison with prior generation methods. Starred columns denote released pretrained checkpoints, corresponding to $*$ in Table 3 . Compared with prior methods, CineGEN produces smoother and more coherent camera paths that better reflect compound motion instructions.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6 : Distribution of linked movie attributes in CineScript . Left: era distribution. Middle: genre distribution. Right: director distribution, highlighting only the top 10 named directors alongside the grouped long tail. This inherent imbalance motivates our use of targeted, frequent-class subsets for rigorous attribute evaluation.

Dataset	#Samples	#Frames	Avg(s)	motion	logline	movie attributes	Source
	Scale			Annotations
CCD [ 8 ]	25 K	4.5 M	7.2	✓	✗	✗	Synthetic
E.T. [ 9 ]	115 K	11 M	3.8	✓	✗	✗	Movie
DataDoP [ 10 ]	29 K	11 M	14.4	✓	✗	✗	Movie
CineScript (ours)	28 K	10 M	12.0	✓	✓	✓ ( $3.2$ K)	Movie

Appendix

Table 4: Comparison of datasets for text-to-camera-trajectory generation. CineScript is distinguished by the joint availability of motion captions, loglines, and linked real movie attributes.

Attribute	Classes	Type
Era	Film era ( $<2005$ ); Early digital ( $2005$ – $2012$ ); Digital mature ( $\geq 2013$ )	single-label
Genre	Drama/Romance; Comedy; Action/Thriller/Sci-Fi	multi-label
Director	Christopher Nolan; Wes Anderson; Steven Spielberg	single-label

Appendix

Table 5: Attribute task definitions. Clips outside the target classes are completely dropped rather than assigned to a generic “Other” class. This strictly prevents the classifiers from cheating the metric by defaulting to a trivial majority-class background prediction.

	Trajectory Quality			Text–Trajectory Alignment			Movie Attributes
Method	F1 $\uparrow$	FCD $\downarrow$	Coverage $\uparrow$	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$	Era $\uparrow$	Genre $\uparrow$	Director $\uparrow$
CCD ∗ [ 8 ]	0.117	52.45	0.298	4.90	0.08	1032.5	44.3	46.3	12.1
CCD [ 8 ]	0.128	49.19	0.270	18.82	0.43	397.0	47.2	56.3	32.7
CCD † [ 8 ]	0.168	25.96	0.576	11.71	0.35	690.0	42.5	48.5	39.8
E.T. ∗ [ 9 ]	0.015	135.74	0.034	0.00	0.08	960.0	40.7	56.7	25.0
E.T. [ 9 ]	0.194	16.39	0.596	29.78	0.81	251.0	42.0	57.5	28.1

Appendix

Table 6: Extended comparison against prior generation methods on CineScript . ∗ denotes released pretrained checkpoints; unmarked rows denote baselines retrained on CineScript using their native formulations; and † denotes diagnostic variants retrained with continuous DirSpeed prediction and motion-caption conditioning. Movie-attribute metrics report macro-F1 (

\times 100

Method	F1 $\uparrow$	FCD $\downarrow$	Coverage $\uparrow$	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$	Era $\uparrow$	Genre $\uparrow$	Director $\uparrow$
CineGEN	0.437	6.77	0.783	57.79	3.35	52	47.8	60.7	48.1
CineGEN caption-only	0.419	6.76	0.749	65.83	5.82	42	45.6	60.6	41.6
GenDoP [ 10 ]	0.234	22.10	0.586	33.11	0.93	195	33.2	52.5	29.9

Appendix

Table 7: Conditioning-matched comparison. The caption-only CineGEN variant removes both logline and first-pose conditioning.

Figure 7 : Blinded human evaluation across 30 clips and 24 participants. Multiple selections are allowed, so the selection-rate bars do not sum to

100\%

Method	FCD $\downarrow$	Coverage $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$
CCD ∗ [ 8 ]	195.23	0.293	0.39	1158.5
CCD [ 8 ]	567.67	0.253	0.43	965.5
E.T. ∗ [ 9 ]	1311.94	0.006	0.04	1175.0
E.T. [ 9 ]	111.71	0.593	0.39	558.0
GenDoP ∗ [ 10 ]	339.73	0.188	0.12	810.0
GenDoP [ 10 ]	59.85	0.656	0.27	467.5

Appendix

Table 8: Independent evaluation using the frozen official CLaTr encoder. ∗ denotes released pretrained checkpoints, while unmarked baselines are retrained on CineScript using their native formulations.

Evaluator	Matched cosine $\uparrow$	Mismatched cosine	R@1 $\uparrow$	MedR $\downarrow$
CLaTr- DirSpeed	70.6	$3.2\pm 31.4$	19.7	23
Ours- DirSpeed	66.3	$-0.5\pm 22.8$	25.2	11

Appendix

Table 9: Matched and mismatched cosine statistics for the two DirSpeed evaluators.

	Trajectory Quality			Text–Trajectory Alignment			Movie Attributes
Setting	F1 $\uparrow$	FCD $\downarrow$	Coverage $\uparrow$	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$	Era $\uparrow$	Genre $\uparrow$	Director $\uparrow$
CineGEN (ours)	0.437	6.77	0.783	57.79	3.35	52.0	47.8	60.7	48.1
Pose9D rep.	0.189	20.99	0.638	41.11	1.94	110.5	42.5	58.2	29.8
w/o $\mathbf{x}_{\mathrm{fp}}$	0.425	15.13	0.722	53.07	3.84	54.0	40.2	61.2	35.2
w/o $\mathbf{e}_{\mathrm{logline}}$	0.402	9.75	0.769	51.16	2.44	72.0	47.7	56.1	45.1
w/o variance	0.401	8.84	0.784	53.60	2.79	58.0	42.1	61.2	43.2

Appendix

Table 10: Ablation study of CineGEN on CineScript . Underlined entries indicate specific metrics where an ablated variant marginally exceeds the full model.

Feature	R@1 $\uparrow$	R@5 $\uparrow$	R@10 $\uparrow$	MedR $\downarrow$	AlignScore $\uparrow$
direction-only	19.7	33.0	40.1	22	61.2
speed-only	5.7	9.1	12.3	290	46.8
velocity	3.8	8.0	9.3	377	48.4
DirSpeed (ours)	25.2	40.7	49.0	11	66.3

Appendix

Table 11: Component ablation of DirSpeed for text–trajectory alignment. All variants utilize the identical contrastive architecture trained from scratch. DirSpeed dominates retrieval metrics, confirming the vital synergy between normalized direction and log-speed.

Rep.	Text	AlignScore $\uparrow$	R@1 $\uparrow$	R@5 $\uparrow$	R@10 $\uparrow$	MedR $\downarrow$
DirSpeed	motion	66.3	25.2	40.7	49.0	11
	motion + logline	50.6	4.6	14.6	23.7	41
	logline	31.1	1.0	2.9	4.8	227
Pose9D	motion	38.7	8.7	15.9	21.5	66
	motion + logline	33.3	2.3	7.7	12.3	102
	logline	26.2	0.7	1.9	3.0	428

Appendix

Table 12: Effect of text conditioning on alignment retrieval across

2{,}578

validation clips. We train and evaluate our contrastive protocol using varying text inputs. Motion captions provide the unambiguous signal necessary for geometric alignment, whereas narrative loglines fail to act as reliable singular targets. Note: Random motion-to-text R@1 is approximately

0.04\%

Representation	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$
First-frame-canonical Pose9D	46.4	15.2	51
+ Scale normalization Pose9D	49.0	18.9	40
DirSpeed	66.3	25.2	11

Appendix

Table 13: Matched coordinate and scale controls for Pose9D .

Magnitude parameterization	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$
Linear speed	63.6	21.5	14
Global z-score	64.7	24.4	11
$\log(1+\mathrm{speed})$	64.5	22.4	14
$\log(\mathrm{speed}+\varepsilon)$	66.3	25.2	11

Appendix

Table 14: Ablation of the speed parameterization with direction features held fixed.

Representation	Short F1 $\uparrow$	Long F1 $\uparrow$	Short FCD $\downarrow$	Long FCD $\downarrow$	Short Cov. $\uparrow$	Long Cov. $\uparrow$
Pose9D	0.240	0.187	10.14	33.08	0.794	0.635
DirSpeed	0.411	0.419	10.72	17.72	0.820	0.786

Appendix

Table 15: Generation quality on short and long trajectories.

Representation	Input dim.	AlignScore $\uparrow$	R@1 $\uparrow$	MedR $\downarrow$
Pose9D	9	46.6	17.8	51
DirSpeed	8	66.3	25.2	11
Pose9D + DirSpeed	17	66.9	27.0	9

Appendix

Table 16: Joint use of pose and motion features.

Training data	Pose9D R@1 $\uparrow$	DirSpeed R@1 $\uparrow$
10%	11.3	15.1
25%	11.0	15.9
50%	12.2	20.2
100%	17.8	25.2

Appendix

Table 17: Data scaling for Pose9D and DirSpeed .

Model	Trainable params	Pose9D R@1 $\uparrow$	DirSpeed R@1 $\uparrow$
Small	0.53M	16.3	24.2
Base	3.49M	17.8	25.2
Large	26.14M	17.1	25.0

Appendix

Table 18: Trajectory-encoder scaling.

Trajectory Encoder (Trainable, ${\sim}3.6$ M params)
Hyperparameter	Value
Input dimension	8 ( DirSpeed ) or 9 ( Pose9D )
Linear projection	$256$
Positional encoding	Sinusoidal (max length $5000$ )
Transformer layers	$4$
Transformer dimensions	$d_{\mathrm{model}}=256$ , $4$ heads, $d_{\mathrm{ff}}=1024$

Appendix

Table 19: Architecture and training hyperparameters for the Contrastive Alignment Evaluator.

Conditioning and Architecture
Hyperparameter	Value
Text encoder	CLIP ViT-B/32 (frozen, $512$ dims)
Logline embedding dim	$64$
Sequencer	TransformerAdaLN ( $1$ block)
Sequencer dimensions	$d_{\mathrm{model}}=512$ , $8$ heads, $d_{\mathrm{ff}}=4096$ , Dropout $0.2$
Conditioning fusion	Text ( $512$ ) + First Pose ( $64$ ) + Logline ( $64$ ) = $640$ dims $\rightarrow$ $512$ dims

Appendix

Table 20: Architecture, sampling, and training hyperparameters for CineGEN .

Figure 8 : Impact of logline conditioning on trajectory generation. We generate camera paths using the same motion caption (Top row: “truck left”, Bottom row: “pedestal up”) but varying loglines. While the fundamental camera movement firmly aligns with the motion caption, the logline modulates the trajectory’s scale, pace, and subtle dynamics to suit the specific narrative context.

Explore similar work

CinemaTraj: Composing Atomic Camera Trajectories for 3D Scenes with LLM Agents
Jul 29, 2026Qianru Li, Xuyang Chen, Erkin Türköz +5Camera Trajectories Cinematic Ideal
Technical University of Munich Munich, Germany · Huawei Dresden Research Center Munich, Germany
LAMP: Language-Assisted Motion Planning for Controllable Video Generation
Dec 3, 2025Muhammed Burak Kizil, Enes Sanli, Niloy J. Mitra +3Camera Trajectories Motion
Koç University · University College London · Adobe Research +1
Auteur: Language-Driven Cinematographic Framing for Human-Centric Video Generation
Jun 1, 2026Muhammed Burak Kizil, Enes Sanli, Niloy J. Mitra +4Cinematic Ideal Dramadirector
Koç University · University College London, Adobe · Adobe +1

Jul 29, 2026, cs.CVJ/K move · Enter open · S save

CinemaTraj: Composing Atomic Camera Trajectories for 3D Scenes with LLM Agents

Qianru Li, Xuyang Chen, Erkin Türköz, Lu Liu+4

Technical University of Munich Munich, Germany · Huawei Dresden Research Center Munich, Germany

Camera Trajectories Cinematic Ideal Films Trajectory Generation Spatial Reasoning Large Language Model Agents

Open paper AskPDF arXiv

Automatically generating cinematically expressive camera trajectories through 3D scenes from natural language descriptions is a challenging task of high practical value, with applications ranging from real-estate advertising to virtual tour creation. Existing methods either lack true 3D spatial awareness by relying on 2D image priors, or treat trajectory generation as a geometric path planning problem divorced from cinematographic semantics. We present CinemaTraj, a framework that reframes camera trajectory planning as a language-grounded spatial reasoning problem. Given a set of RGB-D images and a user prompt, CinemaTraj equips an LLM agent with a structured 3D scene graph: the agent decomposes the prompt into a sequence of atomic cinematographic movements (dolly, orbit, crane, pan, tilt, zoom, arc). Each movement is instantiated via a novel parametric trajectory representation that is both cinematographically expressive and optimizable for collision avoidance. The scene graph acts as a structured spatial prior, grounding the agent's reasoning in accurate geometric and semantic knowledge of the environment. CinemaTraj further generates synchronized voiceover and subtitles aligned with camera motion, producing narrated cinematic video outputs. We evaluate CinemaTraj on real-world ScanNet++ environments, and show that it produces prompt-faithful, collision-free trajectories with high cinematographic quality, outperforming existing approaches on prompt alignment, trajectory quality, and safety metrics.

Abstract

Explore similar work

CinemaTraj: Composing Atomic Camera Trajectories for 3D Scenes with LLM Agents

LAMP: Language-Assisted Motion Planning for Controllable Video Generation

Auteur: Language-Driven Cinematographic Framing for Human-Centric Video Generation