VERTIGO: Visual Preference Optimization for Cinematic Camera Trajectory Generation

Mengtian Li; Yuwei Lu; Feifei Li; Chenqi Gan; Zhifeng Xie; Xi Wang

cs.CVApr 2, 2026

VERTIGO: Visual Preference Optimization for Cinematic Camera Trajectory Generation

Authors: Mengtian Li, Yuwei Lu, Feifei Li, Chenqi Gan, Zhifeng Xie, Xi Wang

Organizations: Shanghai University · Shanghai Engineering Research Center of Motion Picture Special Effects · LIX, École Polytechnique, CNRS, IPP

Abstract

Cinematic camera control relies on a tight feedback loop between director and cinematographer, where camera motion and framing are continuously reviewed and refined. Recent generative camera systems can produce diverse, text-conditioned trajectories, but they lack this "director in the loop" and have no explicit supervision of whether a shot is visually desirable. This results in in-distribution camera motion but poor framing, off-screen characters, and undesirable visual aesthetics. In this paper, we introduce VERTIGO, the first framework for visual preference optimization of camera trajectory generators. Our framework leverages a real-time graphics engine (Unity) to render 2D visual previews from generated camera motion. A cinematically fine-tuned vision-language model then scores these previews using our proposed cyclic semantic similarity mechanism, which aligns renders with text prompts. This process provides the visual preference signals for Direct Preference Optimization (DPO) post-training. Both quantitative evaluations and user studies on Unity renders and diffusion-based Camera-to-Video pipelines show consistent gains in condition adherence, framing quality, and perceptual realism. Notably, VERTIGO reduces the character off-screen rate from 38% to nearly 0% while preserving the geometric fidelity of camera motion. User study participants further prefer VERTIGO over baselines across composition, consistency, prompt adherence, and aesthetic quality, confirming the perceptual benefits of our visual preference post-training.

Figures & tables

Figure 2: Pipeline of VERTIGO. From a camera prompt, the generator produces 3D trajectories rendered into preview sequences by a graphics engine. A VLM performs inverse reasoning to caption the realized motion; the original prompt and generated caption are compared in latent space to derive preference scores for DPO post-training.

Figure 3: Different VLM scoring and fine-tuning strategies of VERTIGO. We explore three preference scoring methods: (a) cyclic semantic scoring via latent-space similarity. (b) tag-consistency scoring; (c) direct scalar regression via RAFT-style fine-tuning on interpolated trajectories;

Methods	Camera trajectory quality						Render Quality			Video Quality
Methods	FCD↓	CS↑	P↑	R↑	D↑	C↑	MisR.↓	Cons.↑	Aes.↑	Cons.↑	Aes.↑
CCD [ 21 ]	104.60	30.03	0.58	0.01	0.41	0.10	0.999	0.820	0.460	0.901	0.296
DIRECTOR [ 7 ]	26.28	32.20	0.87	0.29	0.79	0.44	0.803	0.892	0.464	0.908	0.293
GenDoP [ 63 ]	4.17	67.98	0.92	0.72	0.93	0.78	0.387	0.904	0.515	0.967	0.722
VERTIGO	4.22	68.40	0.92	0.73	0.93	0.78	0.008	0.908	0.518	0.969	0.735

Table 1: Quantitative comparison of camera trajectory and video quality. We evaluate geometric fidelity using CLaTr-based metrics [ 7 ] and visual quality via VBench [ 27 ] metrics on both Unity-rendered sequences and controllable video generation. VERTIGO achieves competitive geometric performance while substantially improving visual metrics and target framing stability. Shaded indicates best performance.

Figure 4: Qualitative comparison of camera generators. VERTIGO accurately adheres to spatial composition instructions and maintains framing, whereas GenDoP and DIRECTOR misplace subjects and occasionally lose them.

Figure 5: Qualitative comparison of video-to-video transfer results. We compare VACE-based video transfer on two trajectories. For the first trajectory, we show GenDoP vs. VERTIGO transfer results; for the second, we compare static and animated character renderings transferred under the same trajectory. VERTIGO produces more robust framing after transfer, while character animation does not affect transfer quality, as trajectories are defined in the subject’s local coordinate space.

Metric	GenDoP	VERTIGO (Scoring Strategy)			VERTIGO ( $\beta$ ablation)			Anim. Characters
Metric	GenDoP	Tag	Interp.	Sem. ( $\beta{=}0.1$ )	$\beta{=}0.01$	$\beta{=}0.5$	$\beta{=}0.9$	GenDoP	Ours
MisR. $\downarrow$	0.387	0.327	0.286	0.008	0.097	0.132	0.414	0.387	0.008
Cons. $\uparrow$	0.904	0.902	0.906	0.908	0.897	0.905	0.901	0.876	0.902
Aes. $\uparrow$	0.515	0.516	0.515	0.518	0.482	0.517	0.515	0.479	0.488

Table 2: Ablation study. Starting from the GenDoP baseline, we compare three VLM-based preference scoring strategies for VERTIGO, evaluate generalization on animated characters, and ablate DPO

\beta

. All models are evaluated on Unity-rendered sequences using VBench metrics.

Figure 6: User study best-of-4 results. Preference rates across evaluation dimensions for Unity rendering and video-to-video transfer (Wan 2.2 VACE). See supplementary for questionnaire details and additional results.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Method	Statistics			Annotations		Intrinsics
Dataset	Method	#Samples	#Frames	#Movement	Caption	Tags	(FOV)
RealEstate10k	SLAM / BA	79K	11M	-	✗	✗	✗
E.T.	SfM / HMR	115K	11M	7	✓	✗	✗
DataDoP	SLAM	29K	11M	7	✓	✗	✗
CCD	Synthetic	25K	4.5M	10	✓	✗	✗
LenScript (Ours)	Synthetic	120K	21.6M	13	✓	✓	✓

Appendix

Table 3: Dataset comparison. We compare the dataset with (i) RealEstate10k, a pure camera trajectory dataset designed for tasks like novel view synthesis; and (ii) three trajectory datasets captioned with detailed description on lens feature. Notice that Movement Types here only count basic motion types, excluding combinations of basic motions (as they can be easily achieved in game engines).

Figure 7: Unity interface. Panels: (A) 3D Scene and Game views for real-time asset layout and visual preview; (B) node-based storyboard with script editor and shot attribute panels for planning and prompt construction; (C) Timeline and Animation editors for fine-grained trajectory refinement, preview, and batch export supporting VLM scoring and DPO data collection.

Figure 8: Questionnaire interface. Top: the best-of-4 interface shows the camera prompt together with five evaluation dimensions, where participants select the best result among four methods for each dimension. Bottom: the full-ranking interface asks participants to rank the four methods by overall cinematic quality.

	Unity ranking					Video Gen ranking
Method	Avg. $\downarrow$	1st	2nd	3rd	4th	Avg. $\downarrow$	1st	2nd	3rd	4th
CCD	2.97	8.8	14.7	47.1	29.4	3.03	5.9	20.6	38.2	35.3
DIRECTOR	3.27	6.1	12.1	30.3	51.5	3.21	8.8	11.8	29.4	50.0
GenDoP	2.26	14.7	55.9	17.6	11.8	2.18	29.4	35.3	23.5	11.8
VERTIGO	1.47	70.6	17.6	5.9	5.9	1.59	55.9	32.4	8.8	2.9

Appendix

Table 4: Full-ranking results. Percentages indicate how often each method is assigned each rank. A lower average rank is better.

Figure 9: Additional qualitative comparison of camera trajectories. Across diverse scenes and prompts, VERTIGO maintains stable framing, consistent subject retention, and smooth motion while adhering to compositional intent.

Figure 10: Additional qualitative comparison in Unity rendering and video generation. We compare CCD, DIRECTOR, GenDoP, and VERTIGO in both Unity-rendered previews and downstream video generation. VERTIGO produces more stable framing and stronger target retention in Unity, which transfers to more reliable composition and fewer visual failures in generated videos.

Camera-Controlled Video Generation

Camera-controlled video generation creates videos that follow specified viewpoints and camera paths while preserving coherent scenes and motion.

See all 142 papers

FilmFlow: Agentic System for 3D Virtual Film Cinematography
Oct 22, 2026Yiming Zhang, Jiarui Guo, Haojia Wei +5
Cornell University · Shanghai Jiao Tong University · University of California, Los Angeles +1
Auteur: Language-Driven Cinematographic Framing for Human-Centric Video Generation
Jun 1, 2026Muhammed Burak Kizil, Enes Sanli, Niloy J. Mitra +4Camera-Controlled Video Generation Video Generation
Koç University · University College London, Adobe · Adobe +1
CinemaTraj: Composing Atomic Camera Trajectories for 3D Scenes with LLM Agents
Jul 29, 2026Qianru Li, Xuyang Chen, Erkin Türköz +5Trajectory Generation LLM Agent Trajectory Synthesis
Technical University of Munich Munich, Germany · Huawei Dresden Research Center Munich, Germany

Oct 22, 2026J/K move · Enter open · S save

FilmFlow: Agentic System for 3D Virtual Film Cinematography

Yiming Zhang, Jiarui Guo, Haojia Wei, Yifei Liu+4

Cornell University · Shanghai Jiao Tong University · University of California, Los Angeles+1

Open paper AskPDF arXiv

Complex camera motion is central to visual storytelling, yet translating cinematographic intent into controllable, executable camera trajectories remains challenging. We introduce FilmFlow, an agentic framework that converts natural-language instructions or reference videos into explicit camera trajectories for dynamic 3D environments. To generate these trajectories, a Director agent constructs executable camera plans by composing high-level motions from concurrent atomic operations that jointly control translation, rotation, and framing. The Director is supported by three specialized agents: an Interpreter that extracts scene-grounded cinematographic intent, a Scout that selects initial viewpoints, and a Critic that evaluates rendered results and provides visual feedback for iterative plan refinement. We also introduce CinemaBench, comprising 150 Blender scenes from self-created and licensed sources, with five camera instructions per scene for 750 text-directed tasks and 60 reference videos associated with a subset of scenes. CinemaBench supports evaluation of both text-directed camera generation and video reference-directed camera generation. Across a diverse set of evaluated baselines, FilmFlow achieves the strongest benchmark performance and the highest user preference, demonstrating effective spatial reasoning, temporal coordination, and visual composition for executable 3D cinematography.

Abstract

Camera-Controlled Video Generation

Explore similar work

FilmFlow: Agentic System for 3D Virtual Film Cinematography

Auteur: Language-Driven Cinematographic Framing for Human-Centric Video Generation

CinemaTraj: Composing Atomic Camera Trajectories for 3D Scenes with LLM Agents