Cinematic camera control relies on a tight feedback loop between director and cinematographer, where camera motion and framing are continuously reviewed and refined. Recent generative camera systems can produce diverse, text-conditioned trajectories, but they lack this "director in the loop" and have no explicit supervision of whether a shot is visually desirable. This results in in-distribution camera motion but poor framing, off-screen characters, and undesirable visual aesthetics. In this paper, we introduce VERTIGO, the first framework for visual preference optimization of camera trajectory generators. Our framework leverages a real-time graphics engine (Unity) to render 2D visual previews from generated camera motion. A cinematically fine-tuned vision-language model then scores these previews using our proposed cyclic semantic similarity mechanism, which aligns renders with text prompts. This process provides the visual preference signals for Direct Preference Optimization (DPO) post-training. Both quantitative evaluations and user studies on Unity renders and diffusion-based Camera-to-Video pipelines show consistent gains in condition adherence, framing quality, and perceptual realism. Notably, VERTIGO reduces the character off-screen rate from 38% to nearly 0% while preserving the geometric fidelity of camera motion. User study participants further prefer VERTIGO over baselines across composition, consistency, prompt adherence, and aesthetic quality, confirming the perceptual benefits of our visual preference post-training.
Figures & tables
Figure 2: Pipeline of VERTIGO. From a camera prompt, the generator produces 3D trajectories rendered into preview sequences by a graphics engine. A VLM performs inverse reasoning to caption the realized motion; the original prompt and generated caption are compared in latent space to derive preference scores for DPO post-training.
Figure 3: Different VLM scoring and fine-tuning strategies of VERTIGO. We explore three preference scoring methods: (a) cyclic semantic scoring via latent-space similarity. (b) tag-consistency scoring; (c) direct scalar regression via RAFT-style fine-tuning on interpolated trajectories;
Methods
Camera trajectory quality
Render Quality
Video Quality
FCD↓
CS↑
P↑
R↑
D↑
C↑
MisR.↓
Cons.↑
Aes.↑
Cons.↑
Aes.↑
CCD [ 21 ]
104.60
30.03
0.58
0.01
0.41
0.10
0.999
0.820
0.460
0.901
0.296
DIRECTOR [ 7 ]
26.28
32.20
0.87
0.29
0.79
0.44
0.803
0.892
0.464
0.908
0.293
GenDoP [ 63 ]
4.17
67.98
0.92
0.72
0.93
0.78
0.387
0.904
0.515
0.967
0.722
VERTIGO
4.22
68.40
0.92
0.73
0.93
0.78
0.008
0.908
0.518
0.969
0.735
Table 1: Quantitative comparison of camera trajectory and video quality. We evaluate geometric fidelity using CLaTr-based metrics [ 7 ] and visual quality via VBench [ 27 ] metrics on both Unity-rendered sequences and controllable video generation. VERTIGO achieves competitive geometric performance while substantially improving visual metrics and target framing stability. Shaded indicates best performance.
Figure 4: Qualitative comparison of camera generators. VERTIGO accurately adheres to spatial composition instructions and maintains framing, whereas GenDoP and DIRECTOR misplace subjects and occasionally lose them.
Figure 5: Qualitative comparison of video-to-video transfer results. We compare VACE-based video transfer on two trajectories. For the first trajectory, we show GenDoP vs. VERTIGO transfer results; for the second, we compare static and animated character renderings transferred under the same trajectory. VERTIGO produces more robust framing after transfer, while character animation does not affect transfer quality, as trajectories are defined in the subject’s local coordinate space.
Metric
GenDoP
VERTIGO (Scoring Strategy)
VERTIGO ( β ablation)
Anim. Characters
Tag
Interp.
Sem. ( β=0.1 )
β=0.01
β=0.5
β=0.9
GenDoP
Ours
MisR. ↓
0.387
0.327
0.286
0.008
0.097
0.132
0.414
0.387
0.008
Cons. ↑
0.904
0.902
0.906
0.908
0.897
0.905
0.901
0.876
0.902
Aes. ↑
0.515
0.516
0.515
0.518
0.482
0.517
0.515
0.479
0.488
Table 2: Ablation study. Starting from the GenDoP baseline, we compare three VLM-based preference scoring strategies for VERTIGO, evaluate generalization on animated characters, and ablate DPO β . All models are evaluated on Unity-rendered sequences using VBench metrics.
Figure 6: User study best-of-4 results. Preference rates across evaluation dimensions for Unity rendering and video-to-video transfer (Wan 2.2 VACE). See supplementary for questionnaire details and additional results.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Method
Statistics
Annotations
Intrinsics
#Samples
#Frames
#Movement
Caption
Tags
(FOV)
RealEstate10k
SLAM / BA
79K
11M
-
✗
✗
✗
E.T.
SfM / HMR
115K
11M
7
✓
✗
✗
DataDoP
SLAM
29K
11M
7
✓
✗
✗
CCD
Synthetic
25K
4.5M
10
✓
✗
✗
LenScript (Ours)
Synthetic
120K
21.6M
13
✓
✓
✓
Appendix
Table 3: Dataset comparison. We compare the dataset with (i) RealEstate10k, a pure camera trajectory dataset designed for tasks like novel view synthesis; and (ii) three trajectory datasets captioned with detailed description on lens feature. Notice that Movement Types here only count basic motion types, excluding combinations of basic motions (as they can be easily achieved in game engines).
Figure 7: Unity interface. Panels: (A) 3D Scene and Game views for real-time asset layout and visual preview; (B) node-based storyboard with script editor and shot attribute panels for planning and prompt construction; (C) Timeline and Animation editors for fine-grained trajectory refinement, preview, and batch export supporting VLM scoring and DPO data collection.
Figure 8: Questionnaire interface. Top: the best-of-4 interface shows the camera prompt together with five evaluation dimensions, where participants select the best result among four methods for each dimension. Bottom: the full-ranking interface asks participants to rank the four methods by overall cinematic quality.
Unity ranking
Video Gen ranking
Method
Avg. ↓
1st
2nd
3rd
4th
Avg. ↓
1st
2nd
3rd
4th
CCD
2.97
8.8
14.7
47.1
29.4
3.03
5.9
20.6
38.2
35.3
DIRECTOR
3.27
6.1
12.1
30.3
51.5
3.21
8.8
11.8
29.4
50.0
GenDoP
2.26
14.7
55.9
17.6
11.8
2.18
29.4
35.3
23.5
11.8
VERTIGO
1.47
70.6
17.6
5.9
5.9
1.59
55.9
32.4
8.8
2.9
Appendix
Table 4: Full-ranking results. Percentages indicate how often each method is assigned each rank. A lower average rank is better.
Figure 9: Additional qualitative comparison of camera trajectories. Across diverse scenes and prompts, VERTIGO maintains stable framing, consistent subject retention, and smooth motion while adhering to compositional intent.
Figure 10: Additional qualitative comparison in Unity rendering and video generation. We compare CCD, DIRECTOR, GenDoP, and VERTIGO in both Unity-rendered previews and downstream video generation. VERTIGO produces more stable framing and stronger target retention in Unity, which transfers to more reliable composition and fewer visual failures in generated videos.
Complex camera motion is central to visual storytelling, yet translating cinematographic intent into controllable, executable camera trajectories remains challenging. We introduce FilmFlow, an agentic framework that converts natural-language instructions or reference videos into explicit camera trajectories for dynamic 3D environments. To generate these trajectories, a Director agent constructs executable camera plans by composing high-level motions from concurrent atomic operations that jointly control translation, rotation, and framing. The Director is supported by three specialized agents: an Interpreter that extracts scene-grounded cinematographic intent, a Scout that selects initial viewpoints, and a Critic that evaluates rendered results and provides visual feedback for iterative plan refinement. We also introduce CinemaBench, comprising 150 Blender scenes from self-created and licensed sources, with five camera instructions per scene for 750 text-directed tasks and 60 reference videos associated with a subset of scenes. CinemaBench supports evaluation of both text-directed camera generation and video reference-directed camera generation. Across a diverse set of evaluated baselines, FilmFlow achieves the strongest benchmark performance and the highest user preference, demonstrating effective spatial reasoning, temporal coordination, and visual composition for executable 3D cinematography.
Yiming Zhang, Jiarui Guo, Haojia Wei +5
Cornell University · Shanghai Jiao Tong University · University of California, Los Angeles +1
Generative video models have achieved remarkable visual fidelity and temporal coherence, yet intentional camera control remains elusive. Existing frameworks treat camera motion as a byproduct of pixel synthesis, producing trajectories that are stochastic, spatially inconsistent, and indifferent to the human subject driving the scene. In this work, we present Auteur, a method for language-driven, human-centric camera framing in generative video. Our core insight is that professional filmmakers conceive shots not as world-space trajectories but as framings defined relative to the actor, encoding shot size, angle, and composition as functions of human pose and motion. We formalize this intuition as a human-centric camera parameterization and introduce a Domain-Specific Language (DSL) that is convertible to standard 6-DoF camera parameters. A fine-tuned multimodal large language model then acts as a virtual director, mapping natural language descriptions and coarse human motion to sparse DSL keyframes that are deterministically interpolated into continuous camera trajectories, which are then provided as input to video generators. We train and evaluate Auteur on a new dataset of 34K aligned text, human motion, and DSL-annotated camera trajectories drawn from procedural synthesis and real-world movie footage from the CondensedMovies dataset. Auteur enables cinematographic framing of human-centered scenes, a capability largely absent in prior generative models. To assess this behavior, we propose new framing-focused metrics, and our experiments show that Auteur consistently outperforms existing methods. Project page is https://cyberiada.github.io/Auteur/
Muhammed Burak Kizil, Enes Sanli, Niloy J. Mitra +4
Koç University · University College London, Adobe · Adobe +1
Automatically generating cinematically expressive camera trajectories through 3D scenes from natural language descriptions is a challenging task of high practical value, with applications ranging from real-estate advertising to virtual tour creation. Existing methods either lack true 3D spatial awareness by relying on 2D image priors, or treat trajectory generation as a geometric path planning problem divorced from cinematographic semantics. We present CinemaTraj, a framework that reframes camera trajectory planning as a language-grounded spatial reasoning problem. Given a set of RGB-D images and a user prompt, CinemaTraj equips an LLM agent with a structured 3D scene graph: the agent decomposes the prompt into a sequence of atomic cinematographic movements (dolly, orbit, crane, pan, tilt, zoom, arc). Each movement is instantiated via a novel parametric trajectory representation that is both cinematographically expressive and optimizable for collision avoidance. The scene graph acts as a structured spatial prior, grounding the agent's reasoning in accurate geometric and semantic knowledge of the environment. CinemaTraj further generates synchronized voiceover and subtitles aligned with camera motion, producing narrated cinematic video outputs. We evaluate CinemaTraj on real-world ScanNet++ environments, and show that it produces prompt-faithful, collision-free trajectories with high cinematographic quality, outperforming existing approaches on prompt alignment, trajectory quality, and safety metrics.
Qianru Li, Xuyang Chen, Erkin Türköz +5
Technical University of Munich Munich, Germany · Huawei Dresden Research Center Munich, Germany