Camera-Controlled Video Generation

Overview

Camera-controlled video generation studies how to make a generated or re-shot video follow a desired camera path, from simple pans and orbits to long-horizon navigation. Approaches condition diffusion or autoregressive models on camera poses, geometric cues such as depth and point tracks, or persistent 3D and visual memories; language-based planners can also turn cinematic intent into trajectories. Evaluation increasingly tests camera adherence alongside visual quality, cross-view consistency, and whether scenes remain stable on revisits, using benchmarks with real and synthetic videos. The main tension is that stronger geometric control and long-term consistency can conflict with photorealism, dynamic scene changes, and fast interactive generation.

Latest papers

152 papers15 papers in the last four weeks, down 40% on the four weeks before. 0.1% of all new papers.

Oct 28, 2025

VividCam: Learning Unconventional Camera Motions from Virtual Synthetic Videos

Although recent video generative models are getting more capable of following external camera controls, imposed by either text descriptions or camera trajectories, they still struggle to generalize to unconventional camera motions, which is crucial in creating truly original and artistic videos. The challenge lies in finding sufficient training videos with the intended uncommon camera motions. To this end, we propose VividCam, a training paradigm that enables diffusion models to learn complex camera motions from synthetic videos, releasing the reliance on collecting realistic training videos. VividCam incorporates multiple disentanglement strategies that isolate camera motion learning from synthetic appearance artifacts, ensuring more robust motion representation and mitigating domain shift. We show that our design synthesizes a wide range of precisely controlled camera motions using surprisingly simple synthetic data. Notably, this synthetic data often consists of basic geometries within a low-poly 3D scene and can be efficiently rendered by engines like Unity. Our video results can be found in https://wuqiuche.github.io/VividCamDemoPage/ .
Aug 11, 2025

Dream4D: Lifting Camera-Controlled I2V towards Spatiotemporally Consistent 4D Generation

The synthesis of spatiotemporally coherent 4D content presents fundamental challenges in computer vision, requiring simultaneous modeling of high-fidelity spatial representations and physically plausible temporal dynamics. Current approaches often struggle to maintain view consistency while handling complex scene dynamics, particularly in large-scale environments with multiple interacting elements. This work introduces Dream4D, a novel framework that bridges this gap through a synergy of controllable video generation and neural 4D reconstruction. Our approach seamlessly combines a two-stage architecture: it first predicts optimal camera trajectories from a single image using few-shot learning, then generates geometrically consistent multi-view sequences via a specialized pose-conditioned diffusion process, which are finally converted into a persistent 4D representation. This framework is the first to leverage both rich temporal priors from video diffusion models and geometric awareness of the reconstruction models, which significantly facilitates 4D generation and shows higher quality (e.g., mPSNR, mSSIM) over existing methods.