Controllable Video Generation

Overview

The field studies how to make generated videos follow user intent, from preserving a subject or style to specifying poses, object trajectories, camera moves, events, and physical properties. Common approaches condition diffusion models with text, images, tracks, masks, or 3D geometry, adapt pretrained models with ControlNet or LoRA modules, or steer sampling at inference; newer world models add explicit state, planning, and memory for interactive, long-horizon generation. Evaluation spans prompt and control adherence, identity and scene consistency, motion accuracy, physical plausibility, and video quality, including tests on driving, human-object interaction, and cinematic scenes. A central tension remains: stronger or denser controls can improve precision but may reduce naturalness or visual quality, while long rollouts can drift or fail to realize intended state changes even when individual frames look convincing.

Latest papers

CardsList

341 papers34 papers in the last four weeks, down 26% on the four weeks before. 0.3% of all new papers.

  1. CustomX: Unified Character, Action, and Scene Customization in Video World Models

    Dec 18, 2025Yitong Wang, Fangyun Wei, Hongyang Zhang +2HOI GenerationAction-Conditioned Video Generation

  2. Elastic3D: Controllable Stereo Video Conversion with Guided Latent Decoding

    Dec 16, 2025Nando Metzger, Prune Truong, Goutam Bhat +2Video Diffusion ModelsStereo Vision

  3. A Study of the Design Space of Motion and Disocclusion Control in Video Generation

    Dec 15, 2025Anran Qi, Changjian Li, Adrien Bousseau +1Image-to-Video GenerationControllable Video Generation

  4. LAMP: Language-Assisted Motion Planning for Controllable Video Generation

    Dec 3, 2025Muhammed Burak Kizil, Enes Sanli, Niloy J. Mitra +3Camera-Controlled Video GenerationTrajectory Generation

  5. Exo2EgoSyn: Unlocking Foundation Video Generation Models for Exocentric-to-Egocentric Video Synthesis

    Nov 25, 2025Mohammad Mahdi, Yuqian Fu, Nedko Savov +3Camera-Conditioned Video GenerationVideo Diffusion Models

  6. Learning to Generate Rigid Body Interactions with Video Diffusion Models

    Oct 2, 2025David Romero, Ariana Bermudez, Viacheslav Iablochnikov +3HOI GenerationPhysical Consistency in Video Generation

  7. When Distillation Breaks Motion Control: Restoring Generative Trajectories for Fast Video Generators

    Jun 24, 2025Jintao Rong, Xin Xie, Xinyi Yu +4Video Diffusion ModelsDiffusion Models for Motion Generation

  8. HunyuanVideo-HOMA: Generic Human-Object Interaction in Multimodal Driven Human Animation

    Jun 10, 2025Ziyao Huang, Zixiang Zhou, Juan Cao +9HOI GenerationPhysical Consistency in Video Generation