cs.CVOct 6, 2026

Diverse Motion Customization via Control-based Dynamic Optimization

Authors: Youngyoon Choi, Kihyun Kim, Jeongwoo Shin, Joonseok Lee

Organizations: Seoul National University · AIM Intelligence

Abstract

Despite recent advances in video generation, motion customization remains challenging due to content leakage, where appearance attributes from the reference video unintentionally propagate into the generated output. We identify this issue as a consequence of the generative process collapsing toward the reference video, which arises from formulating the learning objective as a direct regression on the reference. To address this, we propose Control-based Motion Customization (CMC), a principled training framework that is structurally robust to content leakage. Our key idea is to steer generative dynamics toward desired motion while avoiding collapse toward the reference video, which we formalize using Stochastic Optimal Control (SOC). Under this formulation, customized videos acquire the target motion yet remain within the pre-trained model's prompt-conditional distribution, where appearance is determined by the text prompt rather than the reference video. Furthermore, to improve efficiency, we tailor the SOC formulation to motion customization by eliminating the need for an explicit reward and introducing a timestep-adaptive motion cost that focuses only on early generative stages, accelerating training by 2.5 times. Extensive experiments demonstrate that CMC effectively mitigates content leakage and achieves competitive motion fidelity while preserving the diversity of the base model across diverse scenarios.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. When Distillation Breaks Motion Control: Restoring Generative Trajectories for Fast Video Generators

    Jun 24, 2025Jintao Rong, Xin Xie, Xinyi Yu +4Student-Generated TrajectoriesEffective Distillation

  2. Custom Forcing: Training-Free Subject Customization for Autoregressive Video Generation

    Oct 2, 2026Yunseung Ok, Hyunsoo Kim, Minseo Kim +1Autoregressive Video Generation

  3. Disco-LoRA: Disentangled Composition of Content, Style, and Motion for Multi-concept Video Customization

    Jun 25, 2026Xuancheng Xu, Gengyun Jia, Bing-Kun BaoDuetMotion