Video Diffusion Models

Ask about this topic

Overview

Video diffusion models synthesize or transform frame sequences by denoising video latents, often in diffusion transformers, conditioned on text, images, reference clips, or structured controls. Researchers use temporal attention, autoregressive chunking, and pose, camera, depth, or physics cues to improve motion, identity, geometry, and interaction; related systems support editing, restoration, avatars, and world-model prediction. Evaluation spans prompt alignment and visual quality, using benchmarks such as VBench and FVD, as well as controllability, physical plausibility, and downstream action performance. A central challenge is balancing fidelity and precise control over long videos with the speed and memory needed for streaming, while limiting drift and physically implausible events.

Latest papers

CardsList

738 papers122 papers in the last four weeks, up 171% on the four weeks before. 0.7% of all new papers.

  1. VibeFlow: Versatile Video Chroma-Lux Editing through Self-Supervised Learning

    Apr 15, 2026Yifan Li, Pei Cheng, Bin Fu +2Video RelightingVideo Editing

    Wangxuan Institute of Computer Technology, Peking University · Tencent

  2. FreqFormer: Hierarchical Frequency-Domain Attention with Adaptive Spectral Routing for Long-Sequence Video Diffusion Transformers

    Apr 14, 2026Haopeng JinDiffusion Model Inference AccelerationDiffusion Transformer

    Beijing University of Posts and Telecommunications

  3. ARGen: Affect-Reinforced Generative Augmentation towards Vision-based Dynamic Emotion Perception

    Apr 14, 2026Huanzhen Wang, Ziheng Zhou, Jiaqi Song +4Synthetic Data AugmentationFacial Expression Recognition

    Fudan University · East China Normal University

  4. Ride the Wave: Precision-Allocated Sparse Attention for Smooth Video Generation

    Apr 14, 2026Wentai Zhang, Ronghui Xi, Shiyao Peng +4Sparse AttentionDiffusion Model Inference Acceleration

    Beijing University of Posts and Telecommunications, Beijing, China · Nanyang Technological University, Singapore, Singapore

  5. Representations Before Pixels: Semantics-Guided Hierarchical Video Prediction

    Apr 13, 2026Efstathios Karypidis, Spyros Gidaris, Nikos KomodakisVideo PredictionTemporal Consistency in Video Generation

    Archimedes, Athena Research Center, Greece · National Technical University of Athens · University of Crete +1

  6. Latent-Compressed Variational Autoencoder for Video Diffusion Models

    Apr 12, 2026Jiarui Guan, Wenshuai Zhao, Zhengtao Zou +2Video Diffusion ModelsVariational Autoencoders

    Aalto University · ELLIS Institute Finland · University of Oulu

  7. Prompt Relay: Inference-Time Temporal Control for Multi-Event Video Generation

    Apr 11, 2026Gordon Chen, Ziqi Huang, Ziwei LiuVideo GenerationControllable Video Generation

    S-Lab, Nanyang Technological University

  8. Rays as Pixels: Learning A Joint Distribution of Videos and Camera Trajectories

    Apr 10, 2026Wonbong Jang, Shikun Liu, Soubhik Sanyal +6Camera-Controlled Video GenerationVideo Diffusion Models

    Meta AI, London, United Kingdom

  9. CT-1: Vision-Language-Camera Models Transfer Spatial Reasoning Knowledge to Camera-Controllable Video Generation

    Apr 10, 2026Haoyu Zhao, Zihao Zhang, Jiaxi Gu +10Camera-Controlled Video GenerationVideo Diffusion Models

    Fudan University · Tencent · Xiamen University +1

  10. When Numbers Speak: Aligning Textual Numerals and Visual Instances in Text-to-Video Diffusion Models

    Apr 9, 2026Zhengyang Sun, Yu Chen, Xin Zhou +4T2V GenerationVideo Diffusion Models

    Huazhong University of Science and Technology · Afari Intelligent Drive · Zhejiang University

  11. InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptation

    Apr 9, 2026Zhefan Rao, Bin Zou, Haoxuan Che +5Instruction-Based Video EditingDiffusion-Based Video Editing

    The Hong Kong University of Science and Technology · Celia Research HK · City University of Hong Kong

  12. FrameCrafter: Novel View Synthesis as Video Completion

    Apr 9, 2026Qi Wu, Khiem Vuong, Minsik Jeon +2Video Diffusion ModelsNovel View Synthesis

    Carnegie Mellon University

  13. SyncBreaker:Stage-Aware Multimodal Adversarial Attacks on Audio-Driven Talking Head Generation

    Apr 9, 2026Wenli Zhang, Xianglong Shi, Sirui Zhao +5Adversarial AttacksAudio-Driven Facial Animation

  14. Uni-ViGU: Towards Unified Video Generation and Understanding via A Diffusion-Based Video Generator

    Apr 9, 2026Luozheng Qin, Jia Gong, Qian Qiao +6Unified Multimodal ModelsVideo Generation

    Shanghai Academy of AI for Science · Independent Researcher · Singapore University of Technology and Design +1

  15. Lighting-grounded Video Generation with Renderer-based Agent Reasoning

    Apr 9, 2026Ziqi Cai, Taoyu Yang, Zheng Chang +4Controllable Video GenerationVideo Diffusion Models

    State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University · National Engineering Research Center of Visual Technology, School of Computer Science, Peking University · OpenBayes Information Technology Co., Ltd. +2

  16. ImVideoEdit: Image-learning Video Editing via 2D Spatial Difference Attention Blocks

    Apr 9, 2026Jiayang Xu, Fan Zhuo, Majun Zhang +6Video EditingVideo Diffusion Models

    Zhejiang University

  17. LPM 1.0: Video-based Character Performance Model

    Apr 9, 2026Ailing Zeng, Casper Yang, Chauncey Ge +22Audio-Video GenerationReal-Time Interactive Video Generation

  18. MoZoo:Unleashing Video Diffusion power in animal fur and muscle simulation

    Apr 8, 2026Dongxia Liu, Jie Ma, Xiaochen Yang +7Video Diffusion ModelsControllable Video Generation

    Tsinghua University · HUIJING Digital Media & Entertainment Group · University of Glasgow +1

  19. Dense-Prior-Guided Generative Video Compression

    Apr 8, 2026Ding Ding, Daowen Li, Yixin Gao +4Generative Video CompressionVideo Diffusion Models

    Alibaba Group · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center

  20. Evolution of Video Generative Foundations

    Apr 7, 2026Teng Hu, Jiangning Zhang, Hongrui Huang +7Multimodal Video GenerationVideo Generation

  21. HumANDiff: Articulated Noise Diffusion for Motion-Consistent Human Video Generation

    Apr 7, 2026Tao Hu, Varun JampaniVideo GenerationControllable Video Generation

    Stability AI

  22. Vanast: Virtual Try-On with Human Image Animation via Synthetic Triplet Supervision

    Apr 6, 2026Hyunsoo Cha, Wonjung Woo, Byungjun Kim +1Virtual Try-OnHuman Avatar Animation

    Seoul National University

  23. Beyond Few-Step Inference: Accelerating Video Diffusion Transformer Model Serving with Inter-Request Caching Reuse

    Apr 6, 2026Hao Liu, Ye Huang, Chenghuan Huang +5Diffusion Model CachingDiffusion Model Inference Acceleration

    Sun Yat-sen University, China · Independent Researcher

  24. CRAFT: Video Diffusion for Bimanual Robot Data Generation

    Apr 4, 2026Jason Chen, I-Chun Arthur Liu, Gaurav Sukhatme +1Synthetic Data Generation for RoboticsBimanual Robotic Manipulation

    Thomas Lord Department of Computer Science at the University of Southern California, USA

  25. Not All Frames Deserve Full Computation: Accelerating Autoregressive Video Generation via Selective Computation and Predictive Extrapolation

    Apr 3, 2026Hanshuai Cui, Zhiqing Tang, Zhi Yao +3Autoregressive Video GenerationDiffusion Model Inference Acceleration

    School of Artificial Intelligence, Beijing Normal University, Beijing, China · Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai, China · Shenzhen University of Advanced Technology, Shenzhen, China

  26. MMPhysVideo: Physically Plausible Video Generation Through Joint RGB-Perception Modeling

    Apr 3, 2026Shubo Lin, Xuanyang Zhang, Wei Cheng +3Physical Consistency in Video GenerationVideo Diffusion Models

    State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA · School of Artificial Intelligence, University of Chinese Academy of Sciences · StepFun +2

  27. Generative AI for Video Trailer Synthesis: From Extractive Heuristics to Autoregressive Creativity

    Apr 3, 2026Abhishek Dharmaratnakar, Srivaths Ranganathan, Debanshu Das +1Video GenerationT2V Generation

  28. ActionParty: Multi-Subject Action Binding in Generative Video Games

    Apr 2, 2026Alexander Pondaven, Ziyi Wu, Igor Gilitschenski +4Video World ModelsVideo Diffusion Models

    Snap Research · University of Oxford · University of Toronto +1

  29. Control-DINO: Feature Space Conditioning for Controllable Image-to-Video Diffusion

    Apr 2, 2026Edoardo A. Dominici, Thomas Deixelberger, Konstantinos Vardis +1Image-to-Video GenerationVideo Diffusion Models

    Huawei, Switzerland · Huawei, Austria · Graz University of Technology, Austria

  30. Can Video Diffusion Models Predict Past Frames? Bidirectional Cycle Consistency for Reversible Interpolation

    Apr 2, 2026Lingyu Liu, Yaxiong Wang, Li Zhu +1Video Frame InterpolationVideo Diffusion Models

    School of Software, Xi’an Jiao-tong University, Xi’an, 710049, China. · School of Computer and Information Science, Hefei University of Technology, Jianghuai Advance Technology Center, Anhui Provincial Key Laboratory of Humanoid Robots. Anhui Provincial Industry Innovation Center of Humanoid Robots, Hefei, 230000, China. · Faculty of Science and Technology, and Institute of Collaborative Innovation, University of Macau, Macau, 999078, China.