Video Diffusion Models

Follow this topic Ask about this topic

Video diffusion models synthesize or transform frame sequences by denoising video latents, often in diffusion transformers, conditioned on text, images, reference clips, or structured controls. Researchers use temporal attention, autoregressive chunking, and pose, camera, depth, or physics cues to improve motion, identity, geometry, and interaction; related systems support editing, restoration, avatars, and world-model prediction. Evaluation spans prompt alignment and visual quality, using benchmarks such as VBench and FVD, as well as controllability, physical plausibility, and downstream action performance. A central challenge is balancing fidelity and precise control over long videos with the speed and memory needed for streaming, while limiting drift and physically implausible events.

Latest papers

738 papers122 papers in the last four weeks, up 171% on the four weeks before. 0.7% of all new papers.

Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation
May 27, 2026Mariam Hassan, Kaouther Messaoud, Wuyang Li +1Physical Consistency in Video Generation Image-to-Video Generation
1 École Polytechnique Fédérale de Lausanne (EPFL) · Télécom Paris, IP Paris
PARE: Pruning and Adaptive Routing for Efficient Video Generation
May 26, 2026Yutong Wang, Yunke Wang, Tianfan Xue +4Video Diffusion Models Diffusion Transformer
The University of Sydney · The Chinese University of Hong Kong · Shanghai AI Laboratory
Timestep-Aware SVDQuant-GPTQ for W4A4 Quantization of Wan2.2-I2V
May 26, 2026Junhao Wu, Dezhong Yao, Hai JinVideo Diffusion Models Diffusion Transformer
School of Computer Science and Technology, Huazhong University of Science and Technology Wuhan, Hubei, China 430074
Tail-Aware HiFloat4: W4A4 Post-Training Quantization for Wan2.2
May 26, 2026Zhanfeng Feng, Shuai Guo, Xin Di +3Video Diffusion Models 4-Bit Quantization
University of Science and Technology of China Hefei, China
E $^3$ C: Video Generation with 3D Environmental Memory and Ego-Exo Human Pose Control
May 25, 2026Qiao Gu, Lingni Ma, Adam W Harley +3Video Diffusion Models Egocentric Vision
Meta Reality Labs, Redmond, WA 98052, USA · University of Toronto, Toronto, ON M5S 1A1, Canada
Quantized Keys Steal Attention: Bias Correction for KV-Cache Compression in Video Diffusion
May 25, 2026Tuna Tuncer, Felix Becker, Thomas PfeilVideo Diffusion Models Self-Attention
1Technical University of Munich · 2Tensordyne
Paris 2.0: A Decentralized Diffusion Model for Video Generation
May 25, 2026Ali Rouzbayani, Bidhan Roy, Marcos Villagra +1Video Diffusion Models T2V Generation
Bagel Labs (bagel.com)
Where Concept Erasure Should Occur: Concept-Layer Alignment in Text-to-Video Diffusion Models
May 25, 2026Yiwei Xie, Ping Liu, Zheng ZhangVideo Diffusion Models Diffusion Transformer
The School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China · Department of Computer Science and Engineering, University of Nevada, Reno, NV, USA.
How Accurate are Video Quality Models for Diffusion-Based Video Super-Resolution?
May 25, 2026Benjamin Herb, Steve Göring, Alexander Raake +1Video Diffusion Models Video Quality Assessment
Audiovisual Technology Group, Technische Universität Ilmenau, Germany · Institute for Communications Engineering, RWTH Aachen University, Germany
PixelWizard: Towards Efficient High-Fidelity Video Generation at Ultra-Large Spatial Resolution
May 25, 2026Wenxue Li, Jingjing Ren, Peng Zhang +6Video Generation High-Resolution Video Generation
The Hong Kong University of Science and Technology (Guangzhou) · MiLM Plus, Xiaomi Inc · The Hong Kong University of Science and Technology
Full-4D: Generating Full-Scope 4D Scenes from a Single-View Video
May 25, 2026Tingxi Chen, Ke Hao, Yabo Chen +6Camera-Conditioned Video Generation 4D Reconstruction
Shanghai Jiao Tong University · Institute of Artificial Intelligence, China Telecom (TeleAI)
Pantheon360: Taming Digital Twin Generation via 3D-Aware 360° Video Diffusion
May 25, 2026Ting-Hsuan Chen, Ying-Huan Chen, Tao Tu +10Video-Reference Video Generation Camera-Controlled Video Generation
University of Southern California · National Yang Ming Chiao Tung University · Cornell University +1
Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution
May 25, 2026Tianshuo Xu, Yichen Xie, Depu Meng +5Video Diffusion Models Long-Horizon Video Generation
Applied Intuition · University of California, Berkeley
DeltaCam: Differential Intrinsic Camera Modeling for Video Generation
May 24, 2026Debabrata Mandal, Zhihan Peng, Yujie Wang +1Camera-Conditioned Video Generation Video Diffusion Models
UNC, Chapel Hill, USA
Reasoning to Align: Implicit Reasoning in Diffusion Transformers for Video Editing
May 23, 2026Yan Li, Lin Liu, Xiaopeng Zhang +1Video Diffusion Models Diffusion Transformer
†The Hongkong University of Science and Technology · ‡Huawei Inc.
DexSIM: Real-time Dexterous Simulation with Unified Causal Video Diffusion
May 23, 2026Adam LeeVideo Diffusion Models Real-Time Interactive Video Generation
UC Berkeley
Φ-Noise: Training-Free Temporal Video Conditioning via Phase-Based Noise Manipulation
May 23, 2026Ofir Abramovich, Nadav Z. Cohen, Adi Rosenthal +1Video Diffusion Models Diffusion Models for Motion Generation
Canvas-Lab Department of Computer Science Reichman University
Loki: Representation over Architecture for Diffusion-Based Portrait Animation
May 22, 2026Pouyan Navard, Sernam LimVideo Diffusion Models 3D Morphable Models
The Ohio State University · University of Central Florida
Smart-Insertion-V: Photorealistic Video Insertion via a Closed-Loop Feedback Dual-Stream Framework
May 22, 2026Xiao Cao, Yansong Qu, Xiangzhen Chang +6Video Diffusion Models Video Editing
LaMo: Self-Supervised Latent Motion Priors for Physical Realism in Video Generation
May 22, 2026Bo Jiang, Depu Meng, Yihan Hu +3Physical Consistency in Video Generation Diffusion Model Guidance
Applied Intuition · University of California, Berkeley
One-Forcing: Towards Stable One-Step Autoregressive Video Generation
May 22, 2026Jiaqi Feng, Justin Cui, Yuanhao Ban +1Autoregressive Video Generation One-Step Generative Modeling
Tsinghua University · UCLA
DFSAttn: Dynamic Fine-grained Sparse Attention for Efficient Video Generation
May 22, 2026Jie Hu, Zixiang Gao, Yutong He +1Video Diffusion Models Diffusion Transformer
1Peking University.
SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models
May 22, 2026Zizhao Tong, Yeying Jin, Hongfeng Lai +11Action-Conditioned Video Generation Video Diffusion Models
UCAS-Terminus AI Lab, University of Chinese Academy of Sciences · Tencent · National University of Singapore +6
SimInsert: Seamless Video Object Insertion via Regional Sparse Attention Fusion
May 22, 2026Xinyu Chen, Yuyi Qian, Jiang Lin +9Video Diffusion Models Video Editing
1State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · School of Intelligence Science and Technology, Nanjing University, Suzhou, China · 4Xi’an Jiaotong-Liverpool University +3
Occlusion-Aware Physics-Semantic Keyframe Selection for Robust Video Editing
May 22, 2026Lin Liu, Zhihan Xiao, Haohang Xu +4Video Diffusion Models Video Editing
Huawei, China · Tsinghua University, China · East China Normal University, China
Bernini: Latent Semantic Planning for Video Diffusion
May 21, 2026Bernini Team, Chenchen Liu, Junyi Chen +9Video Diffusion Models Unified Multimodal Models
Bernini Team, Bytedance · Bytedance
ORBIS: Output-Guided Token Reduction with Distribution-Aware Matching for Video Diffusion Acceleration
May 21, 2026Hangyeol Lee, Joo-Young KimToken Merging Video Diffusion Models
KAIST · Daejeon, South Korea
BodyReLux: Temporally Consistent Full-Body Video Relighting
May 20, 2026Li Ma, Mingming He, Xueming Yu +4Video Diffusion Models Controllable Video Generation
Eyeline Labs, United States of America · Eyeline Labs, Canada · Eyeline Labs +2
iTryOn: Mastering Interactive Video Virtual Try-On with Spatial-Semantic Guidance
May 20, 2026Jun Zheng, Zhengze Xu, Mengting Chen +6HOI Generation Virtual Try-On
Shenzhen Campus of Sun Yat-sen University · Taobao & Tmall Group of Alibaba
Q-ARVD: Quantizing Autoregressive Video Diffusion Models
May 20, 2026Siao Tang, Xinyin Ma, Gongfan Fang +2Video Diffusion Models Diffusion Model Quantization
National University of Singapore · The Hong Kong Polytechnic University

May 27, 2026J/K move · Enter open · S save

Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation

Mariam Hassan, Kaouther Messaoud, Wuyang Li, Alexandre Alahi

1 École Polytechnique Fédérale de Lausanne (EPFL) · Télécom Paris, IP Paris

Physical Consistency in Video Generation Image-to-Video Generation Video Diffusion Models T2V Generation

Open paper AskPDF arXiv

Modern video generative models produce visually impressive results, yet frequently violate basic physical principles. We propose Proprio, a training-free framework that enables a frozen video generator to assess and improve the physical plausibility of its own outputs. Inspired by proprioception, the biological sense of one's own movement, Proprio treats the model's flow residual under controlled latent perturbations as a self-scoring signal. Samples that are better explained by the generator's learned dynamics induce smaller and more stable residuals. We aggregate this signal across timesteps and perturbations, focus it on motion-relevant regions with a dynamic spatiotemporal mask, and use it for best-of-N search, gradient-based self-refinement, or both. Across text-to-video and image-to-video benchmarks, Proprio consistently improves physical plausibility, outperforming VLM-based scoring, and external world-model baselines in several settings. With TurboWan2.2, Proprio improves Physics-IQ from 32.2 to 37.5 (+16.5%) and VideoPhy2-hard physical commonsense from 45.6 to 55.0 (+20.6%). Human evaluation further shows that raters prefer Proprio-selected or refined videos for physical plausibility in roughly two-thirds of comparisons. These results suggest that frozen video generators contain actionable internal signals for evaluating and improving the physical plausibility of their own outputs.