Video Diffusion Models

Follow this topic Ask about this topic

Video diffusion models synthesize or transform frame sequences by denoising video latents, often in diffusion transformers, conditioned on text, images, reference clips, or structured controls. Researchers use temporal attention, autoregressive chunking, and pose, camera, depth, or physics cues to improve motion, identity, geometry, and interaction; related systems support editing, restoration, avatars, and world-model prediction. Evaluation spans prompt alignment and visual quality, using benchmarks such as VBench and FVD, as well as controllability, physical plausibility, and downstream action performance. A central challenge is balancing fidelity and precise control over long videos with the speed and memory needed for streaming, while limiting drift and physically implausible events.

Latest papers

738 papers122 papers in the last four weeks, up 171% on the four weeks before. 0.7% of all new papers.

PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation
Jun 26, 2026Peiwen Zhang, Yufan Deng, Shangkun Sun +11Physical Consistency in Video Generation Video Diffusion Models
Peking University, NVIDIA
SIFT: Self-Imagination Fine-Tuning for Physically Plausible Motion in Video Diffusion Models
Jun 26, 2026Ruoyu Wang, Jialun Liu, Huayang Huang +5Physical Consistency in Video Generation Video Diffusion Models
School of Computer Science, Wuhan University · Institute of Artificial Intelligence, China Telecom (TeleAI) · School of Artificial Intelligence, Wuhan University
PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generation
Jun 25, 2026Kexu Cheng, Zicheng Liu, Mingju Gao +2Multimodal RAG Physical Consistency in Video Generation
School of Computer Science, Peking University, Beijing, China · Institute of AI for Industries, Chinese Academy of Sciences, Nanjing, China
NaviCache: Test-Time Self-Calibration Caching for Video Generation
Jun 25, 2026Zheqi Lv, Zhibo Zhu, Jinke Wang +6Video Diffusion Models Diffusion Model Caching
Zhejiang University, Hangzhou, China · Cornell University, New York, USA · Tencent Hunyuan, Shenzhen, China +1
LearniBridge: Learnable Calibration of Feature Caching for Diffusion Models Acceleration
Jun 25, 2026Xuyue Huang, Zhe Chen, Wang Shen +1Video Diffusion Models Diffusion Transformer
Shenzhen Ubiquitous Data Enabling Key Laboratory, Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Central Media Technology Institute, Huawei Technologies Co., Ltd., Shenzhen, China
LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing
Jun 25, 2026Xinyu Wang, Chongbo Zhao, Fangneng Zhan +1Video Diffusion Models Video Editing
THU · HKUST
Disco-LoRA: Disentangled Composition of Content, Style, and Motion for Multi-concept Video Customization
Jun 25, 2026Xuancheng Xu, Gengyun Jia, Bing-Kun BaoDisentangled Representation Learning Video Diffusion Models
Nanjing University of Posts and Telecommunications · Hefei University of Technology · Peng Cheng Laboratory
TryOnCrafter: Unleashing Camera Trajectories for Realistic Video Virtual Try-on via a Renderable 4D Try-on Proxy
Jun 24, 2026Hao Sun, Hao Yan, Mengting Chen +7Camera-Controlled Video Generation Virtual Try-On
Institute of Computing Technology, Chinese Academy of Sciences · Taobao & Tmall Group of Alibaba · University of Chinese Academy of Sciences +1
MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation
Jun 24, 2026JoungBin Lee, Jaewoo Jung, Jongmin Lee +8Multi-View Consistency Camera-Conditioned Video Generation
1KAIST AI · 2Sony AI · 3Sony Group Corporation
Chorus II: Cross-Request Sparsity Reuse for Efficient Image-to-Video Generation
Jun 23, 2026Hao Liu, Chenghuan Huang, Hao Liu +6Image-to-Video Generation Video Diffusion Models
Sun Yat-sen University · WeChat HPC, Tencent Inc. · WeChat Vision, Tencent Inc.
FLAT: Feedforward Latent Triangle Splatting for Geometrically Accurate Scene Generation
Jun 23, 2026Orest Kupyn, Goutam Bhat, Philipp Henzler +3Video Diffusion Models 3D Scene Generation
1Google Research · University of Oxford, Visual Geometry Group · 3TU Munich
Vera: A Layered Diffusion Model for Content-Preserving Video Editing
Jun 22, 2026Hongkai Zheng, Ta-Ying Cheng, Benjamin Klein +2Video Diffusion Models Diffusion Transformer
California Institute of Technology · Netflix, Inc.
Ocean4D: Generative Underwater 4D Reconstruction via Medium-Aware Video Diffusion
Jun 22, 2026Yuqiang Huang, Yuxi Wang, Junyu Dong +1Camera-Conditioned Video Generation 4D Reconstruction
Faculty of Information Science and Engineering, Ocean University of China · Institute of Automation, Chinese Academy of Sciences
SteerVTE: Seamless Video Text Editing with Style and Glyph Control
Jun 22, 2026Kai Zeng, Moran Li, Zhengwei Wang +6Video Diffusion Models Video Editing
Peking University · ByteDance China · ByteDance Singapore
ScalingAttention: Discovering Intrinsic Sparse Attention Topology for Video Diffusion Transformers
Jun 22, 2026Ruiliang Zhou, Xuecheng Wu, Kang He +6Video Diffusion Models Diffusion Transformer
KlingAI Research · Beijing Institute of Technology · NVIDIA +1
ZeroGVC: Zero-Shot Generative Video Compression with Autoregressive Diffusion Priors
Jun 21, 2026Yixin Gao, Xiaohan Pan, Lin Liu +3Video Diffusion Models Neural Video Compression
University of Science and Technology of China · 2Huawei Inc.
Towards Error-Free Long Video Generation
Jun 21, 2026Shuning Chang, Weihua Chen, Jiasheng Tang +8Video Diffusion Models Long Video Generation
Zhejiang University · Alibaba Group
CoDMD: Copula-aware Distribution Matching Distillation for Fast Video Generation
Jun 20, 2026Wenhu Zhang, Kun Cheng, Changyuan Wang +7Video Diffusion Models Diffusion Model Distillation
1HKUST · 2Wan Team, Alibaba Group · 3THU +2
ChronoLock: Protecting Videos from Unauthorized Text-to-Video Personalization
Jun 19, 2026Jiaming He, Jiashu Zhang, Guanyu Hou +4Video Diffusion Models T2V Generation
Nanyang Technological University · Jilin University · The University of Manchester
Go-with-the-Track: Video Compositing and Motion Control with Point Tracking
Jun 18, 2026Koichi Namekata, Yash Kant, Zhizheng Liu +9Point Tracking Camera-Controlled Video Generation
Netflix, USA · Eyeline Labs, USA · University of Oxford, United Kingdom +3
Through the PRISM: Preference Representation in Intermediate States of Video Diffusion Models
Jun 18, 2026Haoxuan Wu, Lai Man Po, Mengyang Liu +3Video Diffusion Models Video Generation Evaluation
City University of Hong Kong · Video Rebirth · The Chinese University of Hong Kong
FrozenDrive: Zero-Shot Text-Guided Driving Scene Generation and Data Augmentation with Parameter-Free Frozen Diffusion Model
Jun 18, 2026Yuhwan Jeong, Hyeonseong Kim, Daehyun We +5Synthetic Data Augmentation Video Diffusion Models
KAIST, Visual Intelligence Lab
SketchKeyAnime: Reference-anchored Sparse Key-Sketch Animation Synthesis
Jun 18, 2026Meixi Li, Xianlin Zhang, Yue Zhang +1Image-to-Video Generation Video Diffusion Models
Beijing University of Posts and Telecommunications
TeleMorpher: Toward Robust Simultaneous Motion-Location Editing
Jun 18, 2026Haengbok ChungVideo Diffusion Models Video Editing
Cinematic Compositing Using Character-Environment-Harmonized Video Generation Models
Jun 17, 2026Tianyi Xiang, Mingming He, Li Ma +1Physical Consistency in Video Generation Video Diffusion Models
1City University of Hong Kong · 2Independent Researcher
Data-Forcing Distillation: Restoring Diversity and Fidelity in Few-Step Video Generation
Jun 16, 2026Siyi Chen, Shaowei Liu, Yixuan Jia +4Video Diffusion Models Diffusion Model Distillation
University of Michigan · NVIDIA · University of Illinois Urbana-Champaign
AoiZora: Topology-Aware Auto-Parallel Optimization for Inference of Diffusion Transformers
Jun 16, 2026Kaijian Wang, Yuanyuan Xu, Fanjiang Ye +5AI Accelerator Inference Video Diffusion Models
Rice University · Independent Researcher · Google
SierpinskiCam: Camera-Controlled Video Retaking with Sierpinski Triangle Pattern Cues
Jun 15, 2026Suttisak Wizadwongsa, Hyelin Nam, Supasorn Suwajanakorn +1Camera-Controlled Video Generation Video Diffusion Models
University of Michigan, Ann Arbor · VISTEC, Thailand
Pulling The REINS: Training-Free Safety Alignment of Video Diffusion Models via Representation Steering
Jun 15, 2026Rohit Kundu, Arindam Dutta, Sarosij Bose +2Diffusion Model Alignment Video Diffusion Models
University of California, Riverside · YouTube (Google)
Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation
Jun 15, 2026Jie Zhang, Xiaoyue Chen, Anzhe Chen +36Action-Conditioned Video Generation Video Diffusion Models
Qwen Team

Jun 26, 2026J/K move · Enter open · S save

PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation

Peiwen Zhang, Yufan Deng, Shangkun Sun, Juncheng Ma+10

Peking University, NVIDIA

Physical Consistency in Video Generation Video Diffusion Models Robotic Manipulation Embodied World Models

Open paper AskPDF arXiv

Video generation models have emerged as a promising paradigm for embodied world simulation. However, both general-domain video generators and robot-specific data fine-tuned models can still produce physically implausible manipulations, including discontinuous motion trajectories and inconsistent robot-object interactions, which limits their reliability as world simulators. Through extensive experiments, we find that such physical instability mainly arises from two factors: deformation of moving objects and implausible spatio-temporal correlations among interacting entities, particularly during contact. Building on this observation, we propose PhysisForcing, a scalable training framework that strengthens physical consistency by focusing supervision on physics-informative regions through joint optimization of pixel-level and semantic-level features. The framework consists of a pixel-level trajectory alignment loss, which supervises DiT features using reference point trajectories, and a semantic-level relational alignment loss, which aligns DiT features with inter-region relations extracted from a frozen video understanding encoder. Extensive experiments on R-Bench, PAI-Bench, and EZS-Bench show that PhysisForcing consistently improves embodied video generation over strong baselines, improving the Wan2.2-I2V-A14B and Cosmos3-Nano base models on R-Bench by 22.3% and 9.2% (7.1% and 3.7% over vanilla finetuning), with the Cosmos3-Nano variant attaining the best overall score. Beyond generation, as a world model under the WorldArena action-planner protocol it raises the closed-loop success rate from 16.0% to 24.0% and further improves downstream policy success, indicating that physically aligned video models yield stronger representations for robotic manipulation.