T2V Generation

T2V: Text-to-Video

Text-to-video generation studies how to turn language into coherent video, enabling uses such as storytelling, instruction, and creative production. Most systems build on diffusion transformers or flow-matching models, with prompt refinement, inference-time guidance, reference conditioning, and preference-based post-training used to improve control. Evaluation combines visual-quality and prompt-alignment measures such as VBench and T2V-CompBench with targeted tests of physics, temporal events, identity, and spatial relations. Models can produce convincing appearances, but reliable motion, physical interactions, and fine-grained prompt compliance remain difficult, while inference cost and safety add deployment challenges.

Latest papers

110 papers14 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

SafeGen-Bench: Benchmarking Safety in Image-Conditioned Text-to-Video Generation
May 31, 2026Yingzi Ma, Xiaogeng Liu, Yawen Zheng +1Image-to-Video Generation T2V Generation
University of Wisconsin-Madison, Madison · Tsinghua University · Johns Hopkins University
Knowledge-Intensive Video Generation
May 31, 2026Chenxu Wang, Mingda ChenT2V Generation Video Generation
Fudan University · Shanghai Jiao Tong University
Boundary-Protection W8A8 HiFloat8 Quantization for Large-Scale Text-to-Video Diffusion Transformers
May 31, 2026Yiming ZhaoVideo Diffusion Models Diffusion Model Quantization
University of Chinese Academy of Sciences Beijing, China
TunerDiT: Training-free Progressive Steering of Diffusion Transformer for Multi-Event Video Generation
May 29, 2026Ruotong Liao, Guowen Huang, Qing Cheng +6Diffusion Transformer T2V Generation
Ludwig Maximilian University of Munich · MCML · Technical University of Munich +2
KGEdit: Ambiguity-Aware Knowledge Graphs for Training-Free Precise Video Generation and Editing
May 28, 2026Mingshu Cai, Miao Zhang, Chenghe Yang +3Video Diffusion Models T2V Generation
Waseda University, Japan · College of Computer Engineering, Jimei University, Xiamen, Fujian, China · Department of Language Science and Technology, The Hong Kong Polytechnic University, Hong Kong, China +1
OSP-Next: Efficient High-Quality Video Generation with Sparse Sequence Parallelism, HiF8 Quantization, and Reinforcement Learning
May 27, 2026Yunyang Ge, Xianyi He, Zezhong Zhang +4Video Diffusion Models T2V Generation
1Peking University · 2Nanyang Technological University, Singapore · 3Rabbitpre AI
Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation
May 27, 2026Mariam Hassan, Kaouther Messaoud, Wuyang Li +1Physical Consistency in Video Generation Image-to-Video Generation
1 École Polytechnique Fédérale de Lausanne (EPFL) · Télécom Paris, IP Paris
Paris 2.0: A Decentralized Diffusion Model for Video Generation
May 25, 2026Ali Rouzbayani, Bidhan Roy, Marcos Villagra +1Video Diffusion Models T2V Generation
Bagel Labs (bagel.com)
MAVEN A Multi-Agent Framework for Multicultural Text-to-Video Generation
May 16, 2026Shuowei Li, Yuming Zhao, Parth Bhalerao +1T2V Generation Multi-Agent Systems
Santa Clara University, Santa Clara, USA
Compositional Video Generation via Inference-Time Guidance
May 14, 2026Ariel Shaulov, Eitan Shaar, Amit Edenzon +2Diffusion Model Guidance Video Diffusion Models
1Tel-Aviv University · 2Independent Researcher · 3Bar Ilan University +1
FaithfulFaces: Pose-Faithful Facial Identity Preservation for Text-to-Video Generation
May 6, 2026Yuanzhi Wang, Xuhua Ren, Jiaxiang Cheng +7T2V Generation Identity-Preserving Video Generation
Nanyang Technological University · Tencent Hunyuan · University of Science and Technology of China +1
TrajShield: Trajectory-Level Safety Mediation for Defending Text-to-Video Models Against Jailbreak Attacks
May 3, 2026Quanchen Zou, Nizhang Li, Wenxin Zhang +4T2V Generation
AI Security Lab, Beijing, China · Macau University of Science and Technology, Macau, China · University of Chinese Academy of Sciences, Beijing, China +3
World-R1: Reinforcing 3D Constraints for Text-to-Video Generation
Apr 27, 2026Weijie Wang, Xiaoxuan He, Youping Gu +9Physical Consistency in Video Generation T2V Generation
Zhejiang University · Microsoft Research · Independent Researcher
BRITE: A Benchmark for Reliable and Interpretable T2V Evaluation on Implausible Scenarios
Apr 24, 2026Advait Tilak, Jiwon Choi, Nazifa Mouli +1T2V Generation Audio-Visual Alignment
Department of Computer Science, Iowa State University, Ames, Iowa, USA
KD-CVG: A Knowledge-Driven Approach for Creative Video Generation
Apr 23, 2026Linkai Liu, Wei Feng, Xi Zhao +9Multimodal RAG T2V Generation
† Sun Yat-sen University · ⋆ JD.COM
TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generation
Apr 21, 2026Hongyu Zhang, Yufan Deng, Zilin Pan +6Image-to-Video Generation T2V Generation
Peking University, Shenzhen Graduate School · Zhejiang University · Nanjing University +3
Training and Agentic Inference Strategies for LLM-based Manim Animation Generation
Apr 20, 2026Ravidu Suien Rammuni Silva, Ahmad Lotfi, Isibor Kennedy Ihianle +2RL for Code Generation T2V Generation
Department of Computer Science, Nottingham Trent University, Nottingham, United Kingdom · Department of Physics and Mathematics, Nottingham Trent University, Nottingham, United Kingdom
FlowC2S: Flowing from Current to Succeeding Frames for Fast and Memory-Efficient Video Continuation
Apr 19, 2026Hovhannes Margaryan, Quentin Bammey, Christian SandorFlow Matching T2V Generation
Team ARAI, Universit´e Paris-Saclay, CNRS, LISN, France · LTCI, T´el´ecom Paris, Institut Polytechnique de Paris, France
Seedance 2.0: Advancing Video Generation for World Complexity
Apr 15, 2026Team Seedance, De Chen, Liyang Chen +168Audio-Video Generation Multimodal Generation
Generative Refinement Networks for Visual Synthesis
Apr 14, 2026Jian Han, Jinlai Liu, Jiahuan Wang +2Binary Quantization T2V Generation
ByteDance
Motif-Video 2B: Technical Report
Apr 14, 2026Junghwan Lim, Wai Ting Cheung, Minsu Ha +25T2V Generation Video Foundation Models
Motif Technologies
When Numbers Speak: Aligning Textual Numerals and Visual Instances in Text-to-Video Diffusion Models
Apr 9, 2026Zhengyang Sun, Yu Chen, Xin Zhou +4T2V Generation Video Diffusion Models
Huazhong University of Science and Technology · Afari Intelligent Drive · Zhejiang University
SceneScribe-1M: A Large-Scale Video Dataset with Comprehensive Geometric and Semantic Annotations
Apr 9, 2026Yunnan Wang, Kecheng Zheng, Jianyuan Wang +8T2V Generation Camera-Controlled Video Generation
Shanghai Jiao Tong University · Ant Group · Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo +2
Not all tokens contribute equally to diffusion learning
Apr 8, 2026Guoqing Zhang, Lu Shi, Wanru Xu +4Diffusion Model Guidance Attention Control in Diffusion Models
State Key Laboratory of Advanced Rail Autonomous Operation, Bejing Jiaotong University, Beijing, China · School of Computer Science and Technology, Bejing Jiaotong University, Beijing, China · Visual Intellgence +X International Cooperation Joint Laboratory of MOE, Bejing Jiaotong University, Beijing, China +3
SCMAPR: Self-Correcting Multi-Agent Prompt Refinement for Complex-Scenario Text-to-Video Generation
Apr 7, 2026Chengyi Yang, Pengzhen Li, Jiayin Qi +3Prompt Optimization for T2V Generation T2V Generation
HiThink Research · East China Normal University · Guangzhou University +1
Generative AI for Video Trailer Synthesis: From Extractive Heuristics to Autoregressive Creativity
Apr 3, 2026Abhishek Dharmaratnakar, Srivaths Ranganathan, Debanshu Das +1Video Generation T2V Generation
TokenDial: Continuous Attribute Control for Text-to-Video Generation in Visual Dial Space
Mar 29, 2026Zhixuan Liu, Peter Schaldenbrand, Yijun Li +5Controllable Video Generation T2V Generation
Adobe Research · Carnegie Mellon University
P-Flow: Prompting Visual Effects Generation
Mar 23, 2026Rui Zhao, Mike Zheng ShouControllable Video Generation T2V Generation
Show Lab, National University of Singapore
Depictions of Depression in Generative AI Video Models: A Preliminary Study of OpenAI's Sora 2
Mar 19, 2026Matthew Flathers, Griffin Smith, Julian Herpertz +2Generative Modeling T2V Generation
Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Boston, MA, USA · Rhode Island School of Design, Providence, RI, USA · Department of Psychiatry and Neuroscience, Charité Berlin University Medicine, Campus Benjamin Franklin, Berlin, Germany +2
Early Failure Detection and Intervention in Video Diffusion Models
Mar 15, 2026Kwon Byung-Ki, Sohwi Lim, Nam Hyeon-Woo +2Video Diffusion Models T2V Generation
Grad. School of Artificial Intelligence, POSTECH · School of Electrical Engineering, KAIST · Dept. of Electrical Engineering, POSTECH +1

May 31, 2026J/K move · Enter open · S save

SafeGen-Bench: Benchmarking Safety in Image-Conditioned Text-to-Video Generation

Yingzi Ma, Xiaogeng Liu, Yawen Zheng, Chaowei Xiao

University of Wisconsin-Madison, Madison · Tsinghua University · Johns Hopkins University

Image-to-Video Generation T2V Generation Video Generation Video Generation Evaluation

Open paper AskPDF arXiv

With the rapid advancements in text-to-image diffusion models, generative video models (T2V models) like Sora can now produce short synthetic videos from a text prompt or an initial image. However, synthetic video generation -- especially when guided by an initial image -- often poses risks, including the potential creation of illegal, politically sensitive, or unethical content. Existing benchmarks have started to consider the safety of generated videos, but they primarily focus on testing models with malicious text prompts, ignoring the scenario where text prompt and image combination may still lead to harmful video content. In practice, this is a common and challenging issue: videos generated from safe text and image inputs can nonetheless convey harmful information. To bridge this gap, we introduce SafeGen-Bench, a benchmark specifically designed to evaluate the safety of conditional T2V models. Our benchmark defines 10 malicious categories, concentrating on risks related to both temporal sequences and depicted behaviors. SafeGen-Bench consists of carefully selected start frames from diverse image and video sources, paired with corresponding text prompts to simulate realistic inputs. We evaluate a variety of conditional T2V models on SafeGen-Bench, and the results indicate that current models struggle to consistently avoid generating malicious content with unsafety scores reaching up to 44.5, especially under conditions requiring high quality. Furthermore, we assess the effectiveness of both text-based and image-based guardrails on our benchmark, finding that unimodal guardrails alone were insufficient to provide a robust defense, with an 80% failure rate across seven malicious categories. We hope that SafeGen-Bench will foster the development of safer and more controllable conditional T2V models.