Video Editing

Follow this topic Ask about this topic

AI video editing studies how to change footage—such as removing or inserting objects, altering motion or appearance, and assembling clips—while preserving the content that should remain. Common approaches adapt video diffusion and flow-matching models, using text, masks, reference images, keyframes, tracks, or 3D geometry to localize and guide edits; agentic systems also plan edits and operate professional timelines. Work increasingly targets streaming and long-video editing, alongside joint audio-visual changes and physics-aware scene effects. A central challenge is balancing strong instruction following with source fidelity, temporal and cross-shot coherence, and physically plausible results, while benchmarks still expose failures on complex instructions and secondary effects.

Latest papers

178 papers23 papers in the last four weeks, up 156% on the four weeks before. 0.1% of all new papers.

Relit-LiVE: Relight Video by Jointly Learning Environment Video
May 7, 2026Weiqing Xiao, Hong Li, Xiuyu Yang +7Physical Consistency in Video Generation Video Diffusion Models
Nanjing University, China · BAAI, China · BUAA, China +5
Sparkle: Realizing Lively Instruction-Guided Video Background Replacement via Decoupled Guidance
May 7, 2026Ziyun Zeng, Yiqi Lin, Guoqiang Liang +1Synthetic Data Augmentation Video Editing
Show Lab, National University of Singapore
LIVEditor-14B: Lightning Unified Video Editing via In-Context Sparse Attention
May 6, 2026Shitong Shao, Zikai Zhou, Haopeng Li +4Video Editing In-Context Learning
Hong Kong University of Science and Technology (GZ), Hong Kong · University of Arizona, USA
VEBench:Benchmarking Large Multimodal Models for Real-World Video Editing
May 5, 2026Andong Deng, Dawei Du, Zhenfang Chen +7Video Editing Benchmarks Video Editing
ByteDance Intelligent Creation · ByteDance, San Jose, USA · CRCV, University of Central Florida
Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoE
May 4, 2026Yangming Shi, Shixiang Zhu, Tao Shen +14Diffusion Model Inference Acceleration Unified Multimodal Models
Mamoda Team, ByteDance
YOSE: You Only Select Essential Tokens for Efficient DiT-based Video Object Removal
Apr 30, 2026Chenyang Wu, Lina Lei, Fan Li +6Diffusion Transformer Video Editing
1VCIP, CS, Nankai University · 3NKIARI, Shenzhen Futian · 2Huawei Noah’s Ark Lab
Subjective Portrait Region Cropping in Landscape Videos with Temporal Annotation Smoothing
Apr 27, 2026Cheng-Han Lee, Maniratnam Mandal, Neil Birkbeck +3Video Editing
FlowAnchor: Stabilizing the Editing Signal for Inversion-Free Video Editing
Apr 24, 2026Ze Chen, Lan Chen, Yuanhang Li +1Video Editing Temporal Consistency in Video Generation
MIPG, Communication University of China Beijing, China
LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing
Apr 18, 2026Weicheng Wang, Zhicheng Zhang, Zhongqi Zhang +6Video Editing Benchmarks Video Diffusion Models
Nankai University · Kuaishou Technology · Pengcheng Laboratory +1
VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects
Apr 17, 2026Xiangbo Gao, Sicong Jiang, Bangya Liu +12Video Editing Benchmarks Video Editing
1Texas A&M University · 2Visko Platform · 3Abaka AI
UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs
Apr 17, 2026Lifan Jiang, Tianrun Wu, Yuhang Pei +3Video Editing Benchmarks Image Editing Evaluation
Zhejiang University Hangzhou, China
Controllable Video Object Insertion via Multiview Priors
Apr 16, 2026Xia Qi, Peishan Cong, Yichen Yao +3Video Editing Video Object Insertion
ShanghaiTech University
DiffMagicFace: Identity Consistent Facial Editing of Real Videos
Apr 15, 2026Huanghao Yin, Shenkun Xu, Kanle Shi +2Video Editing Instruction-Based Video Editing
Tsinghua University · Kuaishou Technology
VibeFlow: Versatile Video Chroma-Lux Editing through Self-Supervised Learning
Apr 15, 2026Yifan Li, Pei Cheng, Bin Fu +2Video Relighting Video Editing
Wangxuan Institute of Computer Technology, Peking University · Tencent
ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videos
Apr 12, 2026Arjun Somayazulu, Kristen GraumanVideo Editing
The University of Texas at Austin, USA
ImVideoEdit: Image-learning Video Editing via 2D Spatial Difference Attention Blocks
Apr 9, 2026Jiayang Xu, Fan Zhuo, Majun Zhang +6Video Editing Video Diffusion Models
Zhejiang University
Physics-Aware Video Instance Removal Benchmark
Apr 7, 2026Zirui Li, Xinghao Chen, Lingyu Jiang +5Video Editing Instruction-Based Video Editing
Tohoku University · University of Washington · Texas A&M University
GLANCE: A Global-Local Coordination Multi-Agent Framework for Music-Grounded Non-Linear Video Editing
Apr 6, 2026Zihao Lin, Haibo Wang, Zhiyang Xu +7Video Editing Multi-Agent Systems
UC Davis Davis, California, USA · Fudan University Shanghai, China · Stanford University Palo Alto, California, USA +2
DIRECT: Video Mashup Creation via Hierarchical Multi-Agent Planning and Intent-Guided Editing
Apr 6, 2026Ke Li, Maoliang Li, Jialiang Chen +4Video Editing Instruction-Based Video Editing
School of Electronics Engineering and Computer Science, Peking University · School of Computer Science, Peking University · Huazhong University of Science and Technology
LumiVideo: An Intelligent Agentic System for Video Color Grading
Apr 2, 2026Yuchen Guo, Junli Gong, Hongmin Cai +2Agentic Video Editing Automated Grading
Northwestern University, Evanston, USA · Northeastern University, Seattle, USA · South China University of Technology, Guangzhou, China +2
VOID: Video Object and Interaction Deletion
Apr 2, 2026Saman Motamed, William Harvey, Benjamin Klein +3Object Removal Video Editing
Netflix · INSAIT, Sofia University “St. Kliment Ohridski”
From Understanding to Erasing: Towards Complete and Stable Video Object Removal
Apr 2, 2026Dingming Liu, Wenjing Wang, Chen Li +2Object Removal Video Editing
Peking University · WeChat Vision, Tencent Inc.
CutClaw: Agentic Hours-Long Video Editing via Music Synchronization
Mar 31, 2026Shifang Zhao, Yihan Hu, Ying Shan +2Agentic Video Editing Video Editing
Beijing Jiaotong University · GVC Lab, Great Bay University · ARC Lab, Tencent
TrajectoryMover: Generative Movement of Object Trajectories in Videos
Mar 31, 2026Kiran Chhatre, Hyeonho Jeong, Yulia Gryaditskaya +3Synthetic Data Generation Video Editing
KTH Royal Institute of Technology · Adobe Research
AutoCut: End-to-end advertisement video editing based on multimodal discretization and controllable generation
Mar 30, 2026Milton Zhou, Sizhong Qin, Yongzhi Li +2Video Editing Multimodal Large Language Models
Tsinghua University · Kuaishou Technology
LightCtrl: Training-free Controllable Video Relighting
Mar 28, 2026Yizuo Peng, Xuelin Chen, Kai Zhang +1Video Relighting Video Editing
Tsinghua University · GVC Lab, Great Bay University · Adobe Research
AutoWeather4D: Autonomous Driving Video Weather Conversion via G-Buffer Dual-Pass Editing
Mar 27, 2026Tianyu Liu, Weitao Xiong, Kunming Luo +43D Scene Relighting Video Editing
The Hong Kong University of Science and Technology · Xiamen University · Meituan-M17, Hong Kong
TRACE: Object Motion Editing in Videos with First-Frame Trajectory Guidance
Mar 26, 2026Quynh Phung, Long Mai, Cusuh Ham +3Video Editing Controllable Video Generation
University of Maryland, College Park · Adobe Research
Accelerating Diffusion-based Video Editing via Heterogeneous Caching: Beyond Full Computing at Sampled Denoising Timestep
Mar 25, 2026Tianyi Liu, Ye Lu, Linfeng Zhang +5Diffusion-Based Video Editing Diffusion Model Inference Acceleration
Nanyang Technological University · Shanghai Jiao Tong University · The Hong Kong Polytechnic University
GSwap: Realistic Head Swapping with Dynamic Neural Gaussian Field
Mar 24, 2026Jingtao Zhou, Xuan Gao, Dongyu Liu +3Video Editing Identity-Preserving Video Generation
School of Mathematical Science, University of Science and Technology of China · Department of Computer Science, City University of Hong Kong

May 7, 2026J/K move · Enter open · S save

Relit-LiVE: Relight Video by Jointly Learning Environment Video

Weiqing Xiao, Hong Li, Xiuyu Yang, Houyuan Chen+6

Nanjing University, China · BAAI, China · BUAA, China+5

Physical Consistency in Video Generation Video Diffusion Models Video Editing Video Relighting

Open paper AskPDF arXiv

Recent advances have shown that large-scale video diffusion models can be repurposed as neural renderers by first decomposing videos into intrinsic scene representations and then performing forward rendering under novel illumination. While promising, this paradigm fundamentally relies on accurate intrinsic decomposition, which remains highly unreliable for real-world videos and often leads to distorted appearances, broken materials, and accumulated temporal artifacts during relighting. In this work, we present Relit-LiVE, a novel video relighting framework that produces physically consistent, temporally stable results without requiring prior knowledge of camera pose. Our key insight is to explicitly introduce raw reference images into the rendering process, enabling the model to recover critical scene cues that are inevitably lost or corrupted in intrinsic representations. Furthermore, we propose a novel environment video prediction formulation that simultaneously generates relit videos and per-frame environment maps aligned with each camera viewpoint in a single diffusion process. This joint prediction enforces strong geometric-illumination alignment and naturally supports dynamic lighting and camera motion, significantly improving physical consistency in video relighting while easing the requirement of known per-frame camera pose. Extensive experiments demonstrate that Relit-LiVE consistently outperforms state-of-the-art video relighting and neural rendering methods across synthetic and real-world benchmarks. Beyond relighting, our framework naturally supports a wide range of downstream applications, including scene-level rendering, material editing, object insertion, and streaming video relighting. The Project is available at https://github.com/zhuxing0/Relit-LiVE.