Instruction-Based Video Editing

Overview

Instruction-based video editing studies how to turn requests such as changing an object, action, or background into coherent changes across a video. Most systems adapt video diffusion transformers, sometimes drawing on image-editing models, and use methods such as masks, reference images, attention control, or spatial planning to localize edits. Benchmarks test instruction following, source-content preservation, temporal consistency, and increasingly complex or audio-visual edits. A central challenge is making multiple requested changes accurately without introducing artifacts, altering untouched content, or losing consistency over long clips.

Latest papers

CardsList

67 papers3 papers in the last four weeks, down 75% on the four weeks before. 0.0% of all new papers.

  1. Motion-Aware Reward Modeling for Instruction-Guided Video Editing

    Nov 17, 2026Li Li, Xiangbo Gao, Bo Ni +11Reward ModelingVideo Reward Modeling

  2. VideoEditor-MoE: Scaling Complex Video Editing with Mixture of Experts

    Nov 2, 2026Xiao Cao, Fuchen Long, Yansong Qu +5Mixture-of-Experts ModelsInstruction-Based Video Editing

  3. FilmTogether: Cinematic Video Generation from Low-Cost Inputs

    Oct 31, 2026Shiying Zhang, Xiaobin Hu, Qianchi Zhang +4Video EditingInstruction-Based Video Editing

  4. Cassette-Editor: Shared Timeline Semantics for Human and Agent Editing

    Oct 19, 2026Zhixin Zeng, Jiahao ZhongAgentic Video EditingVideo Editing

  5. QK-Edit: Revisiting Attention-based Injection in MM-DiT for Image and Video Editing

    Oct 13, 2026Tiancheng Shen, Zilong Huang, Xiangtai Li +6Diffusion-Based Image EditingAttention Control in Diffusion Models

  6. VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling

    Oct 8, 2026Leigang Qu, Feng Cheng, Ziyan Yang +7Video EditingInstruction-Based Video Editing

  7. Transforming Image Editors into Video Editors

    Oct 8, 2026Feng Wang, Zijie Li, Ceyuan Yang +2Video EditingInstruction-Based Video Editing

  8. CrossEdit: Cross-Modal Training Enables Rich Audio-Visual Editing

    Oct 7, 2026William Chen, Prem Seetharaman, Ke Chen +10Instruction-Based Video EditingUnified Multimodal Models

  9. ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing

    Oct 6, 2026Zhenghong Zhou, Zhe Lin, Jiebo Luo +1Video EditingInstruction-Based Video Editing

  10. A Spatiotemporal Semantic Importance-Guided Unified Compression and Editing Framework for AI-Generated Videos

    Oct 5, 2026Xihua Sheng, Dong Liu, Chang Wen ChenInstruction-Based Video EditingVideo Compression

  11. Reimagine Video Dynamics

    Sep 29, 2026Yu Yuan, Yawen Lu, Guoxian Song +5Disentangled Representation LearningVideo Editing

  12. Precise Editing and Flexible Referencing for Interactable Worlds

    Sep 28, 2026Xinyao Liao, Xianfang Zeng, Zhu Liang +5Instruction-Based Video EditingAutoregressive Video Generation

  13. SemantiScene-Edit: Expert-Synthesized Supervision and Dual-Path Source Conditioning for Instruction-Based Video Editing

    Sep 22, 2026Xinyu Chen, Song Wu, Dingbao Shao +12Video EditingInstruction-Based Video Editing

  14. Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing

    Sep 18, 2026Chongbo Zhao, Jiangming Wang, Xilai Wang +5Video EditingInstruction-Based Video Editing

  15. Object-Aware Background-Controlled Editing via Weighted Velocity Guidance

    Sep 5, 2026Wuji Wang, Yue Wu, Chenhao Yi +1Image EditingDiffusion-Based Image Editing

  16. One Editor, Many Edits: A Unified Training-Free Framework for Diverse Video Editing

    Sep 3, 2026Adheesh Sunil Juvekar, Onkar Kishor Susladkar, Kiet A. Nguyen +6Video EditingInstruction-Based Video Editing

  17. AVENUE: Audio-Video EditiNg Understanding and Evaluation

    Aug 31, 2026Hayeon Kim, Yoojin Jang, Jaejun YooVideo Editing BenchmarksInstruction-Based Video Editing

  18. EditaLive! Unified Character Video Editing for Live Streaming

    Aug 27, 2026Zhiyuan Li, Chi-Man Pun, Peng-Tao Jiang +2Video EditingInstruction-Based Video Editing

  19. Thinking on Shots: Consistent Multi-Shot Video Editing with Agentic Reasoning

    Aug 27, 2026Chenyang Wu, Fuchen Long, Binyuan Huang +4Agentic Video EditingVideo Editing

  20. RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editing

    Aug 26, 2026Bojia Zi, Xiaoyan Yang, Yu Zhou +7Video EditingInstruction-Based Video Editing

  21. InfinityEdit: Infinite Video Editing with a Lightweight Edit-Ignition Adapter

    Aug 21, 2026Yunze Tong, Mushui Liu, Canyu Zhao +9Video EditingInstruction-Based Video Editing

  22. CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editing

    Aug 18, 2026Fuchen Long, Cong Wang, Zitao Gao +8Video EditingInstruction-Based Video Editing

  23. VicEdit: Learning to Edit Videos from Visual In-Context Examples

    Aug 17, 2026Yuji Wang, Teng Hu, Yuheng Chen +6Video EditingInstruction-Based Video Editing

  24. Instruction-Based Video Editing by Repurposing an Image Editing Model

    Aug 14, 2026Yunpeng Bai, Yossi Gandelsman, Michaël Gharbi +1Video EditingInstruction-Based Video Editing

  25. OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing

    Aug 5, 2026Chenxuan Miao, Yutong Feng, Yi Lu +6Video Editing BenchmarksVideo Editing