Instruction-Based Video Editing
Overview
Instruction-based video editing studies how to turn requests such as changing an object, action, or background into coherent changes across a video. Most systems adapt video diffusion transformers, sometimes drawing on image-editing models, and use methods such as masks, reference images, attention control, or spatial planning to localize edits. Benchmarks test instruction following, source-content preservation, temporal consistency, and increasingly complex or audio-visual edits. A central challenge is making multiple requested changes accurately without introducing artifacts, altering untouched content, or losing consistency over long clips.
Latest papers
63 papers11 papers in the last four weeks, up 120% on the four weeks before. 0.1% of all new papers.