Instruction-Based Video Editing

Instruction-based video editing studies how to turn requests such as changing an object, action, or background into coherent changes across a video. Most systems adapt video diffusion transformers, sometimes drawing on image-editing models, and use methods such as masks, reference images, attention control, or spatial planning to localize edits. Benchmarks test instruction following, source-content preservation, temporal consistency, and increasingly complex or audio-visual edits. A central challenge is making multiple requested changes accurately without introducing artifacts, altering untouched content, or losing consistency over long clips.