cs.CVAug 14, 2026

Instruction-Based Video Editing by Repurposing an Image Editing Model

Authors: Yunpeng Bai, Yossi Gandelsman, Michaël Gharbi, Qixing Huang

Organizations: UT Austin · Reve

Abstract

Instruction-based video editing is commonly built on video-pretrained generative backbones: a video diffusion transformer is adapted, at considerable cost, to condition on a source video and an editing instruction. In this report we explore a different route and show that a strong instruction-based image editing model can edit videos by operating directly on video-VAE latents. Starting from Qwen-Image-Edit, we arrange the latent frames of a Wan2.1 video VAE as tiles of one large virtual image, reuse the editor's image positional encoding for every tile, and bridge the two latent spaces with a pair of lightweight input/output projections warm-started from the editor's own patchify and unpatchify layers, so that at initialization a (static) video is embedded exactly as an image the model already understands. The whole system is then fine-tuned on the public Ditto-1M editing triplets, and a few denoising steps of Wan2.2 serve as an optional temporal enhancer. We motivate the design with a chain of zero-training observations: the stock image editor already edits a video presented as a contact sheet; it is indifferent to whether the sheet's tokens come from one joint encode or from per-frame encodes stitched in latent space; and it even edits genuine video latents zero-shot to a clearly recognizable degree, leaving fine-tuning only a fidelity gap to close. Our results suggest that, despite the large investment in training video latent spaces, per-frame video latents remain close enough to the image domain that mature image editing priors transfer with minimal adaptation. Project Page: https://yunpeng1998.github.io/Qwen-Video-Edit-Page Code: https://github.com/yunpeng1998/Qwen-Video-Edit Model: https://huggingface.co/yunpeng1998/Qwen-Video-Edit

Figures & tables

Explore similar work

CardsList
  1. InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptation

    Apr 9, 2026Zhefan Rao, Bin Zou, Haoxuan Che +5Instruction-Based Video EditingDiffusion-Based Video Editing

  2. VideoX-Qwen: Data-Centric Instruction-Based Video Editing

    Sep 22, 2026JJiahang Li, Dingbao Shao, Xinyu Chen +8Video EditingInstruction-Based Video Editing

  3. VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling

    Oct 8, 2026Leigang Qu, Feng Cheng, Ziyan Yang +7Video EditingInstruction-Based Video Editing