EditDistill: Is It Possible to Guide Video Editing with Image Editing?

Guojun Lei; Hong Li; Hongbing Yang; Lixue Gong; Chi Wang; Zheng Dong

cs.CVOct 25, 2026

EditDistill: Is It Possible to Guide Video Editing with Image Editing?

Authors: Guojun Lei, Hong Li, Hongbing Yang, Lixue Gong, Chi Wang, Zheng Dong

Organizations: State Key Lab of CAD&CG, Zhejiang University · Beihang University · Tsinghua University · Zhejiang Sci-Tech University

Abstract

Recent advances in image editing have enabled fine-grained and highly controllable visual modifications. However, achieving similar levels of controllability in video editing remains an open challenge, particularly for precise local edits, dynamic temporal effects, and consistent lighting across frames.In this paper, we propose EditDistill, a framework that distills image-level visual transformations from edits to guide video editing.Specifically, given a source video and an edit prompt, we first generate an edited version of the initial video frame using an off-the-shelf image editing model. We then distill the visual transformation of this original-edited image pair into a compact representation termed an edit descriptor.This descriptor is injected into a video diffusion model through our lightweight fusion module, which steers the denoising process toward the desired edit. To align the video trajectory with the demonstrated image edit while preserving motion and layout, we train with edit-descriptor alignment and DINO structural consistency losses, and derive an equivalent inference-time guidance formulation under flow matching.Extensive experiments demonstrate that our method achieves strong performance on standard benchmarks and produces high-quality video editing results across diverse scenarios. For more details and videos, please refer to the anonymous webpage: https://yu-shaonian.github.io/EditDistill-Web/.

Figures & tables

Figure 1: EditDistill distills an image edit pair into a compact transformation descriptor and injects it into a video diffusion model, enabling local object edits, stylization, background replacement, and global relighting while preserving source motion and temporal coherence.

Figure 2: EditDistill encodes the visual transformation between image editing pair into a compact edit descriptor, and combines the edit descriptor alignment constraint with the DINO structural similarity loss to distill the image editing capability into the denoising trajectory of video editing.

Figure 3: Overview of EditDistill . Stage I encodes the transformation between an image pair $(I^{s},I^{e})$ into the edit descriptor ${\bf d}$ . Stage II keeps the Wan2.1-I2V-14B video diffusion frozen and trains the inject modules to inject ${\bf d}$ into selected transformer blocks by lightweight feature fusion modules steering the diffusion denoising toward the desired edit to produce a temporally coherent video $V^{e}$ from the video $V^{s}$ .

Figure 4: Descriptor attention. Cross-attention maps show that local edits concentrate on the regions that need to change, while global edits such as relighting or stylization produce broader but still structured responses. The descriptor behaves as a transformation operator rather than a fixed mask or copied target image.

Method	CLIP-E $\uparrow$	CLIP-T $\uparrow$	DINO-sim $\uparrow$	Temp. $\uparrow$	Warp $\downarrow$	FVD $\downarrow$
InstructPix2Pix ( Brooks et al., 2023 )	0.221	0.288	0.534	0.892	7.15	1284.7
TokenFlow ( Geyer et al., 2024 )	0.196	0.275	0.668	0.948	4.21	1052.3
AnyV2V ( Ku et al., 2024 )	0.137	0.251	0.591	0.856	6.94	1485.2
RF-Solver-Edit ( Wang et al., 2025b )	0.224	0.286	0.654	0.926	4.58	996.3
MOFA-Video ( Niu et al., 2024 )	0.182	0.271	0.621	0.943	4.83	1024.1
Prompt-only Wan2.1 ( Wang et al., 2025a )	0.158	0.291	0.612	0.951	3.84	982.4

Table 1: Quantitative comparisons. All methods run at matched

480\times 832

and

81

-frame experimental settings. For evaluation metrics, CLIP-E and CLIP-T measure edit fidelity; DINO-sim measures preservation of unedited regions; Temp. and Warp measure temporal coherence; FVD measures perceptual video quality. Best results are in bold and second best results are underlined .

Method	SubC $\uparrow$	BkgC $\uparrow$	MoS $\uparrow$	AesQ $\uparrow$	DyaD $\uparrow$
InstructPix2Pix ( Brooks et al., 2023 )	0.842	0.871	0.924	0.429	0.612
TokenFlow ( Geyer et al., 2024 )	0.946	0.951	0.972	0.514	0.524
AnyV2V ( Ku et al., 2024 )	0.812	0.768	0.912	0.431	0.512
RF-Solver-Edit ( Wang et al., 2025b )	0.928	0.939	0.967	0.571	0.628
MOFA-Video ( Niu et al., 2024 )	0.932	0.945	0.974	0.568	0.718
Prompt-only Wan2.1 ( Wang et al., 2025a )	0.921	0.943	0.944	0.562	0.681

Table 2: Comparisons of video editing consistency. SubC: subject consistency; BkgC: background consistency; MoS: motion smoothness; AesQ: aesthetic quality; DyaD: dynamic degree. Best results are in bold ; second best results are underlined .

Figure 5: Qualitative comparison. EditDistill performs both local and global edits while keeping the main subject, background, and motion stable. In contrast, other methods may miss the desired edit, unintentionally change nearby regions, or produce unstable results across frames.

Figure 6: Relighting comparison. The distilled descriptor captures a global lighting transformation and guides coherent relighting across frames while preserving scene geometry and motion.

Variant	Trainable params	CLIP-E $\uparrow$	DINO-sim $\uparrow$	Warp $\downarrow$	FVD $\downarrow$
Cross-attention only	312MB	0.243	0.692	3.41	921.4
FiLM only	187MB	0.218	0.704	3.55	938.2
Additive only	154MB	0.196	0.681	3.78	967.5
All branches (ours)	480MB	0.245	0.738	2.97	852.1

Table 3: Ablation study of the edit feature injection module. The three branches are complementary: cross-attention is strongest for local routing, FiLM improves global appearance control, and the additive branch contributes a stable residual edit bias.

Variant	CLIP-E $\uparrow$	DINO-sim $\uparrow$	Temp. $\uparrow$	Warp $\downarrow$	FVD $\downarrow$
$\mathcal{L}_{\mathrm{FM}}$ only	0.184	0.671	0.943	3.86	967.4
$\mathcal{L}_{\mathrm{FM}}+\mathcal{L}_{\mathrm{edit}}$	0.273	0.682	0.948	3.61	902.1
$\mathcal{L}_{\mathrm{FM}}+\mathcal{L}_{\mathrm{DINO}}$	0.198	0.731	0.961	3.04	884.6
Full objective	0.245	0.738	0.964	2.97	852.1

Table 4: Auxiliary-loss ablation. Removing

\mathcal{L}_{\mathrm{edit}}

weakens edit transfer; removing

\mathcal{L}_{\mathrm{DINO}}

increases structural drift and warp error.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Stage I edit descriptor model. The Delta VAE encoder extracts a stochastic descriptor from the source and edited VAE latents by combining shared features, their explicit feature difference, and a multi-scale encoder. The Delta VAE decoder receives only the source latent and descriptor, and reconstructs the edited latent through descriptor-conditioned residual blocks with self- and cross-attention.

Figure 8: Frame-wise descriptor attention. The same edit descriptor re-localizes to the relevant moving regions across different video frames.

Figure 9: Qualitative ablation. Removing individual branches or alignment losses weakens either edit fidelity or temporal preservation, matching the quantitative trends in Section 5.3 .

Figure 10: More results in traffic environments.

Video Editing

Video editing uses AI to alter, compose, relight, or reorganize moving footage while controlling how changes persist across frames.

See all 178 papers

LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing
Apr 18, 2026Weicheng Wang, Zhicheng Zhang, Zhongqi Zhang +6Video Editing Benchmarks Video Diffusion Models
Nankai University · Kuaishou Technology · Pengcheng Laboratory +1
One Editor, Many Edits: A Unified Training-Free Framework for Diverse Video Editing
Sep 3, 2026Adheesh Sunil Juvekar, Onkar Kishor Susladkar, Kiet A. Nguyen +6Video Editing Instruction-Based Video Editing
University of Illinois Urbana-Champaign
Transforming Image Editors into Video Editors
Oct 8, 2026Feng Wang, Zijie Li, Ceyuan Yang +2Video Editing Instruction-Based Video Editing
ByteDance Seed · Johns Hopkins University

Apr 18, 2026J/K move · Enter open · S save

LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing

Weicheng Wang, Zhicheng Zhang, Zhongqi Zhang, Juncheng Zhou+5

Nankai University · Kuaishou Technology · Pengcheng Laboratory+1

Video Editing Benchmarks Video Diffusion Models Video Editing Instruction-Based Video Editing

Open paper AskPDF arXiv

Video editing aims to modify input videos according to user intent. Recently, end-to-end training methods have garnered widespread attention, constructing paired video editing data through video generation or editing models. However, compared to image editing, the high annotation costs of video data severely constrain the scale, quality, and task diversity of video editing datasets when relying on video generative models or manual annotation. To bridge this gap, we propose LIVE, a joint training framework that leverages large-scale, high-quality image editing data alongside video datasets to bolster editing capabilities. To mitigate the domain discrepancy between static images and dynamic videos, we introduce a frame-wise token noise strategy, which treats the latents of specific frames as reasoning tokens, leveraging large pretrained video generative models to create plausible temporal transformations. Moreover, through cleaning public datasets and constructing an automated data pipeline, we adopt a two-stage training strategy to anneal video editing capabilities. Furthermore, we curate a comprehensive evaluation benchmark encompassing over 60 challenging tasks that are prevalent in image editing but scarce in existing video datasets. Extensive comparative and ablation experiments demonstrate that our method achieves state-of-the-art performance. The source code will be publicly available.

Abstract

Video Editing

Explore similar work

LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing

One Editor, Many Edits: A Unified Training-Free Framework for Diverse Video Editing

Transforming Image Editors into Video Editors