PropFly: Learning to Propagate via On-the-Fly Supervision from Pre-trained Video Diffusion Models

Wonyong Seo; Jaeho Moon; Jaehyup Lee; Soo Ye Kim; Munchurl Kim

cs.CVFeb 24, 2026

PropFly: Learning to Propagate via On-the-Fly Supervision from Pre-trained Video Diffusion Models

Authors: Wonyong Seo, Jaeho Moon, Jaehyup Lee, Soo Ye Kim, Munchurl Kim

Organizations: KAIST · Kyungpook National University · Adobe Research

Abstract

Propagation-based video editing enables precise user control by propagating a single edited frame into following frames while maintaining the original context such as motion and structures. However, training such models requires large-scale, paired (source and edited) video datasets, which are costly and complex to acquire. Hence, we propose the PropFly, a training pipeline for Propagation-based video editing, relying on on-the-Fly supervision from pre-trained video diffusion models (VDMs) instead of requiring off-the-shelf or precomputed paired video editing datasets. Specifically, our PropFly leverages one-step clean latent estimations from intermediate noised latents with varying Classifier-Free Guidance (CFG) scales to synthesize diverse pairs of 'source' (low-CFG) and 'edited' (high-CFG) latents on-the-fly. The source latent serves as structural information of the video, while the edited latent provides the target transformation for learning propagation. Our pipeline enables an additional adapter attached to the pre-trained VDM to learn to propagate edits via Guidance-Modulated Flow Matching (GMFM) loss, which guides the model to replicate the target transformation. Our on-the-fly supervision ensures the model to learn temporally consistent and dynamic transformations. Extensive experiments demonstrate that our PropFly significantly outperforms the state-of-the-art methods on various video editing tasks, producing high-quality editing results.

Figures & tables

Figure 2: An illustration of our on-the-fly data pair generation process based on one-step clean latent estimation. (a) Pre-trained VDM sampling process from intermediate noised latents $\mathbf{x}_{t}$ with an edited text prompt $\mathbf{c}_{\text{aug}}$ , showing clean latent estimation after one-step sampling (Eq. 3 ) and full sampling (an iterative ODE solve from $t$ to $0$ ). (b) Increasing the CFG scale ( $\omega$ ) progressively strengthens the semantic edit (i.e., altering style, texture, and color). (c) Our method leverages this phenomenon efficiently: instead of performing computationally expensive full sampling, we utilize one-step clean latent predictions generated at a low CFG scale ( $\omega_{L}$ ) and a high CFG scale ( $\omega_{H}$ ). These on-the-fly predictions serve as the aligned source ( $\hat{\mathbf{x}}_{0|t}^{\text{low}}$ ) and target ( $\hat{\mathbf{x}}_{0|t}^{\text{high}}$ ) pair for training our PropFly.

Figure 3: Overview of our PropFly training pipeline. (a) A pair of video $\mathbf{x}_{0}$ and text prompt $\mathbf{c}_{\text{text}}$ is sampled from the video dataset and an augmented text $\mathbf{c}_{\text{aug}}$ is synthesized, by appending random style prompt $\mathbf{c}_{\text{style}}$ to $\mathbf{c}_{\text{text}}$ . (b) A frozen, pre-trained VDM $\theta$ synthesizes a data pair ( $\hat{\mathbf{x}}_{0|t}^{\text{low}},\hat{\mathbf{x}}_{0|t}^{\text{high}}$ ) on the fly from a single noised latent $\mathbf{x}_{t}$ using low and high CFG scales (guided by $\mathbf{c}_{\text{aug}}$ ). (c) A trainable adapter $\phi$ with the frozen VDM $\theta$ is then conditioned on the source video latent $\hat{\mathbf{x}}_{0|t}^{\text{low}}$ (for structure) and the edited first frame latent of $\hat{\mathbf{x}}_{0|t}^{\text{high}}$ . The adapter is trained via GMFM loss to predict the VDM’s text-guided, high-CFG velocity, effectively learning to edit the remaining video frames.

Figure 4: Qualitative comparison against propagation-based baselines AnyV2V [ 29 ] and Señorita-2M [ 63 ] . Our PropFly successfully propagates diverse edits (including object, background, and style changes) while preserving the motion of the source videos. In contrast, the baseline methods often fail to propagate the edits accurately or introduce severe visual artifacts. Zoom in for better visualization.

Methods	VE	Pick $\uparrow$	Frame $\uparrow$	Video $\uparrow$	CLIP $\uparrow$	DINO $\uparrow$	# Param.
TokenFlow [ 12 ]	Te	20.02	26.80	24.30	98.68	98.78	0.9B
STDF [ 58 ]	Te	19.73	26.35	23.64	96.37	96.00	1.7B
InsV2V [ 8 ]	Te	19.55	26.14	23.33	97.15	96.67	0.9B
Lucy Edit [ 50 ]	Te	19.52	26.64	23.74	98.51	98.43	-
VACE [ 21 ]	Te	20.04	27.33	24.50	98.96	98.79	14B
EditVerse [ 22 ]	Te	20.06	27.95	25.48	98.58	98.56	-

Table 1: Quantitative comparison on the EditVerseBench-Appearance subset [ 22 ] . ‘ Te ’ and ‘ Pr ’ in the second column denote text-guided and propagation-based video editing (VE) methods, respectively. We evaluate video quality (Pick), text alignment (Frame & Video), and temporal consistency (CLIP & DINO).

\uparrow

indicates higher is better.

Methods	VE	Pick $\uparrow$	CLIP $\uparrow$	$\text{ViCLIP}_{dir}$ $\uparrow$	$\text{ViCLIP}_{out}$ $\uparrow$
TAV [ 54 ]	Te	20.36	0.924	0.162	0.243
SDEdit [ 35 ]	Te	20.18	0.896	0.172	0.253
STDF [ 58 ]	Te	20.40	0.933	0.110	0.226
Fairy [ 53 ]	Te	19.80	0.933	0.164	0.208
InsV2V [ 8 ]	Te	20.76	0.911	0.208	0.262
EVE [ 49 ]	Te	20.76	0.922	0.221	0.262

Table 2: Quantitative comparison on the TGVE benchmark [ 55 ] . ‘ Te ’ and ‘ Pr ’ in the second column denote text-guided and propagation-based video editing (VE) methods, respectively. We evaluate video quality (Pick), temporal consistency (CLIP), and text alignment (

\text{ViCLIP}_{dir}

\text{ViCLIP}_{out}

Method	Pick $\uparrow$	Frame $\uparrow$	Video $\uparrow$	CLIP $\uparrow$	DINO $\uparrow$
w/ Full sampling	19.75	27.20	24.77	98.77	98.51
w/ FM loss (Eq. 1 )	19.50	26.33	21.98	98.52	98.29
w/o RSPF (Sec. 4.2 )	20.28	28.35	25.61	98.96	98.55
w/ Paired dataset	19.53	27.12	24.69	98.13	97.85
PropFly-1.3B	20.35	28.37	25.37	99.03	98.63

Table 3: Ablation study of our key components on the EditVerseBench-Appearance subset [ 22 ] .

Figure 5: Visual results showing the effect of our key components. (a) Baseline trained with full sampling fails to align object motion, while the baseline trained with the conventional FM objective fails to propagate the edit. (b) Baselines trained without our RSPF or with the paired dataset lack generalization, failing to perform complex edits. In contrast, our PropFly achieves robust propagation performance and high-fidelity edits. Zoom-in for details.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Weather	Artistic Style	Material	Mood	Backgrounds
snowy	in the style of Van Gogh	made of glass	monochrome	Gradient Background
rainy	in the style of Monet	made of crystal	vibrant colors	Moving Particle Background
drizzling	in the style of Picasso	made of ice	pastel colors	Geometric Pattern Background
foggy	in the style of Dali	glowing	muted colors	Neon Line Background
misty	in the style of Rembrandt	holographic	dreamy	Abstract Digital Art Background
sunny	in the style of Hokusai	iridescent	ethereal	Matrix Code Background

Appendix

Table 4: Style prompt set used for RSPF during the training of PropFly.

$\omega_{H}$	Pick Score $\uparrow$	Frame $\uparrow$	Video $\uparrow$	CLIP $\uparrow$	DINO $\uparrow$
2	19.98	27.24	24.57	98.83	98.41
5	20.32	28.33	25.52	99.01	98.62
7	20.35	28.37	25.37	99.03	98.63
10	20.27	28.24	25.34	99.00	98.80
20	20.19	28.33	25.27	98.85	98.62

Appendix

Table 5: Impact of CFG scaling on performance metrics during on-the-Fly data pair generation. Best performance for each metric is highlighted in bold , and the second best performance is indicated by an

\underline{\text{underline}}

Figure 6: Samples of full sampling with varying CFG scales.

Figure 7: Samples of input videos and their on-the-fly data pairs.

Target Videos	Text Alignment $\uparrow$	Motion Alignment $\uparrow$
Señorita-2M [ 63 ]	20.41	78.08
On-the-fly (Ours)	20.97	82.57

Appendix

Table 6: Synthetic video data quality comparison. We evaluate the text alignment and motion alignment.

\uparrow

indicates higher is better.

Backbone	Pick Score $\uparrow$	Frame $\uparrow$	Video $\uparrow$	CLIP $\uparrow$	DINO $\uparrow$
LTX-2B	19.87	27.37	24.68	99.02	98.61
Wan-1.3B	20.35	28.37	25.37	99.03	98.63
Wan-14B	20.42	28.71	26.05	99.21	99.05

Appendix

Table 7: Generalization across backbones. We evaluate the generalization capability of PropFly by applying it to both Wan [ 50 ] and LTX-Video [ 14 ] , measured on the EditVerseBench-Appearance subset [ 22 ] .

Figure 8: Video quality comparison between propagation-based video editing methods on the DAVIS dataset [ 40 ] .

Figure 9: Video quality comparison between propagation-based video editing methods on the DAVIS dataset [ 40 ] .

Figure 10: Video quality comparison on the Propagation task in EditVerseBench [ 22 ]

Figure 11: Video quality comparison on the Change object task in EditVerseBench [ 22 ]

Figure 12: Video quality comparison on the Add object and Remove object tasks in EditVerseBench [ 22 ]

Video Editing

Video editing uses AI to alter, compose, relight, or reorganize moving footage while controlling how changes persist across frames.

See all 178 papers

StreamEdit: Training-Free Video Editing via Few-Step Streaming Video Generation
May 20, 2026Guanlong Jiao, Chenyangguang Zhang, Jia Jun Cheng Xian +2Video Editing
The University of British Columbia · Vector Institute · ETH Zürich +2
LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing
Apr 18, 2026Weicheng Wang, Zhicheng Zhang, Zhongqi Zhang +6Video Editing Benchmarks Video Diffusion Models
Nankai University · Kuaishou Technology · Pengcheng Laboratory +1
VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling
Oct 8, 2026Leigang Qu, Feng Cheng, Ziyan Yang +7Video Editing Instruction-Based Video Editing
National University of Singapore · ByteDance Seed · University of Science and Technology of China

May 20, 2026J/K move · Enter open · S save

StreamEdit: Training-Free Video Editing via Few-Step Streaming Video Generation

Guanlong Jiao, Chenyangguang Zhang, Jia Jun Cheng Xian, Zewei Zhang+1

The University of British Columbia · Vector Institute · ETH Zürich+2

Video Editing

Open paper AskPDF arXiv

Although existing video editing methods are generally feasible, they often require many costly iterations and still struggle to deliver high-quality yet satisfying editing results. We attribute this limitation to the prevalent data-to-data paradigm, which is less compatible with modern generative models than noise-to-data generation. To address this gap, we revisit video editing from a noise-to-data perspective and propose Streaming-Generation-based Video Editing (StreamEdit), which preserves few-step sampling while seamlessly injecting source-video conditions. Built on pre-trained streaming generation models, StreamEdit introduces dual-branch fast sampling with a self-attention bridge and cross-attention grounding/boosting to satisfy both sampling and conditioning requirements. We further propose source-oriented guidance to improve target-generation quality, and a visual prompting strategy to enhance editing flexibility and practicality. The method is effective, robust, and generalizable across different models. Extensive experiments on diverse video editing tasks show that StreamEdit consistently outperforms existing approaches, even in few-step settings with minimal time cost. Code and results are available at: https://dsl-lab.github.io/StreamEdit/.

Abstract

Video Editing

Explore similar work

StreamEdit: Training-Free Video Editing via Few-Step Streaming Video Generation

LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing

VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling