Current instruction-guided video editing models struggle to simultaneously balance precise semantic modifications with faithful motion preservation. While existing approaches rely on injecting explicit external priors (e.g., VLM features or structural conditions) to mitigate these issues, this reliance severely bottlenecks model robustness and generalization. To overcome this limitation, we present SAMA (factorized Semantic Anchoring and Motion Alignment), a framework that factorizes video editing into semantic anchoring and motion modeling. First, we introduce Semantic Anchoring, which establishes a reliable visual anchor by jointly predicting semantic tokens and video latents at sparse anchor frames, enabling purely instruction-aware structural planning. Second, Motion Alignment pre-trains the same backbone on motion-centric video restoration pretext tasks (cube inpainting, speed perturbation, and tube shuffle), enabling the model to internalize temporal dynamics directly from raw videos. SAMA is optimized with a two-stage pipeline: a factorized pre-training stage that learns inherent semantic-motion representations without paired video-instruction editing data, followed by supervised fine-tuning on paired editing data. Remarkably, the factorized pre-training alone already yields strong zero-shot video editing ability, validating the proposed factorization. SAMA achieves state-of-the-art performance among open-source models and is competitive with leading commercial systems (e.g., Kling-Omni). Code, models, and datasets will be released.
Figures & tables
Figure 1: Teaser and overview. Top: qualitative comparisons on VIE-Bench, comparing SAMA with representative open- and closed-source systems. Bottom left: illustration of SAMA’s semantic–motion training objectives. Bottom right: fine-grained VIE-Bench performance comparison.
Figure 2: Overall pipeline. SAMA first performs factorized pre-training (stage 0) on additional perturbed videos by completing a pretext task conditioned on the given captions. It then performs normal supervised fine-tuning (stage 1) on original source videos. Semantic Anchoring is incorporated in both stages to jointly facilitate semantic representation learning and instruction-guided video editing.
Figure 3: Illustration of pretext perturbations.
Training stage
Dataset
# Pairs
Type
Stage 0 Factorized Pre-training
NHR-Edit [ 67 ]
720,087
image editing
GPT-Image-Edit [ 69 ]
1,015,170
image editing
X2Edit [ 70 ]
768,470
image editing
Koala-36M [ 65 ]
1,532,716
text-to-video ★
MotionBench [ 66 ]
53,879
text-to-video ★
Stage 1 Supervised Fine-tuning
NHR-Edit [ 67 ]
720,087
image editing
Table 1: Statistics of training data for each stage. ★ denotes we use specific text-to-video data for Motion Alignment by solving pretext transformations.
Method
Instruct follow
Preser- vation
Quality
Avg.
Instruct follow
Preser- vation
Quality
Avg.
Add
Swap / Change
Kling1.6
6.000
8.230
5.576
6.602
9.000
9.060
8.333
8.800
Kling-Omni
9.333
9.589
8.622
9.181
9.495
9.448
8.638
9.194
Runway
8.607
8.913
7.823
8.447
9.580
8.628
9.275
9.161
Pika
-
-
-
-
7.542
7.847
6.837
7.408
InsV2V
3.552
5.891
3.402
4.281
5.304
6.428
4.971
5.567
Table 2: Comparison results on VIE-Bench . The best results are shown in bold . Gray shading indicates closed-source models.
Task
Method
E dit A ccuracy
V ideo N aturalness
V ideo Q uality
Average Score
SA
SP
CP
AN
SN
MN
VF
TS
ES
SEA
SVN
SVQ
S
Add
InsViE
2.60
2.79
2.78
2.33
3.98
3.74
3.71
3.91
3.58
2.60
3.10
3.46
3.05
Lucy-Edit
6.27
6.32
7.75
4.63
7.08
6.08
6.31
6.82
7.57
6.47
5.70
6.77
6.31
Ditto
7.46
7.24
6.30
6.30
8.85
8.30
8.13
8.55
9.03
6.70
7.57
8.41
7.56
UniVideo
9.39
9.27
9.69
7.27
9.23
8.80
8.44
8.89
9.75
9.40
8.31
8.99
8.90
ReCo
8.65
8.40
9.22
6.39
8.78
8.28
8.02
8.61
9.61
8.54
7.55
8.61
8.23
Table 4: Comparison results on ReCo-Bench with Gemini-2.5-Flash-Thinking . The best results are shown in bold . Abbreviations: SA, semantic accuracy; SP, scope precision; CP, content preservation; AN, appearance naturalness; SN, scale naturalness; MN, motion naturalness; VF, visual fidelity; TS, temporal stability; ES, edit stability. SEA / SVN / SVQ are category scores and S is the overall score.
Figure 4: Qualitative comparisons with prior methods on VIE-Bench and ReCo-Bench.
Figure 5: Zero-shot qualitative results on VIE-Bench at two training stages.
Figure 6: Ablations for Semantic Anchoring (SA).
Figure 7: Qualitative comparison of SAMA w/ and w/o MA .
method
instruct follow
preser- vation
quality
Overall
baseline
6.575
6.261
6.100
6.312
w/ SA
7.002
6.744
6.342
6.696
w/ MA
6.969
6.620
6.544
6.711
SAMA
7.402
6.998
6.884
7.095
Table 5: Ablations of SAMA modules.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
method
instruct follow
preser- vation
quality
Overall
w/ PE
6.705
7.533
6.686
6.975
w/o PE
6.619
6.257
6.619
6.498
Appendix
Table 6: Ablations of Type Embeddings (TE) modules.
Figure 8: Illustration of Type Embedding (TE).
Figure 9: Illustration of Pretext Prediction.
Figure 10: More qualitative results on VIE-Bench .
Figure 11: More qualitative results on OpenVE-Bench .
Figure 12: More qualitative results on ReCo-Bench .
Video editing spans diverse editing paradigms, yet achieving high-quality instruction-guided and subject-guided editing within a single unified framework remains challenging. We introduce EditVid, a training-free framework combining sparse causal memory for local coherence, correspondence-based post-attention token injection for long-range identity preservation, and soft latent blending for edit locality. The same framework supports instruction-guided and reference-guided edits, including style transfer, attribute modification, object insertion, part-level editing, and subject replacement. On FiVE, EditVid achieves 78.16 FiVE-Acc, compared with 58.95 for the strongest evaluated training-free baseline, while obtaining competitive results on IVEBench. A user study further shows a 51.8% overall preference for EditVid over 7 competing methods.
Adheesh Sunil Juvekar, Onkar Kishor Susladkar, Kiet A. Nguyen +6
Instruction-based video editing is a natural way to control video content with text, but adapting a video generation model into an editor usually appears data-hungry. At the same time, high-quality video editing data remains scarce. In this paper, we show that a video generation backbone can become a strong video editor without large scale video editing data. We present InsEdit, an instruction-based editing model built on HunyuanVideo-1.5. InsEdit combines a visual editing architecture with a video data pipeline based on Mutual Context Attention (MCA), which creates aligned video pairs where edits can begin in the middle of a clip rather than only from the first frame. With only O(100)K video editing data, InsEdit achieves state-of-the-art results among open-source methods on our video instruction editing benchmarks. In addition, because our training recipe also includes image editing data, the final model supports image editing without any modification.
Zhefan Rao, Bin Zou, Haoxuan Che +5
The Hong Kong University of Science and Technology · Celia Research HK · City University of Hong Kong
Instruction-based video editing has witnessed rapid progress, yet current methods often struggle with precise visual control, as natural language is inherently limited in describing complex visual nuances. Although reference-guided editing offers a robust solution, its potential is currently bottlenecked by the scarcity of high-quality paired training data. To bridge this gap, we introduce a scalable data generation pipeline that transforms existing video editing pairs into high-fidelity training quadruplets, leveraging image generative models to create synthesized reference scaffolds. Using this pipeline, we construct RefVIE, a large-scale dataset tailored for instruction-reference-following tasks, and establish RefVIE-Bench for comprehensive evaluation. Furthermore, we propose a unified editing architecture, Kiwi-Edit, that synergizes learnable queries and latent visual features for reference semantic guidance. Our model achieves significant gains in instruction following and reference fidelity via a progressive multi-stage training curriculum. Extensive experiments demonstrate that our data and architecture establish a new state-of-the-art in controllable video editing. All datasets, models, and code is released at https://github.com/showlab/Kiwi-Edit.