Reward models for instruction-guided video editing can favor temporal consistency at the expense of motion. We study this failure, termed motion collapse, using controlled motion-suppression variants of real edits. On VE-Bench, the evaluated composite reward prefers a motion-suppressed candidate in 97% of cases. Learned video rewards and a VLM judge also struggle with mild suppression, while latent optimization and reward-guided fine-tuning exploit this weakness to increase reward as motion declines. A linear probe of our base reward backbone performs near chance on mild suppression, motivating an explicit inter-frame dynamics input. We propose a motion-aware reward model that combines an RGB frame-difference stream with preference training against motion-suppressed and fabricated-motion negatives. A distillation anchor preserves alignment with human quality judgments. On the VE-Bench motion ladder, our model reduces the rate of selecting motion-suppressed candidates to 0% while achieving a Spearman correlation of 0.57 with human quality ratings. It also detects unseen suppression variants and preserves motion during reward-guided fine-tuning on two editing pipelines. These results support explicitly modeling motion in rewards for video editing.
Figures & tables
Figure 1: Motion-aware reward model. The pixel stream encodes source and edited videos, while a dynamics stream computes tanh-normalized RGB differences of the edited video. Their features are fused at block 4 of the Qwen3-VL-32B backbone and adapted with LoRA. Training combines pairwise preferences over original edits and motion-suppressed or fabricated-motion negatives with a distillation anchor to the base preference reward.
Reward
genuine
hold-2
hold-4
frozen
capture ↓
Field metrics
RAFT warping error ( ↓ )
33.20
23.39
16.08
0.03
100%
DINO temporal consistency ( ↑ )
0.869
0.919
0.950
1.000
100%
Field composite (appearance + temporal)
{ z-normalized weighted combination }
97%
Field composite (appearance only)
62%
Learned video reward models
Table 1: Mean scores across the VE-Bench motion ladder and top-1 capture rates. The composite combines appearance and temporal terms; the appearance-only variant removes the latter. VideoReward is reported overall and by head.
Figure 2: Motion collapse on VE-Bench. Left: mean rewards along the motion ladder, min–max normalized per reward; VideoReward uses its motion-quality head. Right: top-1 capture rate: the fraction of cases where a motion-suppressed candidate scores highest (lower is better). Learned reward models use their overall scores.
hold-2
hold-4
flow-swap
real 2
motion attacks
(subtle)
(extreme)
fake
(control)
combined
Humans prefer real
96%
98%
100%
62%
98%
Wilson 95% CI
[92, 98]
[94, 99]
[97, 100]
[48, 74]
[96, 99]
RM agreement with human majority
Field composite
23%
13%
60%
30%
32%
VideoScore-v1.1
33%
44%
59%
50%
46%
Table 2: Human preferences and reward agreement across 100 trials from 30 sources. Top: vote fractions favoring the original edit, with Wilson 95% CIs. Bottom: agreement with the human majority, excluding tied reward predictions. The motion-attack aggregate excludes real-vs-real controls.
Figure 3: Left: motion during reward-guided fine-tuning on UniVideo, normalized to its initial value (genuine-motion subset, mean ± std). Right: leave-one-source-out linear-probe accuracy on the base reward features, with Wilson 95% CIs. Mild suppression remains near chance; see Section 4.1 .
Prior-work rewards
Ours
Optimizer regime
field
VideoScore
RAFT flow
motion-aware RM
Best-of- N (natural outputs)
≈0 (observed)
DRaFT-1 (UniVideo)
−61% / +0.05
−21% / +0.01
−35% / +0.05
+1% / +0.00
DRaFT-1 (CogVideoX-2b)
+19% / +0.02
−42% / +0.01
−88% / −0.01
+9% / +0.02
Table 3: Motion change and CLIP-T change (italic) under reward optimization. UniVideo fine-tuning results use the genuine-motion subset; CogVideoX-2b results use eight sources. DRaFT-1 [ 2 ] implements reward-guided fine-tuning. Additional analyses appear in Appendix B.6 .
Reward
Cap. ↓
h2
frzn
segFrz
burstFrz
flowAtt
Corr(mot.)
MOS Sp. ↑
Field composite
97%
6%
1%
11%
12%
34%
−0.32
+0.55
VideoScore
80%
58%
50%
40%
48%
74%
+0.15
+0.18
VideoReward
76%
35%
43%
57%
53%
40%
−0.19
+0.46
MQ head
98%
11%
7%
57%
69%
11%
+0.15
+0.09
Ours
0%
100%
100%
91%
100%
95%
−0.10
+0.57
Table 4: Reward robustness and quality alignment on VE-Bench. Cap. is ladder capture rate; h2 / frzn / segFrz / burstFrz / flowAtt are preference rates for original edits over suppressed variants. Additional variants are defined in Appendix B.2 . MQ denotes VideoReward’s motion-quality head.
Figure 4: Reward distributions on the VE-Bench motion ladder. The base reward gives overlapping scores to original edits and hold variants. The motion-aware reward lowers scores for suppressed candidates while retaining a broad distribution on original edits.
Config
hold-2 detection ↑
Fake mot. rej. ↑
MOS Sp. ↑
Final motion-aware RM (all components)
100%
100%
+0.57
w/o distillation anchor
87%
100%
+0.40↓
w/o fake-motion negatives
100%
53% (chance) ↓
+0.49
w/o motion input block
47% (chance) ↓
67%
+0.56
Table 5: Leave-one-out ablation of the motion-aware RM: each row removes a single component from the final configuration and reports what breaks. Hold-2 detection is measured on a held-out editing model; fake-motion rejection is on held-out sources with per-frame jitter negatives; MOS Spearman is on VE-Bench real edits (editing models disjoint from training). Each component targets a distinct axis: motion input block for hold-2 detection, fake-motion negatives for fake motion rejection, and distillation anchor for human MOS alignment.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1 : Motion ladders for three VE-Bench edits. Columns show six uniformly sampled frames. Hold-2 and hold-4 repeat every second or fourth frame, respectively; frozen repeats the first frame. Variants retain the clip length but use subsets of the original frames.
Model
native motion
human MOS
field comp. ↓
VideoScore ↓
VideoReward ↓
TokenFlow [ 6 ]
11.69
5.70
93%
80%
100%
T2V-Zero [ 11 ]
11.12
2.94
100%
87%
100%
FRESCO [ 39 ]
10.46
4.98
100%
100%
100%
TuneAVideo [ 35 ]
9.93
4.73
100%
60%
100%
FLATTEN [ 4 ]
9.60
5.12
100%
93%
93%
RAVE [ 10 ]
8.86
5.46
100%
67%
100%
Appendix
Table S1: Per-model capture rates on VE-Bench (lower is better), alongside native motion and human MOS. VideoReward uses its motion-quality head; Table 1 reports overall and per-head results.
Candidate
claude-opus-4.8 score ( 1 – 7 )
real edit
5.29
hold-2 (subtle)
5.38
hold-4 (subtle)
5.29
frozen (extreme)
3.71
Appendix
Table S2: Mean VLM quality scores on the UniVideo motion ladder ( 1 – 7 scale). The judge rates frozen clips lowest but assigns similar scores to original edits and hold variants.
Current instruction-guided video editing models struggle to simultaneously balance precise semantic modifications with faithful motion preservation. While existing approaches rely on injecting explicit external priors (e.g., VLM features or structural conditions) to mitigate these issues, this reliance severely bottlenecks model robustness and generalization. To overcome this limitation, we present SAMA (factorized Semantic Anchoring and Motion Alignment), a framework that factorizes video editing into semantic anchoring and motion modeling. First, we introduce Semantic Anchoring, which establishes a reliable visual anchor by jointly predicting semantic tokens and video latents at sparse anchor frames, enabling purely instruction-aware structural planning. Second, Motion Alignment pre-trains the same backbone on motion-centric video restoration pretext tasks (cube inpainting, speed perturbation, and tube shuffle), enabling the model to internalize temporal dynamics directly from raw videos. SAMA is optimized with a two-stage pipeline: a factorized pre-training stage that learns inherent semantic-motion representations without paired video-instruction editing data, followed by supervised fine-tuning on paired editing data. Remarkably, the factorized pre-training alone already yields strong zero-shot video editing ability, validating the proposed factorization. SAMA achieves state-of-the-art performance among open-source models and is competitive with leading commercial systems (e.g., Kling-Omni). Code, models, and datasets will be released.
Xinyao Zhang, Wenkai Dong, Yuxin Song +10
Baidu · Tsinghua University · City University of Hong Kong +1
Video editing spans diverse editing paradigms, yet achieving high-quality instruction-guided and subject-guided editing within a single unified framework remains challenging. We introduce EditVid, a training-free framework combining sparse causal memory for local coherence, correspondence-based post-attention token injection for long-range identity preservation, and soft latent blending for edit locality. The same framework supports instruction-guided and reference-guided edits, including style transfer, attribute modification, object insertion, part-level editing, and subject replacement. On FiVE, EditVid achieves 78.16 FiVE-Acc, compared with 58.95 for the strongest evaluated training-free baseline, while obtaining competitive results on IVEBench. A user study further shows a 51.8% overall preference for EditVid over 7 competing methods.
Adheesh Sunil Juvekar, Onkar Kishor Susladkar, Kiet A. Nguyen +6
Video editing models achieve strong performance in modifying appearance, backgrounds, styles, and motions, empowered by large-scale training data. Beyond general scenarios, professional motions are significant in performance, sports, and martial arts. This paper focuses on professional motion editing, which involves fine-grained motion technique and temporal structure. Given professional motion names but lacking detailed descriptions, existing models often fail to generate professional motions, such as incorrectly lifting the front wheel under a Stoppie instruction. To address this challenge, we introduce MotionCoach, a trained multimodal agent that uses multi-turn tools and performs self-verification, producing a grounded prompt and a reference clip as reliable and sufficient multimodal guidance to inject external professional motion knowledge into video editing models. To supply training data, we build a data pipeline to construct trajectories. We cold-start the agent on MotionCoach-SFT-10k and apply Group Relative Policy Optimization (GRPO) on MotionCoach-RL-2k with rewards on the motion guidance prompt, the motion outcome video, and the format. For evaluation, we introduce VPMEBench, comprising 314 professional motion editing tasks with fine-grained references and annotations. Extensive experiments show that MotionCoach improves the overall score from 55.1 to 65.6 on Seedance 2.0 and from 51.2 to 64.5 on MiniMax-H3. Learning to construct multimodal guidance enables precise professional motion editing without retraining video editing models.
Siyi Qian, Hanqing Yang, Weihuang Lin +4
Peking University · Alibaba Token Hub, Alibaba Group · University of Science and Technology of China +1