Reward models for instruction-guided video editing can favor temporal consistency at the expense of motion. We study this failure, termed motion collapse, using controlled motion-suppression variants of real edits. On VE-Bench, the evaluated composite reward prefers a motion-suppressed candidate in 97% of cases. Learned video rewards and a VLM judge also struggle with mild suppression, while latent optimization and reward-guided fine-tuning exploit this weakness to increase reward as motion declines. A linear probe of our base reward backbone performs near chance on mild suppression, motivating an explicit inter-frame dynamics input. We propose a motion-aware reward model that combines an RGB frame-difference stream with preference training against motion-suppressed and fabricated-motion negatives. A distillation anchor preserves alignment with human quality judgments. On the VE-Bench motion ladder, our model reduces the rate of selecting motion-suppressed candidates to 0% while achieving a Spearman correlation of 0.57 with human quality ratings. It also detects unseen suppression variants and preserves motion during reward-guided fine-tuning on two editing pipelines. These results support explicitly modeling motion in rewards for video editing.
Figures & tables
Figure 1: Motion-aware reward model. The pixel stream encodes source and edited videos, while a dynamics stream computes tanh-normalized RGB differences of the edited video. Their features are fused at block 4 of the Qwen3-VL-32B backbone and adapted with LoRA. Training combines pairwise preferences over original edits and motion-suppressed or fabricated-motion negatives with a distillation anchor to the base preference reward.
Reward
genuine
hold-2
hold-4
frozen
capture ↓
Field metrics
RAFT warping error ( ↓ )
33.20
23.39
16.08
0.03
100%
DINO temporal consistency ( ↑ )
0.869
0.919
0.950
1.000
100%
Field composite (appearance + temporal)
{ z-normalized weighted combination }
97%
Field composite (appearance only)
62%
Learned video reward models
Table 1: Mean scores across the VE-Bench motion ladder and top-1 capture rates. The composite combines appearance and temporal terms; the appearance-only variant removes the latter. VideoReward is reported overall and by head.
Figure 2: Motion collapse on VE-Bench. Left: mean rewards along the motion ladder, min–max normalized per reward; VideoReward uses its motion-quality head. Right: top-1 capture rate: the fraction of cases where a motion-suppressed candidate scores highest (lower is better). Learned reward models use their overall scores.
hold-2
hold-4
flow-swap
real 2
motion attacks
(subtle)
(extreme)
fake
(control)
combined
Humans prefer real
96%
98%
100%
62%
98%
Wilson 95% CI
[92, 98]
[94, 99]
[97, 100]
[48, 74]
[96, 99]
RM agreement with human majority
Field composite
23%
13%
60%
30%
32%
VideoScore-v1.1
33%
44%
59%
50%
46%
Table 2: Human preferences and reward agreement across 100 trials from 30 sources. Top: vote fractions favoring the original edit, with Wilson 95% CIs. Bottom: agreement with the human majority, excluding tied reward predictions. The motion-attack aggregate excludes real-vs-real controls.
Figure 3: Left: motion during reward-guided fine-tuning on UniVideo, normalized to its initial value (genuine-motion subset, mean ± std). Right: leave-one-source-out linear-probe accuracy on the base reward features, with Wilson 95% CIs. Mild suppression remains near chance; see Section 4.1 .
Prior-work rewards
Ours
Optimizer regime
field
VideoScore
RAFT flow
motion-aware RM
Best-of- N (natural outputs)
≈0 (observed)
DRaFT-1 (UniVideo)
−61% / +0.05
−21% / +0.01
−35% / +0.05
+1% / +0.00
DRaFT-1 (CogVideoX-2b)
+19% / +0.02
−42% / +0.01
−88% / −0.01
+9% / +0.02
Table 3: Motion change and CLIP-T change (italic) under reward optimization. UniVideo fine-tuning results use the genuine-motion subset; CogVideoX-2b results use eight sources. DRaFT-1 [ 2 ] implements reward-guided fine-tuning. Additional analyses appear in Appendix B.6 .
Reward
Cap. ↓
h2
frzn
segFrz
burstFrz
flowAtt
Corr(mot.)
MOS Sp. ↑
Field composite
97%
6%
1%
11%
12%
34%
−0.32
+0.55
VideoScore
80%
58%
50%
40%
48%
74%
+0.15
+0.18
VideoReward
76%
35%
43%
57%
53%
40%
−0.19
+0.46
MQ head
98%
11%
7%
57%
69%
11%
+0.15
+0.09
Ours
0%
100%
100%
91%
100%
95%
−0.10
+0.57
Table 4: Reward robustness and quality alignment on VE-Bench. Cap. is ladder capture rate; h2 / frzn / segFrz / burstFrz / flowAtt are preference rates for original edits over suppressed variants. Additional variants are defined in Appendix B.2 . MQ denotes VideoReward’s motion-quality head.
Figure 4: Reward distributions on the VE-Bench motion ladder. The base reward gives overlapping scores to original edits and hold variants. The motion-aware reward lowers scores for suppressed candidates while retaining a broad distribution on original edits.
Config
hold-2 detection ↑
Fake mot. rej. ↑
MOS Sp. ↑
Final motion-aware RM (all components)
100%
100%
+0.57
w/o distillation anchor
87%
100%
+0.40↓
w/o fake-motion negatives
100%
53% (chance) ↓
+0.49
w/o motion input block
47% (chance) ↓
67%
+0.56
Table 5: Leave-one-out ablation of the motion-aware RM: each row removes a single component from the final configuration and reports what breaks. Hold-2 detection is measured on a held-out editing model; fake-motion rejection is on held-out sources with per-frame jitter negatives; MOS Spearman is on VE-Bench real edits (editing models disjoint from training). Each component targets a distinct axis: motion input block for hold-2 detection, fake-motion negatives for fake motion rejection, and distillation anchor for human MOS alignment.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1 : Motion ladders for three VE-Bench edits. Columns show six uniformly sampled frames. Hold-2 and hold-4 repeat every second or fourth frame, respectively; frozen repeats the first frame. Variants retain the clip length but use subsets of the original frames.
Model
native motion
human MOS
field comp. ↓
VideoScore ↓
VideoReward ↓
TokenFlow [ 6 ]
11.69
5.70
93%
80%
100%
T2V-Zero [ 11 ]
11.12
2.94
100%
87%
100%
FRESCO [ 39 ]
10.46
4.98
100%
100%
100%
TuneAVideo [ 35 ]
9.93
4.73
100%
60%
100%
FLATTEN [ 4 ]
9.60
5.12
100%
93%
93%
RAVE [ 10 ]
8.86
5.46
100%
67%
100%
Appendix
Table S1: Per-model capture rates on VE-Bench (lower is better), alongside native motion and human MOS. VideoReward uses its motion-quality head; Table 1 reports overall and per-head results.
Candidate
claude-opus-4.8 score ( 1 – 7 )
real edit
5.29
hold-2 (subtle)
5.38
hold-4 (subtle)
5.29
frozen (extreme)
3.71
Appendix
Table S2: Mean VLM quality scores on the UniVideo motion ladder ( 1 – 7 scale). The judge rates frozen clips lowest but assigns similar scores to original edits and hold variants.