Large video diffusion models offer expressive priors for embodied prediction and learning, yet their many-step sampling remains costly for interactive downstream use. Distribution Matching Distillation (DMD) enables few-step video generation, but can suppress robot--object motion while preserving visual quality. Examining DMD's teacher and fake-score signals, we find that weak re-noising keeps the teacher posterior concentrated near motion-deficient rollouts, limiting motion-restoring guidance. Meanwhile, stronger-motion rollouts tend to incur larger fake-score fitting errors, which can hinder the generator's learning of interaction dynamics. We propose DyMD, a DMD framework that adapts both teacher supervision and critic fitting to the evolving student. Temporal affinity--conditioned re-noise sampling adapts the timestep distribution to each rollout's current interaction fidelity by mixing the base schedule with a teacher prior motivated by local posterior variation, thereby balancing motion recovery and appearance refinement. To better track stronger-motion rollouts, dynamics-guided fake-score tracking uses a noise-conditioned predictor to estimate noise-relative fitting difficulty from latent temporal dynamics, then upweights predicted-hard rollouts in the critic loss. Using DyMD, we distill a 14B teacher into a four-step 1.3B student with no auxiliary modules at inference. On embodied-video benchmarks, the student improves R-Bench task adherence by 9.6 percentage points and PAI-Bench-G Domain score by 5.1 points over Base DMD while maintaining comparable visual quality. As a backbone for downstream action planning, our student achieves 34% mean success across two WorldArena tasks, compared with 16% for Base DMD.
Figures & tables
Figure 1: Preserving interaction dynamics in few-step video world models. Top: Base DMD shows implausible deformation and limited motion; DyMD preserves a physically plausible robot–object interaction. Bottom: gains in R-Bench task adherence, PAI-Bench-G domain score, and WorldArena planning success. Both are 1.3B four-step students.
Figure 2
Figure 4: Overview of DyMD. Top (Secs. 3.2 – 3.3 ): temporal affinity controls each rollout’s mixture of the teacher prior and base schedule. Middle (Sec. 3.1 ): the teacher–critic score difference updates the student. Bottom (Sec. 3.4 ): a dynamics- and noise-conditioned predictor assigns mean-one weights to critic FM losses without changing their targets. Generator and critic share the timestep sampling rule; all auxiliary modules are training-only.
Figure 5: Teacher-velocity turning profile. Mean turning rate of future-block-centered velocity across PF-Wan teacher trajectories. The light-blue band shows the interquartile range across trajectories; pale-yellow shading highlights the high-noise region σ∈[0.9,1] .
Method
Params
NFE
R-Bench: TAC
PAI-Bench-G
EZS-Bench
Overall
Dual
Hum.
Single
Quad.
Domain
Space
Physics
Time
Quality
Domain
Space
Physics
Time
PF-Wan
14B
80
75.3
72.5
79.8
71.8
77.2
89.0
91.8
92.2
88.4
76.9
83.2
80.9
89.2
78.0
Wan2.1-Fun
1.3B
100
47.3
42.5
53.2
30.8
63.2
79.0
79.7
82.5
76.4
75.3
75.5
71.7
83.9
68.8
Base DMD
1.3B
4
34.6
25.2
33.4
25.8
54.0
74.4
77.0
82.8
64.7
78.1
73.3
66.9
86.0
64.8
DyMD
1.3B
4
44.2
38.8
48.0
31.2
58.8
79.5
82.4
81.2
73.5
78.1
76.2
71.8
84.4
70.8
Table 1: Main comparison on embodied-video benchmarks. The teacher and undistilled student are reference models and are separated from the two matched four-step DMD variants. Bold with blue shading marks the better result within the matched pair. NFE counts denoiser evaluations, including both CFG branches. Benchmark scores are percentages and higher is better. R-Bench embodiment columns are abbreviated as Dual (dual-arm), Hum. (humanoid), Single (single-arm), and Quad. (quadruped).
Figure 6: Visual comparison of interaction dynamics between Base DMD and DyMD at four NFE. Uniformly spaced frames from two PAI-Bench cases. Top: lifting a blue shipping container. Bottom: handing a menu to a person. Base DMD fails to lift the target container and produces limited handover motion, whereas DyMD realizes the instructed interaction.
Method
Components
R-Bench
PAI-Bench-G
EZS-Bench
Sampling
Tracking
TAC
Domain
Quality
Domain
Base DMD
×
×
34.6
74.4
78.1
73.3
w/o Tracking
✓
×
43.5
78.5
78.1
75.1
w/o Sampling
×
✓
38.3
75.4
78.0
74.0
DyMD
✓
✓
44.2
79.5
78.1
76.2
Table 2: Sampling and fake-score tracking ablation. Sampling denotes temporal affinity–conditioned re-noise sampling; Tracking denotes dynamics-guided critic weighting. All variants retain the critic and use the same update and inference budgets. Bold with blue shading marks the best result in each column.
Figure 7: Representations for generator sampling and critic tracking. (a) Rank comparison between temporal affinity and CoTracker3-based trajectory agreement across 32 student–target video pairs. Solid lines show linear fits and the dashed diagonal denotes identical rankings. (b) Spearman ρ between MLP predictions and log-FM loss, computed separately at each noise level on a 32-rollout bank. FM losses are averaged over three independent noise draws per rollout and noise level.
Sampling schedule
R-Bench
PAI-Bench-G
EZS-Bench
TAC
Domain
Quality
Domain
Base DMD
34.6
74.4
78.1
73.3
Shift-25
40.0
77.2
78.2
74.6
Teacher prior
42.5
77.9
78.0
75.0
Affinity-conditioned
43.5
78.5
78.1
75.1
Table 3: Re-noise sampling-schedule ablation. We compare Base DMD with a high-noise control (Shift-25, raising the flow shift from 5 to 25), the teacher prior, and our temporal affinity–conditioned sampler. All variants disable critic reweighting ( β=0 ). Bold with blue shading marks the best result in each column.
β
R-Bench
PAI-Bench-G
EZS-Bench
TAC
Domain
Quality
Domain
0.1
43.8
79.1
78.0
75.4
0.5
44.2
79.5
78.1
76.2
1.0
43.6
78.4
78.1
75.1
Table 4: Sensitivity to critic-weighting strength β . We vary only β , with α=0.3 . The β=0.5 row repeats the full-model results from Table 1 . Bold with blue shading marks column-wise best values, including ties.
Model
Adjust Bottle
Click Bell
Avg.
Genie Envisioner
10.0
20.0
15.0
TesserAct
1.0
35.0
18.0
RoboMaster
8.0
20.0
14.0
Vidar
2.0
19.0
10.5
WoW
20.0
21.0
20.5
Wan2.2-TI2V-5B
12.0
20.0
16.0
Table 5: WorldArena action-planner success rate (%). Upper block: results reported in PhysisForcing [ 36 ] . Lower block: our 1.3B backbones with separately trained, task-specific VPP heads. Bold with blue shading compares the lower block only.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Stage
Criterion
Input
Output
Deduplication
greedy cover, 0.95
2,824,039
448,528
Image–text
top- 50% cosine
448,528
224,264
Motion
tracks ≥50 , vis. ≥0.6
224,264
32,897
Appendix
Table S1: Training-data curation funnel.
Noise σ
0.5
0.8
0.97
Predictor ρ
0.758
0.898
0.952
Appendix
Table S2 : Difficulty ranking by the trained predictor. Predictor ρ is the Spearman correlation between predicted fitting difficulty ri and measured critic FM loss on 32 fresh rollouts. FM losses average three independent noise draws at each noise level.
Quantity
Choice
Latent measurement / spatial size
21 blocks
V-JEPA input / output blocks
16 frames / 8 blocks
Temporal-affinity lags / weights
{1,2,4} / (0.2,0.3,0.5)
Teacher-prior probability floor α
0.3
Turning-rate offset ϵκ
0.1median(κ)
Critic timestep sampling
Same rule as generator
Appendix
Table S3 : Default hyperparameters for re-noise sampling and fake-score tracking.
Method
Quality
SC
BC
MS
AQ
PF-Wan 14B
76.9
90.3
90.6
98.9
50.1
Wan2.1-Fun 1.3B
75.3
87.8
91.6
98.6
47.6
Base DMD
78.1
95.3
91.9
99.5
48.6
DyMD
78.1
95.3
92.2
99.6
48.5
Appendix
Table S4 : PAI-Bench-G Quality breakdown. Quality is the unweighted mean of the eight official metrics. SC: subject consistency; BC: background consistency; MS: motion smoothness; AQ: aesthetic quality; IQ: imaging quality; OC: overall consistency; IS/IB: image-to-video subject/background fidelity. All values are percentages; higher is better. Column-wise best values, including ties, are bold and shaded blue.
Figure S1 : Additional interaction-dynamics comparisons at four NFE. Eight uniformly spaced frames are shown for each of two PAI-Bench-G cases, with prompts, seeds, and step-1500 checkpoints matched to the main qualitative results. Base DMD often preserves the initial configuration or incompletely executes the action, whereas DyMD produces clearer object-directed motion.
Encoder
R-Bench
PAI-Bench-G
EZS-Bench
TAC
Domain
Quality
Domain
V-JEPA 2.1-L
43.5
78.5
78.1
75.1
DINOv2-L
41.8
76.7
77.8
75.0
Appendix
Table S5 : Video-native versus framewise temporal-affinity features. We compare V-JEPA 2.1-L and DINOv2-L for affinity-conditioned sampling under the same training budget. Bold with blue shading marks the better result.
α
R-Bench
PAI-Bench-G
EZS-Bench
TAC
Domain
Quality
Domain
0.1
39.2
77.4
78.1
73.2
0.3
43.5
78.5
78.1
75.1
0.5
43.8
78.2
78.1
74.8
Appendix
Table S6 : Sensitivity to the teacher-prior probability floor α . Only α varies, with tracking disabled. The α=0.3 row repeats the sampling-only result from Table 2 . Column-wise best values, including ties, are bold and shaded blue.
Few-step distillation for video diffusion models has attracted significant attention, driven by the urgent demand for efficient deployment in real-world scenarios. However, Distribution Matching Distillation (DMD), a leading paradigm, tends to degrade under limited NFE budgets, manifesting in video generation as layout instability, oversaturation, and broken motion dynamics. We trace this failure to a structural limitation: standard DMD is an intra-sample distribution-matching objective with coordinate-wise gradients, and thus imposes no explicit constraint on the relational geometry across batch elements or temporal frames, leaving the underlying copula largely unregulated. Combined with the mode-seeking tendency of its reverse-KL objective, this absence of relational guidance makes DMD prone to collapsing into local optima in the few-step regime. Motivated by this insight, we propose Copula-aware DMD (CoDMD), a lightweight relational regularizer that reuses score estimates already produced by the frozen teacher and the online fake model to construct pairwise relation matrices across samples and frames. These are matched through a supplementary distributional objective that requires no additional networks, datasets, or sampling trajectories. On the Wan-2.1-T2V model series at 1.3B & 14B scales, CoDMD distills 50-step teachers into 4-step students, achieving an approximate 25× speed-up while attaining VBench scores of 84.46 & 84.87, outperforming prior trajectory-based (rCM 82.81 & 84.05) and distribution-based (DMD 83.38 & 83.81) methods.
Distribution Matching Distillation (DMD) is a widely used paradigm for accelerating inference in few-step video diffusion models. However, DMD-style video distillation faces two coupled challenges: the fake score must track a continuously evolving generator, making training costly when frequent updates are required, while reverse-KL-style matching can be mode-seeking and conservative for preserving strong motion dynamics. To address these issues, we propose Score Gradient Matching Distillation (SGMD). SGMD adopts a fake-score perspective by directly optimizing the fake score toward the teacher, while using teacher stop-gradient Fisher as a stable distribution-matching objective. We provide a gradient analysis that motivates this objective choice under ideal tracking. Building on this, SGMD introduces a pair of dual potentials: negative-residual (NR) for outer-loop correction and residual-contraction (RC) for inner-loop tracking. Empirically, compared to DMD2, SGMD achieves an approximately ∼3× training speedup and substantially improves motion dynamics for 4-step distilled models while preserving temporal consistency. A human study confirms that SGMD is preferred in motion quality and overall preference, while visual quality and text alignment remain comparable. Code is available at https://github.com/ModelTC/LightX2V.
Zhuguanyu Wu, Ruihao Gong, Yang Yong +5
Beihang University · SenseTime Research · Hong Kong University of Science and Technology
Modern video diffusion models require tens of denoising evaluations over long spatiotemporal token sequences. Distribution Matching Distillation (DMD) reduces the number of function evaluations (NFE) to just a few. However, DMD samples can degrade during training, exhibiting progressive oversaturation and artifacts. We trace this instability to critic errors, which enter successive student updates and accumulate over time. We introduce Projected Distribution Matching Distillation (PDMD) to filter critic errors. PDMD projects out the component of the DMD update parallel to the student-critic endpoint residual. At a fixed noisy query, we prove that this residual is an unbiased estimate of the critic's endpoint error. Under high-dimensional assumptions, this projection removes a constant fraction of critic error while discarding only a vanishing fraction of ideal DMD signal. Empirically, the projection stabilizes training and improves sample quality where DMD degrades and develops unnatural textures. PDMD requires only a one-line code change to DMD, with no extra loss, network, data, model pass, or multi-stage training. With Wan2.1, PDMD achieves a VBench total score of 83.73 at 4 NFE, surpassing matched DMD by 1.03 points. On MiniMax-H3 joint video-audio generation, PDMD achieves a VideoGen-Eval visual total score of 83.17, 0.41 points above the strongest distilled baseline. PDMD also achieves the best performance on all six audio metrics among the compared 4-NFE models. Qualitative comparisons and user studies favor PDMD over the distilled baselines in visual quality, motion, and audio quality. Code and models are available at https://pdmd2026.github.io/.
Zimo Wang, Junkun Yuan, Angtian Wang +9
University of California, San Diego · ByteDance Inc.