Probing Physics in Motion Diffusion Models Against Their Own Estimate
Organizations: Independent Researcher, Toronto, Canada
Abstract
Recent studies report that video diffusion models "know more than they show": physical properties decode better from their activations than from their inputs or outputs. But a diffusion model computes its clean-sample estimate from those same activations, so such a comparison measures how easily a readout extracts the information. Whether the model hides anything is a separate question. We set out to find hidden physical knowledge in kinematic text-to-motion diffusion models, where the physics of every intermediate estimate can be computed exactly, and recorded every layer along the sampling trajectories of MDM (two checkpoints) and MotionDiffuse. Against the usual baselines the layers do look ahead of the model's own estimate on balance and flight. With every readout tuned the same way, the estimate closes the gap: for balance and flight, adding a layer gains at most 0.01, a size that a positive control shows we would detect. The fall rate of jumps is the one clear exception. Early in denoising, MDM's layers add about 0.08 in (up to 0.11 in a refit) that the estimate does not yet show, and the gain fades as sampling proceeds; on MotionDiffuse it is small and not robust. The estimate also flags floating samples without training, but aborting them saves under a tenth of the compute. In a video model, plausibility that linear probes miss in Wan 2.1's clean latent becomes readable after one fixed nonlinearity.
Figures & tables
| Measure | 50-step (3 seeds) | 1000-step (3 seeds) |
|---|---|---|
| Floating, % of frames, jump prompts | 0.6–0.8 | 4.8–9.3 |
| Floating, % of frames, test captions | 3.8–4.7 | 8.6–9.7 |
| Jump gravity, median (m/s 2 ), jump prompts | 9.54–9.75 | 9.13–9.22 |
| Jump gravity, median (m/s 2 ), test captions | 9.07–9.84 | 8.59–9.78 |
| Raised contact ( 5 cm), % of contact frames, jump prompts | 1.2–2.4 | 3.0–8.4 |
| Raised contact ( 5 cm), % of contact frames, test captions | 2.9–3.3 | 4.9–5.4 |
| Model | Target | Best layer | Explicit | Estimate, linear | Noisy input |
|---|---|---|---|---|---|
| 50-step | Dynamic balance ( ) | 0.838 0.001 | 0.783 0.009 | 0.723 0.007 | 0.383 0.006 |
| Static balance ( ) | 0.759 0.014 | 0.769 0.019 | 0.673 0.014 | 0.241 0.007 | |
| Lowest foot ( ) | 0.868 0.022 | 0.921 0.005 | 0.817 0.029 | 0.396 0.022 | |
| In flight (AUC) | 0.973 0.002 | 0.876 0.005 | 0.935 0.008 | 0.801 0.010 | |
| Flight gravity ( ) | 0.820 0.035 | 0.707 0.079 | 0.547 0.053 | 0.209 0.046 | |
| 1000-step | Dynamic balance ( ) | 0.697 0.029 | 0.655 0.008 | 0.550 0.031 | 0.175 0.003 |
| Tuned on held-out prompts | Gain from adding the layer [95% CI] | |||||||
|---|---|---|---|---|---|---|---|---|
| Target | Model | Explicit | Layer, ridge | Estimate | Layer | Both | Pooled score | Within prompt and frame |
| Dynamic balance | 50-step | 0.784 | 0.850 | 0.889 | 0.863 | 0.889 | [ , ] | [ , ] |
| Dynamic balance | 1000-step | 0.655 | 0.720 | 0.779 | 0.765 | 0.777 | [ , ] | [ , ] |
| Dynamic balance | MotionDiffuse | 0.124 | 0.620 | 0.823 | 0.700 | 0.823 | (all folds) | (all folds) |
| Static balance | 50-step | 0.771 | 0.780 | 0.830 | 0.800 | 0.828 | [ , ] | [ , ] |
| Static balance | 1000-step | 0.548 | 0.592 | 0.671 | 0.647 | 0.671 | (all folds) | (all folds) |
| Readout of the clean VAE latent | Per-video (%) | Pairwise (%) |
|---|---|---|
| Linear in the latent | ||
| Pooled over time and space | 50.1 0.5 | 52.1 1.5 |
| Time kept | 51.5 0.7 | 50.4 1.3 |
| Time + space | 50.9 1.1 | 54.4 1.2 |
| Time + space + signed changes | 52.8 1.1 | 52.2 2.2 |
| Linear after a fixed nonlinearity: absolute frame-to-frame change | ||
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Estimate | Layer | ||||||
| Target | Model | raw, ridge | raw, MLP | all it shows, ridge | all it shows, MLP | ridge | MLP |
| Dynamic balance | 50-step | 0.767 | 0.869 | 0.866 | 0.889 | 0.850 | 0.862 |
| Dynamic balance | 1000-step | 0.594 | 0.762 | 0.746 | 0.779 | 0.720 | 0.765 |
| Dynamic balance | MotionDiffuse | 0.583 | 0.825 | 0.606 | 0.823 | 0.620 | 0.700 |
| Static balance | 50-step | 0.697 | 0.798 | 0.825 | 0.830 | 0.780 | 0.801 |
| Static balance | 1000-step | 0.544 | 0.646 | 0.641 | 0.671 | 0.592 | 0.647 |
| Target | Model | Gain with fallback | Gain without fallback | Estimate kept | Gain, rescaled |
|---|---|---|---|---|---|
| Dynamic balance | 50-step | [ , ] | [ , ] | 4 | +0.001 |
| Dynamic balance | 1000-step | [ , ] | [ , ] | 3 | 0.006 |
| Dynamic balance | MotionDiffuse | (all folds) | [ , ] | 5 | +0.000 |
| Static balance | 50-step | [ , ] | [ , ] | 2 | 0.004 |
| Static balance | 1000-step | (all folds) | [ , ] | 5 | +0.000 |
| Static balance | MotionDiffuse | (all folds) | [ , ] | 5 | +0.000 |
| 20% | 40% | 60% | 80% | |||||
|---|---|---|---|---|---|---|---|---|
| Model | SNR | SNR | SNR | SNR | ||||
| MDM, 50-step | 0.094 | 0.104 | 0.341 | 0.517 | 0.617 | 1.614 | 0.879 | 7.273 |
| MDM, 1000-step | 0.094 | 0.104 | 0.341 | 0.517 | 0.646 | 1.825 | 0.898 | 8.781 |
| MotionDiffuse | 0.002 | 0.002 | 0.026 | 0.027 | 0.194 | 0.240 | 0.656 | 1.910 |
| Added to the estimate | Pooled score | Within prompt and frame | |
|---|---|---|---|
| Dynamic balance ( ) | the layer (real) | [ , ] | [ , ] |
| injected, 0.005 | [ , ] | [ , ] | |
| injected, 0.01 | [ , ] | [ , ] | |
| injected, 0.02 | [ , ] | [ , ] | |
| In flight (AUC) | the layer (real) | [ , ] | [ , ] |
| injected, 0.005 | [ , ] | [ , ] |
| Model | Target | Explicit | Estimate, linear | Estimate, MLP | Layer, linear | Layer, MLP | Estimate + layer, MLP |
|---|---|---|---|---|---|---|---|
| 50-step | Dynamic balance ( ) | 0.783 0.009 | 0.723 0.007 | 0.878 0.005 | 0.838 0.001 | 0.829 0.018 | 0.867 0.007 |
| Static balance ( ) | 0.769 0.019 | 0.673 0.014 | 0.816 0.028 | 0.759 0.014 | 0.712 0.059 | 0.791 0.028 | |
| In flight (AUC) | 0.876 0.005 | 0.935 0.008 | 0.972 0.001 | 0.972 0.003 | 0.964 0.004 | 0.967 0.008 | |
| Flight gravity ( ) | 0.707 0.079 | 0.547 0.053 | 0.703 0.068 | 0.814 0.040 | 0.846 0.010 | 0.828 0.038 | |
| 1000-step | Dynamic balance ( ) | 0.655 0.008 | 0.550 0.031 | 0.747 0.014 | 0.697 0.029 | 0.725 0.026 | 0.746 0.022 |
| Static balance ( ) | 0.546 0.019 | 0.508 0.066 | 0.620 0.043 | 0.557 0.063 | 0.570 0.058 | 0.601 0.057 |
| Model | Target | Explicit | Estimate, linear | Estimate, MLP | Layer, linear | Layer, MLP | Estimate + layer, MLP |
|---|---|---|---|---|---|---|---|
| 50-step | Dynamic balance ( ) | 0.797 0.011 | 0.756 0.007 | 0.892 0.003 | 0.851 0.000 | 0.867 0.002 | 0.889 0.002 |
| Static balance ( ) | 0.746 0.020 | 0.644 0.025 | 0.801 0.023 | 0.729 0.015 | 0.749 0.020 | 0.795 0.019 | |
| In flight ( ) | 0.710 0.007 | 0.562 0.014 | 0.755 0.010 | 0.662 0.010 | 0.734 0.007 | 0.760 0.008 | |
| Flight gravity ( ) | 0.439 0.053 | 0.219 0.047 | 0.340 0.039 | 0.445 0.071 | 0.536 0.109 | 0.508 0.081 | |
| 1000-step | Dynamic balance ( ) | 0.736 0.004 | 0.664 0.008 | 0.816 0.001 | 0.772 0.011 | 0.800 0.006 | 0.817 0.003 |
| Static balance ( ) | 0.607 0.010 | 0.572 0.010 | 0.679 0.003 | 0.610 0.010 | 0.642 0.010 | 0.673 0.005 |
| Floating, test captions (% of frames) | Jump gravity, curated jump prompts (m/s 2 ) | Early rule at 20% | ||||
|---|---|---|---|---|---|---|
| Setting | 50-step | 1000-step | 50-step | 1000-step | Precision | Recall |
| Default | 3.8–4.7 | 8.6–9.7 | 9.54–9.75 | 9.13–9.22 | 0.73–0.89 | 0.44–0.68 |
| Contact heights | 6.4–7.6 | 13.1–13.8 | 9.68–9.92 | 9.03–9.40 | 0.66–0.78 | 0.46–0.58 |
| Contact heights | 2.8–3.6 | 6.1–6.6 | 9.39–9.72 | 9.18–9.38 | 0.77–0.94 | 0.49–0.69 |
| Support rule 10 cm | 3.8–4.7 | 8.7–9.7 | 9.54–9.75 | 9.13–9.22 | 0.72–0.91 | 0.45–0.65 |
| Support rule 20 cm | 3.8–4.7 | 8.6–9.6 | 9.54–9.75 | 9.13–9.22 | 0.74–0.90 | 0.44–0.67 |
Motion Diffusion Models
Motion diffusion models generate, edit, reconstruct, and refine articulated movement from text, audio, video, and other conditions.
See all 26 papers