Comparing Each Generation with Independent Siblings Gives Video World Model Post-Training a Proper Objective for Outcome Frequencies
Organizations: University of Southern California · Shanghai University of Sport
Abstract
Video world models are used as simulators, so when a first frame leaves several physical outcomes open, repeated generations should produce each outcome as often as it occurs in the data. Physics post-training scores every generation against the single recorded future of its history, which makes single videos more faithful, but the expected score is linear in the model distribution and is maximized by always producing the most likely outcome. The log-likelihood of the recorded outcome is maximized by the data distribution, and its gradient weights agreement with the record by the inverse of how often the model produces the generated outcome, a frequency that other generations of the same history with independent noise, its siblings, can estimate. To train with this weight from a finite group, we propose SibPO, which performs Sibling-paired Policy Optimization: siblings estimate a truncated inverse-frequency weight for agreement with the record, and a second term subtracts the same weight times the agreement with the siblings, which makes the expected objective a strictly proper divergence over outcome cells at every group size. On PisaBench, SibPO lowers the KS distance to simulated landing times by 9.1-10.2% relative to the strongest method outside its reward family on three of four video models without degrading single videos, and it lowers the error of estimated robot policy success rates by 13%.
Figures & tables
| Method | Trained mechanisms | Held-out mechanisms | Single-video quality | GPU-h | ||||||||||||
| PisaBench | Physion | Kubric ‡ | Pisa. | Physion | PAW. | Cali. | PisaBench | Phys.-IQ | VBench | |||||||
| KS | IQR | Brier | Pear. | Acc. | KS | KS | Brier | TVD | mnTV | L2 | IoU | Score | Dyn. | |||
| Base model | 0.585 | 0.71 | 0.255 | 0.352 | 0.189 | 60.2 | 0.412 | 0.585 | 0.255 | 30.5 | 0.606 | 12.0 | 9.3 | 25.3 | 40.7 | 0.0 |
| SFT | 0.302 | 0.81 | 0.250 | 0.375 | 0.201 | 60.9 | 0.301 | 0.503 | 0.253 | 29.4 | 0.628 | 9.4 | 11.0 | 25.4 | 42.0 | 21.6 |
| Rewards against the recorded future | ||||||||||||||||
| RLVR-World [ 49 ] | 0.331 | 0.56 | 0.276 | 0.361 | 0.185 | 59.9 | 0.322 | 0.524 | 0.272 | 31.9 | 0.641 | 9.0 | 11.5 | 24.3 | 39.3 | 21.1 |
| Reward | Theory | PisaBench / Physion | |||||
| Rest. | KS | IQR | Pear. | ||||
| Single comparison | 0 | 0 | 1 | 0.334 | 0.54 | 0.352 | |
| EBFT features [ 16 ] | 0 | 1 | n/a | n/a | 0.268 | 0.76 | 0.371 |
| Success count [ 43 ] | 7 | 0 | 0.658 | 2.79 | 0.249 | 0.81 | 0.379 |
| SibPO, | 0 | 1 | 0.650 | 1.00 | 0.256 | 0.80 | 0.377 |
| SibPO, group mean | 6 | 1 | 0.651 | 3.52 | 0.243 | 0.85 | 0.384 |
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Reward | KS at group size | IQR after restart | ||||
| 10 upd. | 25 upd. | 50 upd. | ||||
| Flow-GRPO (single comparison) | 0.326 | 0.329 | 0.334 | 0.55 | 0.54 | 0.53 |
| Success count [ 43 ] | 0.287 | 0.249 | 0.244 | 0.61 | 0.67 | 0.74 |
| SibPO, | 0.259 | 0.256 | 0.254 | 0.59 | 0.62 | 0.67 |
| SibPO | 0.246 | 0.241 | 0.239 | 0.63 | 0.71 | 0.79 |
| Method | Wan2.2-5B | CogVideoX-5B | Cosmos-14B | ||||||
| KS | IQR | Brier | KS | IQR | Brier | KS | IQR | Brier | |
| Base model | 0.653 | 0.64 | 0.269 | 0.537 | 0.75 | 0.254 | 0.537 | 0.59 | 0.262 |
| SFT | 0.348 | 0.74 | 0.264 | 0.294 | 0.85 | 0.251 | 0.296 | 0.72 | 0.255 |
| Rewards against the recorded future | |||||||||
| RLVR-World | 0.371 | 0.51 | 0.287 | 0.312 | 0.61 | 0.279 | 0.352 | 0.47 | 0.281 |
| Flow-GRPO | 0.368 | 0.50 | 0.290 | 0.309 | 0.62 | 0.276 | 0.361 | 0.45 | 0.286 |
| Method | RT-1 ( ) | Real2Sim | ||||
| MSE | LPIPS | Disp. | MAE | Brier | ||
| Base model | 0.659 | 14.8 | 3.53 | 17.2 | 0.71 | 0.214 |
| SFT | 0.602 | 14.2 | 3.29 | 15.4 | 0.74 | 0.207 |
| RLVR-World (released) [ 49 ] | 0.486 | 13.4 | 2.41 | 14.6 | 0.76 | 0.219 |
| RLVR-World (retrained) | 0.491 | 13.6 | 2.47 | 14.3 | 0.76 | 0.217 |
| DynamicsBoost [ 22 ] | 0.561 | 14.0 | 3.02 | 13.8 | 0.80 | 0.209 |
| Method | Wan2.2-TI2V-5B | CogVideoX1.5-5B-I2V | ||||||||||||||
| Physion | Kub. ‡ | Pisa. | Ph.-IQ | VBench | PAW. | Cali. | Physion | Kub. ‡ | Pisa. | Ph.-IQ | VBench | PAW. | Cali. | |||
| Pear. | Acc. | KS | L2 | Score | Dyn. | TVD | mnTV | Pear. | Acc. | KS | L2 | Score | Dyn. | TVD | mnTV | |
| Base model | 0.330 | 59.3 | 0.437 | 12.8 | 23.1 | 46.0 | 31.2 | 0.587 | 0.303 | 60.8 | 0.389 | 13.4 | 27.1 | 33.0 | 29.9 | 0.641 |
| SFT | 0.361 | 59.6 | 0.322 | 10.0 | 23.8 | 46.5 | 29.9 | 0.596 | 0.338 | 59.6 | 0.288 | 10.0 | 27.8 | 35.4 | 31.1 | 0.642 |
| Rewards against the recorded future | ||||||||||||||||
| RLVR-World [ 49 ] | 0.337 | 58.8 | 0.352 | 9.3 | 22.0 | 44.1 | 32.4 | 0.611 | 0.331 | 59.0 | 0.319 | 9.4 | 27.4 | 31.8 | 32.5 | 0.618 |
| Method | Physion | Kub. ‡ | Pisa. | Ph.-IQ | VB. | PAW. | Cali. | |
| Pear. | Acc. | KS | L2 | Score | Dyn. | TVD | mnTV | |
| Base model | 0.371 | 62.0 | 0.384 | 11.2 | 27.9 | 38.6 | 29.4 | 0.592 |
| SFT | 0.389 | 62.3 | 0.284 | 8.9 | 28.4 | 39.5 | 29.0 | 0.598 |
| RLVR-World | 0.372 | 61.5 | 0.326 | 8.3 | 27.9 | 37.1 | 31.7 | 0.619 |
| Flow-GRPO | 0.366 | 61.2 | 0.331 | 8.2 | 28.9 | 37.8 | 32.0 | 0.624 |
| ORO (Seg) | 0.359 | 61.4 | 0.318 | 8.3 | 29.8 | 38.4 | 32.2 | 0.615 |
| Variant | PisaBench | Physion | ||||
| KS | IQR | Acc. | Pear. | |||
| SibPO (default) | 0.241 | 0.86 | 0.90 | 61.0 | 0.386 | 0.198 |
| Mean-of-others baseline | ||||||
| 0.249 | 0.82 | 0.86 | 60.8 | 0.380 | 0.194 | |
| 0.243 | 0.85 | 0.89 | 61.0 | 0.384 | 0.197 | |
| 0.242 | 0.86 | 0.89 | 61.1 | 0.387 | 0.197 | |
| Sampler noise | Share | |||
| Cosmos-Predict2.5-2B [ 32 ] | ||||
| Training value | 0.194 | 0.087 | 0.107 | 0.55 |
| Half | 0.188 | 0.063 | 0.125 | 0.66 |
| Wan2.2-TI2V-5B [ 44 ] | ||||
| Training value | 0.180 | 0.071 | 0.109 | 0.61 |
| Half | 0.181 | 0.052 | 0.129 | 0.71 |
| PisaBench | Physion | ||||||
| KS | IQR | Acc. | Pear. | ||||
| Flow-GRPO (single comparison) | |||||||
| 0 | 0.071 | 0.329 | 0.55 | 0.62 | 59.6 | 0.349 | 0.178 |
| 0.15 | 0.207 | 0.315 | 0.61 | 0.68 | 60.0 | 0.343 | 0.177 |
| 0.30 | 0.339 | 0.308 | 0.66 | 0.73 | 60.3 | 0.357 | 0.183 |
| SibPO (sibling pairing) | |||||||
| Method | Setting | PisaBench | Physion | |||
| KS | IQR | Pear. | ||||
| Sampler at deployment (guidance ) | ||||||
| Base model | ODE | 0.597 | 0.70 | 0.77 | 0.359 | 0.182 |
| Flow-GRPO | ODE | 0.351 | 0.54 | 0.61 | 0.350 | 0.179 |
| SibPO | ODE | 0.272 | 0.79 | 0.84 | 0.381 | 0.195 |
| Guidance scale (stochastic sampler) | ||||||
Video World Models
Video world models generate and predict action-conditioned visual futures to simulate changing scenes for embodied agents.
See all 296 papers