Momentum: Persistent Dynamics for Video World Models
Organizations: INSAIT, Sofia University ”St. Kliment Ohridski” · Harvard University · University of Ulm · TUM ·
Abstract
Video world models must capture how a scene evolves beyond the current camera view to generate consistent observations when its content reappears. Existing approaches rely on observation history, static point clouds, or image caches constructed from past frames. These representations are often expensive and, more importantly, cannot update dynamic scene content outside the field of view: generated vehicles and pedestrians may be forgotten after leaving the field of view or reappear in physically inaccurate states. We represent an evolving scene through synchronized videos from reference-camera rigs, each comprising eight cameras that jointly cover the surrounding scene. We connect these rigs in a graph and propagate video generation between neighboring rigs to coordinate scene dynamics across locations and time. We introduce Momentum, a two-stage pipeline: WorldUpdate jointly generates reference-camera videos as a shared memory of scene dynamics, and WorldInteract grounds camera-controlled video generation in this evolving memory, allowing moving content to remain represented outside the target view and reappear consistently. To support training and evaluation, we develop a CARLA pipeline that captures synchronized videos from reference camera rigs and a moving target camera, together with an evaluation framework measuring consistency within the generated world state and the target observations.Experiments on generated street scenes show that grounding target videos in scene dynamics generated by Momentum improves dynamic scene consistency by 126.2% in unseen scenes over static image conditioning, while also improving visual fidelity. Our models and source code will be made publicly available.
Figures & tables
| Method | mPSNR | mLPIPS | RotErr | TransErr |
|---|---|---|---|---|
| SWM Seo et al. (2026) (reported) | 15.18 | 0.481 | 0.055 | 0.154 |
| Statically Grounded World Model (ours) | 16.30 | 0.426 | 0.025 | 0.238 |
| Method | Cond. | PSNR | LPIPS | RotErr | TransErr | SCS-static | SCS-dyn | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| warp sources: | 1 src | 3 src | 1 src | 3 src | 1 src | 3 src | 1 src | 3 src | 1 src | 3 src | 1 src | 3 src | |
| test 1: seen towns | |||||||||||||
| Statically Grounded WM | image | 19.89 | 19.55 | 0.426 | 0.429 | 0.038 | 0.034 | 0.160 | 0.173 | 0.526 | 0.503 | 0.078 | 0.074 |
| Statically Grounded WM | video | 20.35 | 20.04 | 0.414 | 0.419 | 0.030 | 0.029 | 0.156 | 0.172 | 0.562 | 0.540 | 0.127 | 0.107 |
| Dynamically Grounded WM (ours) | video | 23.16 | 22.89 | 0.335 | 0.337 | 0.022 | 0.020 | 0.056 | 0.053 | 0.652 | 0.647 | 0.265 | 0.230 |
| test 2: unseen town | |||||||||||||
| Method | PSNR | LPIPS | RotErr | TransErr | SCS-static | SCS-dyn |
|---|---|---|---|---|---|---|
| test 1: seen towns | ||||||
| Statically Grounded WM | 19.89 | 0.426 | 0.038 | 0.160 | 0.526 | 0.078 |
| Dynamically Grounded WM (image) | 22.02 | 0.358 | 0.026 | 0.060 | 0.623 | 0.086 |
| Momentum | 24.03 | 0.341 | 0.026 | 0.080 | 0.658 | 0.198 |
| test 2: unseen town | ||||||
| Statically Grounded WM | 20.27 | 0.449 | 0.026 | 0.148 | 0.573 | 0.081 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Method | PSNR | LPIPS | SCS-static | SCS-dyn | Stat | Stat Gap |
|---|---|---|---|---|---|---|
| test 1: seen towns | ||||||
| Persistent Baseline | 33.66 | 0.118 | 0.807 | 0.208 | 0.993 | 0.565 |
| Dynamically Grounded WM | 30.96 | 0.166 | 0.800 | 0.193 | 0.870 | 0.480 |
| Multi-Rig DGWM | 30.06 | 0.191 | 0.841 | 0.124 | 0.498 | 0.204 |
| test 2: unseen town | ||||||
| Persistent Baseline | 31.88 | 0.110 | 0.801 | 0.228 | 0.985 | 0.593 |
Video World Models
Video world models generate and predict action-conditioned visual futures to simulate changing scenes for embodied agents.
See all 241 papers