Towards Data-Efficient, Real-Time World Models
Organizations: Microsoft, Cambridge, United Kingdom · Now at Google DeepMind. · Now at Prometheus. · Now at the University of Oxford. ·
Abstract
State-of-the-art world models have been used to produce sequences of gameplay that accord with provided user-input actions. However, high quality, consistent gameplay generation often comes at the cost of inference speed, making real-time interactive play and ideation challenging. The substantial quantities of data used to train state-of-the-art models have also made custom training of these models appear impractical, limiting many of their potential uses in creative fields. In this work we demonstrate a major step towards efficient world models: achieving both fast high-quality inference, and improved data efficiency. This is done by replacing the next-token prediction paradigm with a two-stage discrete diffusion-based approach, introducing a lightweight refinement transformer which carries out iterative masked predictions. Building on an autoregressive world model capable of generating highly consistent and complex sequences over minutes (Kanervisto et al., 2025), we achieve substantial model speed-up (79x) with a modest quality trade-off. Additionally, we demonstrate that this model can be trained to reproduce high-quality, consistent gameplay using a dataset containing approximately one day (28 hours) of gameplay data. Our results demonstrate progress towards removing barriers to the use of world models in settings where real-time inference and data efficiency are essential.
Figures & tables
| Model | FVD | FPS |
|---|---|---|
| Backbone only (no refinement) | 147.7 | 21.7 |
| WHAM-RT (no context) | 2576.1 | 18.2 |
| WHAM-RT (random compressed context) | 825.5 | 17.3 |
| WHAM-RT | 115.0 | 17.3 |
| Frames | WHAM | WHAM-RT full | h curated | h random |
|---|---|---|---|---|
| Density | Coverage | |||
|---|---|---|---|---|
| Frames | Curated | Random | Curated | Random |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
World Model Learning
World model learning builds predictive models of how environments change under actions to support planning, simulation, and decision-making.
See all 456 papers