ScaleWorld: High-Resolution World Generation through Shared Prediction and Refinement
Organizations: Xiamen University · Southeast University · City University of Hong Kong · National University of Singapore
Abstract
High-resolution generation enables video world models to render fine-grained scene structures and textures, but directly generating high-resolution video incurs substantial computational and memory costs. We present ScaleWorld, a causal video model that unifies low-resolution world prediction and high-resolution refinement within a single shared backbone. By refining its own predictions, ScaleWorld produces high-resolution world videos without requiring a separate refinement model. For each video chunk, the shared backbone first completes a low-resolution prediction and then performs one-step high-resolution refinement conditioned on that prediction. To support efficient streaming generation, we maintain separate causal states with a long low-resolution history for prediction and a short high-resolution history for refinement. To address task interference during distillation, we separate specialist optimization from capability consolidation and introduce task-asymmetric on-policy distillation (TA-OPD). This procedure transfers prediction and refinement behaviors from specialized teachers into a shared student through local velocity matching under their respective sampling and history policies. {Extensive experiments demonstrate that ScaleWorld achieves high-quality world generation with improved computational and memory efficiency.
Figures & tables
| Property | WM | Ref |
|---|---|---|
| Output resolution | low | high |
| Conditions | image, camera, text | LR latent, text |
| Sampling | multi-step | single-step |
| History span | long | short |
| History construction | clean recommit | noisy-forward |
| Method | Average | Video Quality | Setting Adherence | Interaction Adherence | Consistency | Physical |
|---|---|---|---|---|---|---|
| (a) Complete HR generation | ||||||
| Separate cascade ( ) | 81.58 | 83.72 | 78.21 | 86.80 | 87.04 | 72.13 |
| Shared (clean-recommit KV) | 81.53 | 83.83 | 77.81 | 86.98 | 87.01 | 72.03 |
| Shared (noisy-forward KV) | 81.60 | 83.43 | 78.73 | 87.08 | 86.92 | 71.82 |
| (b) Native world prediction without refinement | ||||||
| Matrix-Game 3.5 | 74.760 | 78.46 | 72.10 | 83.12 | 79.63 | 60.51 |
| World model | Refiner | Avg. | Qual. | Set. | Inter. | Cons. | Phys. |
|---|---|---|---|---|---|---|---|
| WorldPlay | Ultra Flash | 78.630 | 79.69 | 72.27 | 85.62 | 87.43 | 68.13 |
| WorldPlay | FlashVSR | 79.160 | 80.08 | 73.09 | 86.52 | 87.09 | 69.01 |
| LingBot-World 2.0 | Ultra Flash | 81.070 | 81.94 | 79.29 | 84.73 | 84.36 | 75.03 |
| LingBot-World 2.0 | FlashVSR | 81.030 | 81.98 | 79.86 | 84.93 | 84.44 | 73.96 |
| Matrix-Game 3.5 | Ultra Flash | 73.990 | 78.56 | 70.81 | 83.43 | 79.48 | 57.69 |
| Matrix-Game 3.5 | FlashVSR | 75.060 | 78.72 | 72.53 | 83.14 | 79.51 | 61.41 |
| Refiner | MUSIQ | CLIP- IQA+ | LR PSNR | MC L1 | MC HF L1 |
|---|---|---|---|---|---|
| FlashVSR center-crop protocol: output | |||||
| FlashVSR Full | 61.9861 | 0.5167 | 23.4840 | 0.0327 | 0.0140 |
| Full frame: | |||||
| Ultra Flash v1.1 | 53.7366 | 0.4417 | 22.2844 | 0.0330 | 0.0090 |
| Ours (clean) | 56.7657 | 0.5156 | 25.3340 | 0.0208 | 0.0081 |
| Ours (noisy) | 58.9656 | 0.5185 | 26.2409 | 0.0212 | 0.0086 |
| Metric | Mixed DMD | CD + mixed DMD | TA-OPD |
|---|---|---|---|
| Avg. | 72.000 | 75.878 | 80.308 |
| Qual. | 79.82 | 75.72 | 83.04 |
| Set. | 63.94 | 64.53 | 77.28 |
| Inter. | 76.79 | 87.98 | 86.05 |
| Cons. | 79.96 | 82.44 | 85.64 |
| Phys. | 59.49 | 68.72 | 69.53 |
| Training step | Global cosine | Negative blocks |
|---|---|---|
| 400 | 2/30 | |
| 600 | 29/30 | |
| 700 | 25/30 | |
| 800 | 30/30 |
| Metric | Clean- recommit | Noisy-forward (reference) |
|---|---|---|
| HR forwards | 2 | 1 |
| HR Q | 56.7657 | 58.9656 |
| PSNR | 25.3340 | 26.2409 |
| TE | 0.0208 | 0.0212 |
| Generator time (s) | 0.60 | 0.30 |
| Generator FPS | 26.7 | 53.3 |
| Metric | Resident | Two copies | Shared copy |
|---|---|---|---|
| Copies | 2 | 2 | 1 |
| Params (B) | 10.28 | 10.28 | 5.14 |
| Weight GiB | 19.16 | 19.16 | 9.58 |
| KV GiB | 1.77 | 1.77 | 1.77 |
| Peak GiB | 45.42 | 45.42 | 35.84 |
| FPS | 10.77 | 11.94 | 11.94 |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Role | Pixels | VAE latent | Patch grid |
|---|---|---|---|
| WM | |||
| Ref |
| Property | WM | Ref |
|---|---|---|
| Condition | image, camera, text | LR latent, text |
| Training layout | ||
| Nominal indices | ||
| Temporal span | 25 latent frames | 8 latent frames |
| Rollout | 25 latent frames | 8–16 latent frames |
| Supervised blocks | 6 content blocks | final block |
| WM teacher | Ref teacher | WM Inter. | WM Cons. | HR Q | TE |
|---|---|---|---|---|---|
| 86.05 | 85.64 | 58.9656 | 0.0212 |
| System | Avg. | Qual. | Set. | Inter. | Cons. | Phys. | MUSIQ | FPS |
|---|---|---|---|---|---|---|---|---|
| Cascade ( ) | 81.58 | 83.72 | 78.21 | 86.80 | 87.04 | 72.13 | 59.2371 | 10.77 |
| (clean-recommit KV) | 81.53 | 83.83 | 77.81 | 86.98 | 87.01 | 72.03 | 56.7657 | 9.41 |
| (noisy-forward KV) | 81.60 | 83.43 | 78.73 | 87.08 | 86.92 | 71.82 | 58.9656 | 11.94 |