Humans interpret multimodal observations of the world and forecast how it will evolve. Recent multimodal generative systems have demonstrated similar capabilities: they can integrate multimodal context and synthesize visual outputs that depict changes in the world. To characterize this emerging trajectory, we organize model capabilities into a four-level hierarchy and discover that higher-level tasks remain relatively underexplored despite model progress. To bridge this gap, we explore model performance on multi-step world state prediction, a high-level task that requires models to forecast successive future world states from multimodal observations and diverse references. Such a task requires models to possess entangled understanding and generation capabilities, as well as extensive world knowledge to translate human action sequences, physical transformations, and abstract visual reasoning into coherent visual outputs. We introduce WorldCast, a benchmark comprising 320 manually designed cases across eight categories, and evaluate 12 representative models. The conducted experiments yield three key insights: (a) advances in frontier models are rapidly pushing the boundaries of intelligent world modeling, with the GPT-Image-2.5 System scoring 72.8 and demonstrating early-stage visual state prediction capabilities on real-world tasks; (b) WorldCast reveals substantial gaps between models: even Nano Banana Pro reaches only 42.8 and the best open-source model scores 20.1, with deficits in handling complex context, entangled references, and reasoning-heavy tasks; (c) strong generative models effectively draw on the capabilities of their underlying language models. These findings highlight the need for multimodal generative models that are capable of dynamic planning and reasoning while also being deeply unified to faithfully simulate real-world states.
Figures & tables
Figure 1: Our generative capability tree and the WorldCast benchmark. (a) The proposed visual generative capability tree organizes intelligent visual generation into a four-layer hierarchy (L1-L4); (b) WorldCast targets L3-L4 level capabilities of dynamically transforming real-world context into successive future world states through unified multimodal understanding and generation.
Figure 2: Evaluation scope and question example of WorldCast . The benchmark requires models to combine understanding and generation to dynamically produce evolving visual states. In this assembling task, GPT-Image-2.5 achieves key milestones while Wan 3.0 struggles in figuring out the correct order.
Figure 3: Representative examples of the eight WorldCast categories. The benchmark comprises 320 manually designed cases in total, with 1,352 requested visual states.
Figure 4: Qualitative examples on WorldCast . Models exhibit distinct failure patterns across the three evaluation dimensions; all scores are reported on a 0–3 scale.
Table 1: Overview of WorldCast performance across twelve image and video generation models and eight benchmark categories. Blue cells mark the best result within each model group, while orange cells mark the best result across all models. Icons indicate the generation format: Interleaved Generation, Grid Generation, Video Generation, Plan & Iterate.
Figure 5: Performance comparison on WorldCast . (a) Category-wise scores across L3 and L4. (b) L4 scores, with dashed extensions and labels showing the absolute gap from L3.
Figure 7
Table 2: Interleaved versus grid generation. Switching to grid generation reduces overall performance for both models, with a much larger drop for GPT-Image-2.5.
Figure 8: Visual coding as procedural guidance. Left: Result comparisons with different model harnesses. Right: code-generated intermediate states guide refinements of a towel-folding sequence.
Model Paired with GPT-6 Astra
Harness Enhancement
Observation- Conditioned Transitions
Physical Simulation
Adaptive Task Completion
Physical Properties
Visual Cues
In-Context Procedural Learning
Logic and Rules
Visual Goal Realization
Overall
GPT-Image-2.5
- (GPT-6 Responses, baseline)
85.6
69.6
66.1
73.6
76.7
71.6
74.3
61.3
72.8
Inspection loop
92.4 ( +6.8 )
86.2 ( +16.6 )
73.2 ( +7.1 )
84.4 ( +10.8 )
69.5 ( −7.2 )
72.9 ( +1.3 )
81.1 ( +6.8 )
69.2 ( +7.9 )
80.2 ( +7.4 )
Coding-first inspection loop
90.7 ( +5.1 )
88.6 ( +19.0 )
74.9 ( +8.8 )
80.1 ( +6.5 )
74.5 ( −2.2 )
74.3 ( +2.7 )
85.7 ( +11.4 )
81.3 ( +20.0 )
82.4 ( +9.6 )
Nano Banana Pro
- (GPT-6 Responses, baseline)
58.9
68.1
56.7
64.0
60.1
63.9
66.9
48.3
61.2
Inspection loop
74.7 ( +15.8 )
82.1 ( +14.0 )
65.3 ( +8.6 )
77.2 ( +13.2 )
68.6 ( +8.5 )
69.8 ( +5.9 )
74.3 ( +7.4 )
60.6 ( +12.3 )
72.5 ( +11.3 )
Coding-first inspection loop
78.1 ( +19.2 )
82.0 ( +13.9 )
65.8 ( +9.1 )
72.1 ( +8.1 )
63.3 ( +3.2 )
66.7 ( +2.8 )
81.9 ( +15.0 )
62.3 ( +14.0 )
72.9 ( +11.7 )
Table 3: Comparisons of harnesses for agentic visual generation using GPT-6 Astra as the language model. The coding-first inspection loop yields the best overall performance.
Figure 9: Validation of the automatic evaluation pipeline. (a) Correlation between human and GPT-5.5 win rates. Scores (win = 1, tie = 0.5, loss = 0) are averaged across pairwise comparisons for each model. (b) Correlation between evaluator scores from agent-selected and human-selected keyframes.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Data distribution across the eight WorldCast categories. Labels report cases/input images/requested visual states.
Figure 11: Performance across TC, CP, and VC on WorldCast .
Figure 12: Overall model scores assigned by three LMM judges.
Method
L3 Tasks
L4 Tasks
Overall
Observation- Conditioned Transitions
Physical Simulation
Adaptive Task Completion
Physical Properties
Visual Cues
In-Context Procedural Learning
Logic and Rules
Visual Goal Realization
SenseNova-U1
24.1 ±0.73
24.6 ±0.43
24.7 ±0.39
19.9 ±0.84
19.1 ±1.21
20.1 ±0.45
5.9 ±0.34
12.2 ±0.21
20.1 ±0.30
Emu3.5
17.4 ±0.43
13.8 ±0.29
18.5 ±0.84
6.6 ±0.49
10.3 ±0.71
8.1 ±0.54
4.8 ±0.50
7.6 ±0.56
12.3 ±0.16
BAGEL
4.9 ±0.43
7.0 ±0.42
8.3 ±0.28
8.5 ±0.48
12.5 ±0.11
7.0 ±1.07
2.9 ±0.32
4.0 ±0.67
6.8 ±0.13
Qwen-Image
7.8 ±0.29
6.8 ±0.16
6.2 ±0.63
4.0 ±0.56
6.9 ±0.31
6.1 ±0.49
0.6 ±0.16
3.4 ±0.21
5.6 ±0.12
GPT-Image-2.5
85.6 ±1.12
69.6 ±0.79
66.1 ±1.01
73.6 ±1.84
76.7 ±1.08
71.6 ±0.92
74.3 ±1.13
61.3 ±0.74
72.8 ±0.39
Appendix
Table 4: Model stability across repeated generations. Scores are measured by GPT-5.5 and reported as mean ± standard deviation over three generation runs. Model names and ordering follow Table 1 .
Figure 13: Usage patterns of coding artifacts under GPT-6 Astra with Codex. Top: the role and category distribution of coding artifacts across 320 cases. Bottom: representative examples using guiding overlays and simulated intermediate states as direct visual inputs.
Figure 14: Detailed Qualitative Examples, Model Outputs and Error Patterns.
World models and multimodal large language models (MLLMs) provide complementary capabilities for predicting future outcomes from static visual observations. World models can generate concrete visual rollouts of possible futures, while MLLMs can reason abstractly over questions, goals, and rules. However, generated rollouts are stochastic and may be visually plausible but task-incorrect, making it necessary to determine when visual simulation is useful, whether a rollout is credible, and how it should influence the final answer. We formulate this problem as controlled concrete reasoning, where a model learns to invoke, verify, and integrate visual future simulation alongside abstract reasoning. To study this setting, we construct two human-verified benchmarks, VRQABench for controllable spatial lookahead and OpenWorldQA for open-domain physical prediction, and propose Privileged-Future On-Policy Self-Distillation (PF-OPSD). During training, PF-OPSD uses ground-truth future videos and answers only as teacher-side privileged context to evaluate on-policy concrete-reasoning trajectories, while the deployable student never observes true futures at test time. Experimental results show that PF-OPSD outperforms baseline by 10.6% and 10.9% on VRQABench and OpenWorldQA, respectively, while increasing robustness to noisy or conflicting rollouts. Our code and dataset are available at https://github.com/yczhou001/PF-OPSD.
Yucheng Zhou, Wei Tao, Yiwen Guo +1
University of Macau · LIGHTSPEED · Independent Researcher
Robotic world models predict action-conditioned future scenes, providing a foundation for understanding action outcomes. However, turning alternative predictions into useful learning signals remains challenging: similar candidates limit informative quality comparisons, while diverging trajectories require persistent maintenance of their individual histories. We introduce FutureWorlds, a framework that unifies candidate construction, history maintenance, and learning from relative quality. Built on a multimodal discrete autoregressive model, FutureWorlds uses diverse beam search during reinforcement learning to construct candidate futures that balance confidence and diversity. Candidate-specific bounded memory preserves scene states and ensures that generation and policy scoring use matching histories. We further propose MemSPO (Memory-Conditioned Search-Guided Policy Optimization), which converts video trajectory rewards into group-relative advantages to optimize the world model. On RT-1, BridgeV2, and RoboCasa, FutureWorlds reduces LPIPS for 32-frame predictions by 14.78%, 20.84%, and 9.12%, respectively, relative to the strongest baseline on each dataset. Under fixed evaluation configurations, only 200 MemSPO updates further improve generation quality and support continued prediction beyond the training horizon. Memory ablations, decoding sensitivity analysis, and optical-flow evaluation show that these gains extend beyond visual quality to more accurate motion prediction and more consistent object states. Project page and code: https://github.com/Alexander-wu/FutureWorlds.
We introduce Orca, an initial instantiation of a general world foundation model. Orca learns a unified world latent space from multimodal world signals and exposes it through multimodal readout interfaces. Rather than optimizing isolated next-token, next-frame, or next-action prediction, we are centered on Next-State-Prediction modeling, offering a unified state-transition modeling route toward understanding, predicting, and acting upon the world. Orca learns through two complementary paradigms: unconscious learning captures dense natural state transitions from continuous videos, and conscious learning models sparse meaningful state transitions by language-described events and VQA supervision. For pre-training, we construct a large-scale world-learning inventory data, including 125K hours of video data and 160M event annotations. After pre-training, Orca learns a unified world latent space. To examine whether the learned latent supports downstream, we evaluate it by three representative downstream readouts: text generation, image prediction, and embodied action generation. Orca's backbone is frozen, and only the lightweight modality-specific decoders are trainable. Experiments show the scalability of the proposed paradigm and verify that stronger world latent enables stronger downstream readouts. Orca outperforms similar-sized specialized baselines. These results show that Orca, as a general world foundation model, presents a promising approach to understanding, predicting, and acting upon the world. Finally, we discuss the current limitations, aiming to provide useful insights and inspiration for the community.