We present WorldStream, an open-ended 3D world model with three simultaneous capabilities: it extends playable terrain beyond any pre-specified map, acts on open-ended language in the running world, and restores visited places with their edits. A hierarchical planner over a typed decision model emits structured decision state: Macro fixes settlement, ecology, transport, and hydrological structure across a group of tiles; Micro specializes that contract into tile semantics under measured terrain and accepted neighbor commitments. Deterministic local code compiles accepted plans into terrain, roads, settlements, vegetation, and entities. Streaming separates their reach: plans run ahead along the predicted corridor, compilation only where movement will soon need it, geometry behind the user released. One world record retains plans, seeds, object identities and mutable state, boundary commitments, and edits by attribute, revised as the user advances and edits; returning tiles recompile from it. The same record grounds interaction: the model acts on existing entities, including stateful NPCs that update as they act, edits the present region, or attaches intent to unplanned regions. Together they yield a world expanding under bounded budgets, connected across independently planned tiles, and interactive.
Figures & tables
Figure 1: Representative WorldStream v1 runtime regions include dense city blocks, a rural settlement, streamed forest, desert terrain, and coastal terrain (top). A shared retained-state interface grounds language actions in the running world: users can ask for directions, direct nearby actors, edit present terrain, and summon distant actors (bottom).
Figure 2: WorldStream workflow. World knowledge conditions a hierarchical Macro–Micro planner; local code compiles the accepted plan into a streamed, interactive 3D world. Plans, identities, seeds, and edits persist independently of rendered geometry, enabling consistent interaction and revisit.
Figure 3: WorldStream planning and realization. (a) Macro planning combines geographic evidence with cross-tile commitments. (b) A selected tile expands into a structured Micro plan encoding land use, roads, facilities, vegetation, water setbacks, and entity anchors. (c) Local generators realize that plan as connected roads, varied buildings, civic space, and a planted waterfront.
Figure 4: Terrain-and-asset realizations from persisted generated worlds. Panels show the same local-generation interface across constructed and natural settings, preserving accepted plans and geographic inputs.
Figure 5: Directional demand and budgeted admission on a square tile lattice. Dark green tiles are required immediately; light green tiles extend the visible and near-future buffer; pale blue tiles are speculative motion- or camera-facing candidates; gray tiles lie outside current demand. A blue outline marks a speculative tile that survives the planning budget and receives a semantic plan, but remains outside the shorter construction horizon. White arrows show observed travel direction. On square tiles, northeast motion crosses the north and east boundaries together and produces an L-shaped first-arrival front; the third panel shows corridor recomputation after an observed turn.
Request
Semantic decision
Runtime realization
Object query
Reference an entity or place
Read state and derive its spatial relation
Entity action
Target + action with parameters
Update state; navigate, avoid, animate
Entity creation
Type + motion intent in a spawn context
Load the asset and start its controller
Future intent
Desired semantics for an unplanned area
Retain intent for later hierarchical planning
Region edit
Operation + relations over semantic fields
Recompile terrain, networks, ecology, assets
Table 1: Language-interaction classes and the division between semantic decisions and local execution.
Figure 6: Sequential visual cases from one continuous WorldStream runtime tour recorded at 1920 × 1080. The ordered samples show interaction and traversal through urban, rural, agricultural, forest, transition, desert, and coastal environments. Panels (a), (c), (e), (f), and (h) are crops of their source frames to make the recorded dialogue, rural dwelling, farmer and crops, forest–desert boundary, rocks, and cacti legible at three-column scale; the remaining panels show complete game-view frames.
Figure 7: Hierarchical planning ablation on the same four-Macro city-to-rural request. (a) Macro only provides settlement envelopes and a connected road skeleton but no local content. (b) Micro only uses accepted neighbors without a regional objective, retains natural land cover, and fails to form the requested settlement. (c) Flat Micro makes independent local decisions and lacks city-scale organization. (d) Without Micro-neighbor context, Macro centers and road ports remain, but tiles are repeatedly overbuilt and abrupt city/non-city cuts appear. (e) The Full hierarchy produces a connected, bounded urban-to-rural gradient.
Selection method
Overall
Biome
Terrain
Macro
Suburban
Landsc.
Water
Independent
37%
55%
69%
74%
82%
84%
88%
Joint solver
100%
100%
100%
100%
100%
100%
100%
Gain
× 2.70
× 1.82
× 1.45
× 1.35
× 1.22
× 1.19
× 1.14
Table 3: Overall and per-constraint success rates on shared model-produced probability distributions. Overall success requires all hard constraints; the final row reports multiplicative gain over independent selection.
Figure 8: Prefetch ablation over 100 route replays. (a) Four methods are compared on arrival coverage and 4 km waiting reduction; directional prefetch prepares almost the full 3×3 neighborhood before arrival. (b) Relative reductions obtained by adaptive lookahead over the directional policy on four lower-is-better metrics; raw directional and adaptive values are shown beside each metric.
Request family
Correct
Tested distinction
Actor / landmark
19 / 20
Action or direction grounded to a known target
World editing
18 / 20
Present-region edit or future-region intent
Clarification
19 / 20
Ambiguous scope or multiple actions
No action
10 / 10
Negation , explanation, or quoted example
Unavailable
10 / 10
Missing target or unsupported operation
Table 4: Accuracy of mapping open-ended requests to the intended world operation or safe non-execution.
Dense forest
Forest– wetland
Light village
Park
Light mountain
Farmland
Coast– wetland
Before (s)
10.780
5.102
1.196
1.218
0.164
0.176
0.205
After (s)
3.323
2.699
0.718
0.960
0.159
0.171
0.188
Speedup
× 3.24
× 1.89
× 1.67
× 1.27
× 1.03
× 1.03
× 1.09
Table 5: Wall-clock time for compiling accepted semantic plans into publishable tile records before and after the complete optimization. Both versions publish identical objects, asset identities, transforms, and instance counts. The final row reports compilation speedup.
Interactive world models continually generate video by responding to a user's actions, enabling open-ended generation capabilities. However, existing models typically lack a 3D representation of the environment, meaning 3D consistency must be implicitly learned from data, and spatial memory is restricted to limited temporal context windows. This results in an unrealistic user experience and presents significant obstacles to downstream tasks such as training agents. To address this, we present PERSIST, a new paradigm of world model which simulates the evolution of a latent 3D scene: environment, camera, and renderer. This allows us to synthesise new frames with persistent spatial memory and consistent geometry. Both quantitative metrics and a qualitative user study show substantial improvements in spatial memory, 3D consistency, and long-horizon stability over existing methods, enabling coherent, evolving 3D worlds. We further demonstrate novel capabilities, including synthesising diverse 3D environments from a single image, as well as enabling fine-grained, geometry-aware control over generated experiences by supporting environment editing and specification directly in 3D space. Project page: https://francelico.github.io/persist.github.io
We introduce CoDeR, a new paradigm for world modeling. Unlike existing video world models that implicitly represent world dynamics through visual observations, our system explicitly constructs an executable world with code and employs video generation models for visual realization. Specifically, we coordinate five complementary roles to translate high-level concepts into structured world rules, executable dynamics, and perceptual observations. This design enables long-term memory, open-ended interactions, autonomous world evolution, and multi-agent scenarios, where multiple entities can act, interact, and evolve persistently beyond the current observation. Extensive experiments demonstrate that our framework substantially extends the capabilities of existing world models, enabling long-term memory, open-ended interactions, autonomous evolution, and persistent multi-agent dynamics, while achieving state-of-the-art performance across multiple evaluation settings. Code and model weights will be made publicly available. Project Page: CoDeR.
Existing world models can generate action-conditioned video from a single image. Many focus on camera movement, allowing users to explore the generated scene. However, navigation alone does not ensure that an action follows persistent rules when its outcome depends on state that is invisible in the current frame. We present Stateful Harness, a framework that separates world evolution from video rendering to support stateful and rule-governed interaction. It maintains an evolving world state under specified rules, renders each resulting event with a frozen video model, and uses the generated clip to update the world state for the next turn. At each turn, the harness resolves the action selected by the user and compiles the resulting event into an instruction, while a preview clip from an editable 3D scene provides spatial and camera guidance. To evaluate rule compliance, we build WorldState-Bench, a benchmark of 50 counterfactual twin pairs that share an initial frame and action but differ in hidden state and required outcome. Under rule-only prompting, the best of the 11 evaluated video backends produces the required endings for both cases in 32% of pairs. For MiniMax-H3, specifying the required endings raises this rate from 32% to 64%, while the full harness reaches 92% with the same frozen model. These results support separating rule execution from video rendering for world interactions with hidden state.
Chan Yan Ki, Yanran Zhang, Wenzhao Zheng +5
Department of Automation, Tsinghua University, China · GigaAI