We introduce CoDeR, a new paradigm for world modeling. Unlike existing video world models that implicitly represent world dynamics through visual observations, our system explicitly constructs an executable world with code and employs video generation models for visual realization. Specifically, we coordinate five complementary roles to translate high-level concepts into structured world rules, executable dynamics, and perceptual observations. This design enables long-term memory, open-ended interactions, autonomous world evolution, and multi-agent scenarios, where multiple entities can act, interact, and evolve persistently beyond the current observation. Extensive experiments demonstrate that our framework substantially extends the capabilities of existing world models, enabling long-term memory, open-ended interactions, autonomous evolution, and persistent multi-agent dynamics, while achieving state-of-the-art performance across multiple evaluation settings. Code and model weights will be made publicly available. Project Page: CoDeR.
Figures & tables
Figure 1: Our conception of a world: the Creator establishes its rules, the Executors build the world accordingly, and the Artist brings it to life.
Figure 2: Method Overview. Left: The pipeline of our CoDeR. The God of Concepts conveys its intent to the Creator. The Creator (LLM) orchestrates the entire world, while the Executors (coding agents) implement the whitebox world. Finally, the Artists (video generation models) render observations along the trajectories sampled by the Travelers. Right: The Observation as World Registration paradigm. Once an observation is generated, it is registered back into the world using its depth map.
Figure 3: We sample a rotational trajectory to examine whether the “Coding World System” mark is consistently maintained on the wall. The results show that ABot-World exhibits severe quality degradation, while Matrix-Game 3.0 fails to preserve the mark. LingBot-World fails to accurately respond to the control signals, resulting in duplicated frames. In contrast, our method successfully follows the rotational trajectory while consistently preserving the mark on the wall.
Figure 4: Left: We support open-ended interactions ranging from playing the piano to riding a dragon. Right: Comparison with EgoSim, which fails to open the door, while our method successfully opens it and reveals the new environment.
Figure 5: We demonstrate the world evolution mechanism of our method in comparison with other approaches. In the first frame, a tap is pouring water into a glass cup. The camera then moves away from the cup and later returns to examine whether the water level has continued to rise. HyDRA produces a static water column, while LiveWorld exhibits obvious visual artifacts. In contrast, our method continuously updates the water level even when the cup is out of sight.
Figure 6: We further explore multi-agent scenarios in our system. When Agent 1 takes down a target, the event is simultaneously observed by Agent 2. Likewise, when Agent 2 moves forward and shoots another target, the event is also observed by Agent 1.
Methods
Camera Ctrl
Object Ctrl
Content Align
3D Consist
Photo Consist
Style Consist
Subjective Qual
Motion Acc
Motion Mag
Motion Smooth
Average
ABot-World ( Jiang et al., 2026 )
92.47
84.23
76.94
80.60
83.56
83.22
39.95
50.33
26.89
74.21
69.24
Matrix-Game 3.0 ( Wang et al., 2026e )
96.72
84.61
83.12
81.10
86.96
83.42
59.04
60.79
24.75
81.50
74.20
LingBot-World ( Robbyant Team et al., 2026 )
90.50
87.73
75.29
86.28
90.03
85.39
65.71
59.48
27.93
79.60
74.79
Ours
98.91
92.25
98.44
88.65
91.10
89.40
69.08
82.76
84.92
83.31
87.88
Table 1: Quantitative Comparison. Our method outperforms all competing methods across all metrics.
Methods
WorldScore
VBench
Content Align
Photo Consist
Subjective Qual
Imaging Quality
Aesthetic Quality
Subject Consistency
Dynamic Degree
w/o Vis. Cue Hack.
82.48
89.21
49.05
63.52
66.18
87.54
97.29
w/o Obs. as WR.
89.06
84.85
53.98
69.37
70.44
84.63
98.02
Full Method
98.44
91.10
69.08
70.77
72.29
96.31
98.16
Table 2: Ablation Study. Both Visual Cue Hacking and Observation as World Registration contribute to improved visual quality across WorldScore and VBench metrics.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: User Study. Human evaluation across Visual Quality, Interaction Consistency, Memory and Persistence, World Evolution, and Overall Preference. Our method consistently achieves the highest scores across all evaluated dimensions.
Figure 8: Effect of Visual Cue Hacking. Directly conditioning on whitebox renderings produces monotonous, render-like videos with rigid edges, whereas our Visual Cue Hacking strategy encourages the model to use the whitebox input as a structural cue, resulting in richer appearance and higher visual quality.
Figure 9: Given the first frame, we obtain its segmentation map and depth map, and then reconstruct a corresponding whitebox world.
Figure 10: Limitation of Long-Horizon Video Generation. Our Artist model is trained with a fixed 124-frame window and generates long videos through chunk-based rollout. Noticeable visual degradation emerges due to accumulated generation errors.
Figure 11: Limitations of Whitebox Modeling. Our current whitebox world may exhibit failure cases in complex hand-object interactions and 3D geometry, including severely distorted hand poses and object interpenetration.
Figure 12: Towards Disentangled Visual Generation. Future Artist models could decompose RGB observations into structured visual representations, including Albedo, Normal, Roughness, and Irradiance, enabling more explicit scene registration, editing, and re-rendering within the CoDeR.
World models aim to simulate how complex environments evolve under actions and events, yet existing video-based world models primarily learn dynamics from visual observations, which reveal outcomes rather than the underlying knowledge, rules, and mechanisms governing world evolution. This makes it difficult to maintain persistent consequences and support coherent, open-ended evolution. We introduce Code World Model, a framework that separates world evolution from visual realization by combining the reasoning and coding capabilities of language models with the generative priors of video models. A coding agent serves as the world brain, reasoning about events and their consequences and generating executable code to maintain persistent world state and perform rule-consistent evolution. To connect executable state with visual generation, we introduce a proxy representation that encodes frame-wise spatiotemporal constraints and is compiled into a proxy video, which conditions a video model to render high-fidelity visual observations. We further develop data pipelines for constructing aligned proxy-observation pairs from gameplay and real-world videos. After fine-tuning on paired gameplay data, MiniMax-H3 follows proxy-based spatiotemporal specifications from simple interactive worlds built by the coding agent while preserving rich visual details and dynamics. These results demonstrate the potential of combining code for persistent world evolution with video models for flexible visual realization, providing a new path toward open-ended world models.
Yiwen Chen, Guosheng Lin, Chi Zhang
AGI Lab, Westlake University · Nanyang Technological University
Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent world state and enforcing programmable rules over extended interactions. We introduce Programmable World Model, a framework that decouples world-state evolution from visual observation generation. An agent translates natural-language instructions into executable programs that specify entity states and state-transition rules, enabling direct control over individual entities and their interactions. A lightweight engine executes these programs to update and maintain an explicit, persistent global world state, including off-screen entities and non-visual attributes. To connect world state with visual generation, we introduce state-augmented 3D oriented bounding boxes (OBBs) as an intermediate representation. This representation, together with the target camera trajectory, is deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model serving as the generative renderer. This design allows users to create playable games with predefined mechanics, direct control over individual entities, and persistent world state throughout gameplay. We further introduce CombatStateBench, a benchmark for evaluating programmable world models. On CombatStateBench, our method achieves 94% Count Accuracy and 98% State Accuracy, substantially outperforming existing interactive video world models while supporting coherent long-horizon generation. These results demonstrate the effectiveness of separating explicit state evolution from generative rendering for building persistent, programmable worlds.
Video world models aim to simulate complex environments with realistic appearance and dynamics. Maintaining consistency over long interactions requires tracking past actions and understanding how those actions affect the environment. Video world models can only generate partial observations of the entire world at a time. Remembering past observations alone may not be sufficient in every scenario, as they may no longer reflect the world after a sequence of actions. We introduce a paradigm that decouples a persistent, updatable abstract state from the visual state. The abstract state keeps track of the entities' changes as the world evolves, and is used to condition the visual state, which represents the current view. We propose StateCraft, a novel architecture that pairs a language model with a video diffusion model to evolve the dynamic world. We evaluate our approach on generative interfaces, which pose a substantial challenge because a single action may change parts of the state that are not currently visible. To support training and evaluation, we develop a data collection pipeline that produces informative, densely annotated trajectories at scale without human or agent supervision, together with a benchmark measuring generation quality, memory, and controllability. On this benchmark, our approach improves surpasses the state-of-the-art baselines. We demonstrate that StateCraft maintains high self-consistency and strong generation quality over long trajectories. Because the abstract state is explicit, it can be edited directly, and generation follows edits to states never observed in the data. Finally, a proof-of-concept 3D experiment, in which actions change the illumination of an unobserved room, shows that the paradigm can also be applied to spatial environments. Project page: https://insait-institute.github.io/StateCraft/.
Lorenzo Venturoli, Nedko Savov, Xi Wang +2
INSAIT, Sofia University "St. Kliment Ohridski" · ETH Zurich · University of Ulm +3