World Editing: Intervening on Executable Worlds at Increasing Depth
Authors: Max Ku, Nok-Kan Law, Yu-Chien Tang, Shih-Ying Yeh, Ping Nie, Andy Zheng, Tat Hei Lai, Fei-Yueh Chen, Nikko Yu, Wei-Chieh Sun, Suzy Huang, Chiao-Wei Hsu, Chih-Chuan Huang, Chak-Wing Mak, Ho Yin Sam Ng, Edisy Kin Wai Chan, Min-Hung Chen, Ho Kei Cheng
Organizations: University of Waterloo · G-G-G · Comfy Org Research · University of Illinois Urbana-Champaign
Interactive world models are increasingly capable of generating environments and acting within them, yet deliberately editing an existing executable world remains underexplored. We formulate world editing as intervening on an existing world while preserving properties that should remain unchanged, and introduce intervention depth as an axis describing how strongly an edit couples world entities, dynamics, and systems. We instantiate this capability through industry-grade game modding and introduce IGMWorld, together with IGMBench, a benchmark of 110 tasks and over 1.1K executable state and behavioral criteria across Minecraft and Terraria. The tasks span property, entity, dynamics, and system interventions and are evaluated through deterministic executability, behavioral, preservation, and visual checks. Frontier coding agents already exhibit substantial world-editing capability: the strongest configuration solves 78.2% of tasks under a strict task-level criterion, while criterion-level performance reaches 94.8%. Reliability generally decreases with intervention depth, and this pattern persists even among tasks with similar numbers of evaluation criteria. Most failed edits still build and load successfully, suggesting that the main difficulty is making the edited world behave as requested. Visual consistency remains a separate weakness, with all evaluated configurations below 50% joint visual pass rate. These results show that world editing is a distinct capability from world generation and interaction, and that executable games provide a practical testbed for studying it.
Figures & tables
Level
World Intervention
Game Modding
Examples
L1
Property Intervention : Modify properties of existing components without changing their identity or underlying behavior.
Parameter Editing : Modify parameters, configurations, or appearance of existing game components.
Increase weapon damage; replace a monster’s visual asset.
L2
Entity Intervention : Introduce new entities or content while largely preserving existing dynamics.
Content Editing : Add new content that conforms to existing mechanics.
Add a crafting recipe or item; add a monster or NPC.
L3
Dynamics Intervention : Modify rules governing entity behavior and interaction.
Mechanic Editing : Modify game logic governing behaviors and interactions.
Add an air-dash mechanic; add a status-effect system.
L4
System Intervention : Modify multiple coupled components and interactions forming a world-level system.
System Editing : Modify multiple interacting game subsystems.
Add a skill tree; overhaul the economy system.
Table 1: World-editing taxonomy and its realization in game modding.
Figure 1: Overview of IGMWorld . A world-editing agent modifies a scaffold mod repository from a natural-language request and produces an edited world, which is evaluated for executability, state and behavioral correctness, and visual consistency with existing in-game assets.
L1
L2
L3
L4
Total
Game
Tasks
Criteria
Tasks
Criteria
Tasks
Criteria
Tasks
Criteria
Tasks
Criteria
Minecraft
14
82
15
144
14
169
14
226
57
621
Terraria
13
92
14
162
13
154
13
132
53
540
Total
27
174
29
306
27
323
27
358
110
1161
Table 2: Distribution of tasks and evaluation criteria across games and world-intervention levels. Each task specifies a natural-language world-edit request and multiple state or behavioral criteria. A task succeeds only if the implementation builds and passes all its criteria.
L1
L2
L3
L4
Overall
Model
CPR
WSR
CPR
WSR
CPR
WSR
CPR
WSR
CPR
WSR
DeepSeek-V4-Pro
92.1
74.1
80.8
24.1
47.7
18.5
47.0
11.1
63.3
31.8
GLM-5.3
97.6
92.6
82.7
62.1
39.8
25.9
39.3
11.1
60.1
48.2
Kimi-K3
98.5
88.9
78.3
65.5
61.0
51.9
73.3
33.3
74.9
60.0
Gemini 3.5 Flash
87.7
77.8
86.4
55.2
83.5
37.0
68.8
59.3
81.0
57.3
Claude Opus 4.8
99.6
92.6
92.8
72.4
94.7
66.7
89.4
55.6
93.6
71.8
Table 3: Criterion Pass Rate (CPR) and World-Editing Success Rate (WSR) on IGMBench , broken down by world-intervention level. CPR measures the fraction of individual state and behavioral criteria satisfied, while WSR measures the fraction of world-editing tasks for which all associated criteria are satisfied. Results are aggregated across Minecraft and Terraria. Visual correctness is evaluated separately in Table 4 .
Minecraft
Terraria
Model / Source
N
Passstyle
Passsem
Passboth
N
Passstyle
Passsem
Passboth
Vanilla (LOO)
1989
0.85
0.85
0.80
15456
0.85
0.85
0.78
DeepSeek-V4-Pro
142
0.17
0.23
0.15
113
0.16
0.23
0.11
GLM-5.3
226
0.49
0.53
0.40
73
0.32
0.37
0.29
Kimi-K3
205
0.52
0.54
0.47
109
0.29
0.35
0.24
Gemini 3.5 Flash
110
0.32
0.36
0.22
104
0.32
0.32
0.22
Table 4: Visual quality of agent-produced assets. N counts registered texture assets; vanilla textures and missing files fail all checks. Passstyle , Passsem , and Passboth denote style, semantic, and joint pass rates, computed using the text-conditioned consistency score Sf(x;c) in Equation 2 and the calibrated category-specific thresholds. Vanilla (LOO) provides the in-game reference, with marginal style and semantic pass rates of 1−α=0.85 by construction.
Figure 2: Reliability across intervention depth. Performance differences persist within comparable criterion-count ranges, showing that intervention depth is not reducible to requirement count alone.
Figure 3: Failure structure across intervention depth. Behavioral failures dominate across all levels. Among behavioral failures, interaction/progression failures increase from L2 to L4, while resource/registration failures remain relatively stable. Numbers inside bars denote failure counts.
Figure 4: Representative successful world edits in Minecraft (top) and Terraria (bottom) across L2–L4. From left to right, the examples introduce a new entity, add new dynamics, and modify a coupled world-level system.
Figure 5: World editing beyond the games included in IGMBench . We show qualitative examples of property, entity, and dynamics interventions in PEAK, Palworld, and Starbound. These examples illustrate that the intervention taxonomy applies to additional executable game worlds, but they are not part of the benchmark.
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Representative failure cases across intervention levels in Minecraft (top) and Terraria (bottom). Columns show entity (L2), dynamics (L3), and system (L4) interventions. Failures include missing registration or assets, incorrect mechanic execution, and unintended system-level behavior.
Figure 7: Minecraft item textures grouped by generation method: left , image-generated; middle , native game assets; right , programmatically drawn. Image-generated assets tend toward smoother shading and higher contrast, while programmatic assets are generally flatter and more geometric than native Minecraft textures. Each panel shows 36 sampled item textures.
Figure 8: Terraria item textures grouped by generation method: left , image-generated; middle , native game assets; right , programmatically drawn. Image-generated assets more closely reproduce Terraria’s dense and outlined visual style, while programmatic assets remain visibly simpler and more geometric. Each panel shows sampled item textures from the same size range.
L1
L2
L3
L4
Overall
Model
BSR
CPR
WSR
BSR
CPR
WSR
BSR
CPR
WSR
BSR
CPR
WSR
BSR
CPR
WSR
Open-weight
DeepSeek-V4-Pro
100
94.9
76.9
100
88.4
42.9
61.5
33.0
23.1
69.2
66.1
23.1
83.0
68.9
41.5
GLM-5.3
100
100
100
92.9
85.0
64.3
84.6
13.5
23.1
46.2
12.0
0.0
81.1
53.0
47.2
Kimi-K3
100
99.4
92.3
85.7
71.5
64.3
84.6
41.8
61.5
84.6
69.3
53.8
88.7
67.8
67.9
Proprietary
Appendix
Table 5: Performance on IGMBench across games and world-intervention levels. Build Success Rate (BSR) measures the percentage of submissions that build and load successfully. Criterion Pass Rate (CPR) measures the percentage of individual state and behavioral criteria satisfied, with criteria associated with non-building or non-loading submissions counted as failed. World-Editing Success Rate (WSR) measures the percentage of tasks that build and load successfully and satisfy all associated state and behavioral criteria. WSR columns are shaded, and the highest reported value in each column is shown in bold. All values are percentages.
Model / Source
N
Custom ↑
Style S−τ↑
Passstyle↑
Sem. S−τ↑
Passsem↑
Passboth↑
Minecraft (item, block, entity)
Vanilla (LOO)
1989
–
+0.041
0.85
+0.062
0.85
0.80
DeepSeek-V4-Pro
142
1.00
-0.026
0.17
-0.029
0.23
0.15
GLM-5.3
226
0.94
+0.001
0.49
+0.003
0.53
0.40
Kimi-K3
205
0.94
-0.005
0.52
-0.000
0.54
0.47
Gemini 3.5 Flash
110
0.95
-0.009
0.32
-0.011
0.36
0.22
Appendix
Table 6: Detailed visual-quality results by game. N counts registered texture assets. Custom is the fraction using custom artwork rather than a vanilla texture or missing file; non-custom assets fail all visual checks. S−τc denotes the margin over the category-specific TPIPS threshold, with positive values indicating a pass; Both Pass requires both style and semantic checks to pass.
Model
Regression pass rate
Open-weight
DeepSeek-V4-Pro
99.1%
GLM-5.3
96.9%
Kimi-K3
99.6%
Proprietary
Gemini 3.5 Flash
98.2%
Appendix
Table 7: Regression preservation on IGMBench . A regression check verifies that a property not targeted by the requested edit remains unchanged, using a paired comparison between the modded world and a mod-free world under the same environment. IGMBench contains 225 regression checks across 30 tasks (14 Minecraft and 16 Terraria), primarily at L1. We report pass rates over checks reached by each configuration; runs that fail to build or load do not execute regression checks.
Minecraft
Terraria
L1
L2
L3
L4
L1
L2
L3
L4
Model
Token
Time
Token
Time
Token
Time
Token
Time
Token
Time
Token
Time
Token
Time
Token
Time
Open-weight
DeepSeek-V4-Pro
1.3M
8m
2.5M
14m
3.8M
19m
4.4M
19m
0.4M
2m
1.8M
10m
2.7M
13m
5.4M
20m
GLM-5.3
2.0M
11m
12.7M
27m
11.3M
27m
12.9M
31m
4.3M
15m
10.9M
25m
11.3M
27m
10.6M
27m
Kimi-K3
1.1M
14m
4.8M
44m
6.6M
47m
6.8M
38m
2.3M
18m
4.5M
25m
9.5M
32m
9.6M
35m
Appendix
Table 8: Cost efficiency of evaluated agents across games and intervention levels. We report the median processed input tokens and agent runtime, where M denotes millions of tokens and m denotes minutes. The lowest value in each column is bolded.
Figure 9: Failure modes in agent-produced world edits. The outer ring shows broad categories and the inner ring shows specific failure types; labels report counts and percentages of categorized failures.
Model
Build
Load
Behavior
DeepSeek-V4-Pro
2
13
60
GLM-5.3
11
3
44
Kimi-K3
5
1
39
Gemini 3.5 Flash
4
2
42
Claude Opus 4.8
0
2
27
GPT-5.6 Luna
0
4
35
Appendix
Table 9: Observed failure stage by model across audited unsuccessful runs. Counts indicate whether failures occur during build, load, or behavioral validation.
Observed domain
Count
Interaction / progression
101
Mechanic semantics
84
Resources / registration
53
API / loader
33
Incomplete delivery
18
Evaluation attribution
18
Appendix
Table 10: Observed failure domains across audited unsuccessful runs. These categories describe where failures are observed and do not always identify the underlying root cause.
GPT-5.6 Sol
GPT-5.6 Luna
Gemini 3.5 Flash
Claude Opus 4.8
Read game sources
97%
99%
52%
91%
Read vanilla data
56%
52%
24%
37%
Searched the Web
9%
9%
72%
3%
Used provided skills
100%
96%
93%
58%
Rebuilt after initial build
85%
90%
83%
76%
Ran game server
15%
19%
0%
6%
Appendix
Table 11: Agent development behavior across all IGMBench runs. Values are the percentage of runs exhibiting each behavior. Trajectory-level behaviors are reported only for harnesses whose execution traces are observable.
L1
L2
L3
L4
Overall
Configuration
CPR
WSR
CPR
WSR
CPR
WSR
CPR
WSR
CPR
WSR
Luna (w/ skills)
96.3
92.6
90.2
58.6
90.4
44.4
80.0
63.0
88.5
64.5
Luna (w/o skills)
97.8
92.6
89.8
44.8
93.3
59.3
78.5
33.3
89.2
57.3
Appendix
Table 12: Effect of host-provided skills on GPT-5.6 Luna. We report Criterion Pass Rate (CPR) and World-Editing Success Rate (WSR) across intervention levels, pooled over both Minecraft and Terraria. All values are percentages.
Game
Configuration
Passstyle
Passsem
Passboth
Minecraft
Luna (w/ skills)
0.16
0.20
0.14
Luna (w/o skills)
0.22
0.36
0.18
Terraria
Luna (w/ skills)
0.18
0.36
0.12
Luna (w/o skills)
0.21
0.38
0.19
Appendix
Table 13: Effect of host-provided skills on visual asset quality for GPT-5.6 Luna. We report art-style, semantic, and joint visual pass rates for Minecraft and Terraria.
CSD
TPIPS art style
Majority vote
0.54
0.66
Appendix
Table 14: Agreement with majority human judgments of whether agent-produced textures match the game’s art style, measured by threshold-free AUC. Nine raters with prior gaming experience evaluated 40 blind-sampled textures; unsure responses were excluded before majority voting.
Model
Harness
Effort
Image Tool
Released
Open-weight
DeepSeek-V4-Pro ( DeepSeek-AI et al., 2026 )
Hermes Agent
high
Nano Banana
2026-08
GLM-5.3 ( GLM-5-Team et al., 2026 )
Hermes Agent
high
Nano Banana
2026-08
Kimi-K3 ( Team et al., 2026 )
Hermes Agent
high
Nano Banana
2026-07
Closed-source
Gemini 3.5 Flash ( Google DeepMind, 2026a )
Gemini-CLI
high
Nano Banana
2026-05
Appendix
Table 15: Agent configurations evaluated on IGMBench . Each configuration specifies a backbone model, coding harness, inference effort, and image-generation backend. All models support approximately 1M-token context windows.
Minecraft
Terraria
Game / server
Minecraft 1.21.1
tModLoader v2026.05.3.0
Mod loader
Fabric loader 0.19.3
tModLoader v2026.05.3.0
Build tool
Gradle 8.12
tML-build
Runtime
Java 21
bundled .NET 8.0
Mod API
Fabric API 0.116.14
tModLoader API
Probe channel
RCON+mineflayer
tModLoader ModSystems
Appendix
Table 16: The two game docker environments used at IGMWorld .
Game
Phase
L1
L2
L3
L4
All levels
Fresh-world load
46.2
44.4
45.2
44.2
44.8
Minecraft
Behavior checks
236.1
582.8
421.1
1,738.1
632.9
Combined
269.6
615.9
463.8
1,770.2
670.2
Fresh-world load
35.3
35.1
34.5
35.7
35.2
Terraria
Behavior checks
80.9
79.0
68.0
68.0
78.3
Combined
116.0
110.5
102.5
101.4
112.1
Appendix
Table 17: Evaluation runtime per task, reported as the 95th percentile in seconds. Measurements were collected on a machine with two AMD EPYC 7302 processors (32 cores total) and 503 GiB of RAM, excluding build and orchestration overhead. Using the overall combined p95 of 508.1 s gives a conservative sequential budget of 15.52 hours for all 110 tasks. In practice, evaluating the full benchmark with five parallel workers takes approximately 6 hours.
Figure 10: Interface used for the preliminary human perceptual study. Raters judged whether agent-produced assets were stylistically compatible with the shown vanilla references and semantically plausible for the requested object category.
Art Style vs. style votes
Semantic vs. object votes
α
Acc.
Rej.
Bal.
Acc.
Rej.
Bal.
Mean Bal. ↑
Vanilla
0.01
0.92
0.09
0.51
0.90
0.22
0.56
0.53
0.98
0.02
0.88
0.27
0.58
0.80
0.33
0.57
0.57
0.97
0.05
0.73
0.45
0.59
0.63
0.33
0.48
0.54
0.92
0.10
0.58
0.64
0.61
0.53
0.56
0.54
0.58
0.85
0.15
0.38
0.82
0.60
0.47
0.78
0.62
0.61
0.78
Appendix
Table 18: Calibration of the pass threshold α using the preliminary human study. Nine raters judged 40 agent-produced textures blind to metric scores. Acc. is the fraction of majority-accepted textures passed by the metric, Rej. the fraction of majority-rejected textures rejected by the metric, and Bal. their mean. Vanilla is the fraction of original game assets passing both checks. The shaded row is the setting used in the main experiments.
Metric
k
Pass
Rank corr.
Same verdict
Model order
Art Style
1
0.32
0.948
0.91
0.95
3
0.32
0.993
0.97
1.00
5
0.33
1.000
1.00
1.00
10
0.37
0.989
0.94
0.98
20
0.42
0.969
0.88
1.00
Semantic
1
0.37
0.943
0.88
0.93
Appendix
Table 19: Sensitivity to the neighbour count k at α=0.15 over 1,760 scored agent textures in sprite-like categories. Thresholds are recalibrated by leave-one-out for each k . Pass is the agent pass rate, Rank corr. is the Spearman correlation of texture rankings with k=5 , Same verdict is the fraction retaining the same pass/fail decision, and Model order is the Spearman correlation of model-level pass rates with those at k=5 . The shaded rows indicate the setting used in the main experiments.
Figure 11: Example prompt provided to the agent for an example world-editing task. The prompt specifies the requested modification, evaluation-facing behavioral requirements, project location, build command, and generic execution rules.
Figure 12: Sample task specification from IGMBench . The task specification records benchmark metadata, the intervention level, the natural-language request, and the evaluation criteria describing the desired world edit.
Figure 13: Sample evaluation specification from IGMBench . The verification view defines the controlled game setup and executable checks used to determine whether the requested world edit is correctly realized.
Figure 14: Task genre distribution across all 110 tasks in IGMBench . Percentages indicate the share of tasks in each genre.
Figure 15: Task genre distributions in IGMBench by game. (a) Minecraft (57 tasks); (b) Terraria (53 tasks). Percentages are computed within each game.
We present EditWorld, a video world model for precise editing and flexible referencing in interactable worlds. Existing video world models primarily focus on navigation, letting users explore generated worlds but offering limited control over how existing world content is modified. EditWorld extends world modeling from exploration to precise modification by streaming editing instructions and reference images during autoregressive generation. To support these capabilities, EditWorld introduces Gated Causal Attention for temporally varying editing conditions and reference images, together with a Sparse Context mechanism that maintains a bounded historical context for long-horizon inference. We further adopt joint autoregressive and bidirectional training with annealed self-resampling, and construct a dedicated data synthesis and annotation pipeline that provides supervision for world editing. We also present WBench-Editing to systematically evaluate streaming world editing capabilities. EditWorld achieves the best overall performance on WBench-Editing with an overall score of 73.8 and an editing score of 80.0, substantially outperforming existing methods on editing-related metrics. https://github.com/leoisufa/EditWorld
Evaluating world models requires assessing both the quality of the worlds they generate and their consistency and responsiveness under exploration, interaction, and modification. We introduce HappyWorld-Bench, a comprehensive benchmark that evaluates whether generated worlds remain reliable as agents interact with them. Our design is built on a hierarchical capability framework of six world capabilities (W1-W6), from generative construction to unified world modeling, instantiated across three independent evaluation tracks: video world models, spatial world models, and embodied world models. HappyWorld-Bench comprises 1,138 video prompts, 300 spatial scenes, and 254 embodied test cases. Across all three tracks, we build and operate HappyWorld-Arena to organize human A/B comparisons and derive model-level Elo ratings, which complement newly designed automated metrics that capture behavioral correctness. We evaluate 14 video world models, 9 spatial systems, and 8 embodied candidates under this unified framework. Results reveal remaining reliability gaps across all three tracks: video models exhibit reduced consistency during extended rollouts and revisits, spatial models achieve at best 70.14% placement accuracy and 73.33% edit execution, and embodied models struggle to preserve state across multi-step actions and respond precisely to altered action conditions and physical rules. These findings highlight the need to evaluate world models not only by visual quality, but also by state consistency and the correctness of their responses to actions and interventions.
Zhiqi Bai, Junai Cai, Yixin Chen +33
Alibaba Token Hub, Alibaba Group · State Key Laboratory of General Artificial Intelligence, BIGAI · Tsinghua University +2
Video world models have shown immense promise for interactive simulation and entertainment, but current systems still struggle with two important aspects of interactivity: user control over the environment for reproducible, editable experiences, and shared inference where players hold influence over a common world. To address these limitations, we introduce an explicit external memory into the system, a persistent state operating independent of the model's context window, that is continually updated by user actions and queried throughout the generation roll-out. Unlike conventional diffusion game engines that operate as next-frame predictors, our approach decomposes generation into Memory, Observation, and Dynamics modules. This design gives users direct, editable control over environment structure via an editable memory representation, and it naturally extends to real-time multiplayer rollouts with coherent viewpoints and consistent cross-player interactions.