World Editing: Intervening on Executable Worlds at Increasing Depth

Max Ku; Nok-Kan Law; Yu-Chien Tang; Shih-Ying Yeh; Ping Nie; Andy Zheng; Tat Hei Lai; Fei-Yueh Chen

cs.AIOct 1, 2026

World Editing: Intervening on Executable Worlds at Increasing Depth

Authors: Max Ku, Nok-Kan Law, Yu-Chien Tang, Shih-Ying Yeh, Ping Nie, Andy Zheng, Tat Hei Lai, Fei-Yueh Chen,

Organizations: University of Waterloo · G-G-G · Comfy Org Research · University of Illinois Urbana-Champaign

Abstract

Interactive world models are increasingly capable of generating environments and acting within them, yet deliberately editing an existing executable world remains underexplored. We formulate world editing as intervening on an existing world while preserving properties that should remain unchanged, and introduce intervention depth as an axis describing how strongly an edit couples world entities, dynamics, and systems. We instantiate this capability through industry-grade game modding and introduce IGMWorld, together with IGMBench, a benchmark of 110 tasks and over 1.1K executable state and behavioral criteria across Minecraft and Terraria. The tasks span property, entity, dynamics, and system interventions and are evaluated through deterministic executability, behavioral, preservation, and visual checks. Frontier coding agents already exhibit substantial world-editing capability: the strongest configuration solves 78.2% of tasks under a strict task-level criterion, while criterion-level performance reaches 94.8%. Reliability generally decreases with intervention depth, and this pattern persists even among tasks with similar numbers of evaluation criteria. Most failed edits still build and load successfully, suggesting that the main difficulty is making the edited world behave as requested. Visual consistency remains a separate weakness, with all evaluated configurations below 50% joint visual pass rate. These results show that world editing is a distinct capability from world generation and interaction, and that executable games provide a practical testbed for studying it.

Figures & tables

Level	World Intervention	Game Modding	Examples
L1	Property Intervention : Modify properties of existing components without changing their identity or underlying behavior.	Parameter Editing : Modify parameters, configurations, or appearance of existing game components.	Increase weapon damage; replace a monster’s visual asset.
L2	Entity Intervention : Introduce new entities or content while largely preserving existing dynamics.	Content Editing : Add new content that conforms to existing mechanics.	Add a crafting recipe or item; add a monster or NPC.
L3	Dynamics Intervention : Modify rules governing entity behavior and interaction.	Mechanic Editing : Modify game logic governing behaviors and interactions.	Add an air-dash mechanic; add a status-effect system.
L4	System Intervention : Modify multiple coupled components and interactions forming a world-level system.	System Editing : Modify multiple interacting game subsystems.	Add a skill tree; overhaul the economy system.

Table 1: World-editing taxonomy and its realization in game modding.

Figure 1: Overview of IGMWorld . A world-editing agent modifies a scaffold mod repository from a natural-language request and produces an edited world, which is evaluated for executability, state and behavioral correctness, and visual consistency with existing in-game assets.

	L1		L2		L3		L4		Total
Game	Tasks	Criteria	Tasks	Criteria	Tasks	Criteria	Tasks	Criteria	Tasks	Criteria
Minecraft	14	82	15	144	14	169	14	226	57	621
Terraria	13	92	14	162	13	154	13	132	53	540
Total	27	174	29	306	27	323	27	358	110	1161

Table 2: Distribution of tasks and evaluation criteria across games and world-intervention levels. Each task specifies a natural-language world-edit request and multiple state or behavioral criteria. A task succeeds only if the implementation builds and passes all its criteria.

	L1		L2		L3		L4		Overall
Model	CPR	WSR	CPR	WSR	CPR	WSR	CPR	WSR	CPR	WSR
DeepSeek-V4-Pro	92.1	74.1	80.8	24.1	47.7	18.5	47.0	11.1	63.3	31.8
GLM-5.3	97.6	92.6	82.7	62.1	39.8	25.9	39.3	11.1	60.1	48.2
Kimi-K3	98.5	88.9	78.3	65.5	61.0	51.9	73.3	33.3	74.9	60.0
Gemini 3.5 Flash	87.7	77.8	86.4	55.2	83.5	37.0	68.8	59.3	81.0	57.3
Claude Opus 4.8	99.6	92.6	92.8	72.4	94.7	66.7	89.4	55.6	93.6	71.8

Table 3: Criterion Pass Rate (CPR) and World-Editing Success Rate (WSR) on IGMBench , broken down by world-intervention level. CPR measures the fraction of individual state and behavioral criteria satisfied, while WSR measures the fraction of world-editing tasks for which all associated criteria are satisfied. Results are aggregated across Minecraft and Terraria. Visual correctness is evaluated separately in Table 4 .

Model / Source	$N$	$\mathrm{Pass}_{\mathrm{style}}$	$\mathrm{Pass}_{\mathrm{sem}}$	$\mathrm{Pass}_{\mathrm{both}}$	$N$	$\mathrm{Pass}_{\mathrm{style}}$	$\mathrm{Pass}_{\mathrm{sem}}$	$\mathrm{Pass}_{\mathrm{both}}$
	Minecraft				Terraria
Vanilla (LOO)	1989	0.85	0.85	0.80	15456	0.85	0.85	0.78
DeepSeek-V4-Pro	142	0.17	0.23	0.15	113	0.16	0.23	0.11
GLM-5.3	226	0.49	0.53	0.40	73	0.32	0.37	0.29
Kimi-K3	205	0.52	0.54	0.47	109	0.29	0.35	0.24
Gemini 3.5 Flash	110	0.32	0.36	0.22	104	0.32	0.32	0.22

Table 4: Visual quality of agent-produced assets.

N

counts registered texture assets; vanilla textures and missing files fail all checks.

\mathrm{Pass}_{\mathrm{style}}

\mathrm{Pass}_{\mathrm{sem}}

, and

\mathrm{Pass}_{\mathrm{both}}

denote style, semantic, and joint pass rates, computed using the text-conditioned consistency score

S_{f}(x;c)

in Equation 2 and the calibrated category-specific thresholds. Vanilla (LOO) provides the in-game reference, with marginal style and semantic pass rates of

1-\alpha=0.85

by construction.

Panel of Figure 2 — Figure 2: Reliability across intervention depth. Performance differences persist within comparable criterion-count ranges, showing that intervention depth is not reducible to requirement count alone.

Panel of Figure 3 — Figure 3: Failure structure across intervention depth. Behavioral failures dominate across all levels. Among behavioral failures, interaction/progression failures increase from L2 to L4, while resource/registration failures remain relatively stable. Numbers inside bars denote failure counts.

Figure 4: Representative successful world edits in Minecraft (top) and Terraria (bottom) across L2–L4. From left to right, the examples introduce a new entity, add new dynamics, and modify a coupled world-level system.

Figure 5: World editing beyond the games included in IGMBench . We show qualitative examples of property, entity, and dynamics interventions in PEAK, Palworld, and Starbound. These examples illustrate that the intervention taxonomy applies to additional executable game worlds, but they are not part of the benchmark.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Representative failure cases across intervention levels in Minecraft (top) and Terraria (bottom). Columns show entity (L2), dynamics (L3), and system (L4) interventions. Failures include missing registration or assets, incorrect mechanic execution, and unintended system-level behavior.

Figure 7: Minecraft item textures grouped by generation method: left , image-generated; middle , native game assets; right , programmatically drawn. Image-generated assets tend toward smoother shading and higher contrast, while programmatic assets are generally flatter and more geometric than native Minecraft textures. Each panel shows 36 sampled item textures.

Figure 8: Terraria item textures grouped by generation method: left , image-generated; middle , native game assets; right , programmatically drawn. Image-generated assets more closely reproduce Terraria’s dense and outlined visual style, while programmatic assets remain visibly simpler and more geometric. Each panel shows sampled item textures from the same size range.

Open-weight
	L1			L2			L3			L4			Overall
Model	BSR	CPR	WSR	BSR	CPR	WSR	BSR	CPR	WSR	BSR	CPR	WSR	BSR	CPR	WSR
DeepSeek-V4-Pro	100	94.9	76.9	100	88.4	42.9	61.5	33.0	23.1	69.2	66.1	23.1	83.0	68.9	41.5
GLM-5.3	100	100	100	92.9	85.0	64.3	84.6	13.5	23.1	46.2	12.0	0.0	81.1	53.0	47.2
Kimi-K3	100	99.4	92.3	85.7	71.5	64.3	84.6	41.8	61.5	84.6	69.3	53.8	88.7	67.8	67.9
Proprietary

Appendix

Table 5: Performance on IGMBench across games and world-intervention levels. Build Success Rate (BSR) measures the percentage of submissions that build and load successfully. Criterion Pass Rate (CPR) measures the percentage of individual state and behavioral criteria satisfied, with criteria associated with non-building or non-loading submissions counted as failed. World-Editing Success Rate (WSR) measures the percentage of tasks that build and load successfully and satisfy all associated state and behavioral criteria. WSR columns are shaded, and the highest reported value in each column is shown in bold. All values are percentages.

Minecraft (item, block, entity)
Model / Source	$N$	Custom $\uparrow$	Style $S{-}\tau$ $\uparrow$	$\mathrm{Pass}_{\mathrm{style}}$ $\uparrow$	Sem. $S{-}\tau$ $\uparrow$	$\mathrm{Pass}_{\mathrm{sem}}$ $\uparrow$	$\mathrm{Pass}_{\mathrm{both}}$ $\uparrow$
Vanilla (LOO)	1989	–	+0.041	0.85	+0.062	0.85	0.80
DeepSeek-V4-Pro	142	1.00	-0.026	0.17	-0.029	0.23	0.15
GLM-5.3	226	0.94	+0.001	0.49	+0.003	0.53	0.40
Kimi-K3	205	0.94	-0.005	0.52	-0.000	0.54	0.47
Gemini 3.5 Flash	110	0.95	-0.009	0.32	-0.011	0.36	0.22

Appendix

Table 6: Detailed visual-quality results by game.

N

counts registered texture assets. Custom is the fraction using custom artwork rather than a vanilla texture or missing file; non-custom assets fail all visual checks.

S-\tau_{c}

denotes the margin over the category-specific TPIPS threshold, with positive values indicating a pass; Both Pass requires both style and semantic checks to pass.

Model	Regression pass rate
Open-weight
DeepSeek-V4-Pro	99.1%
GLM-5.3	96.9%
Kimi-K3	99.6%
Proprietary
Gemini 3.5 Flash	98.2%

Appendix

Table 7: Regression preservation on IGMBench . A regression check verifies that a property not targeted by the requested edit remains unchanged, using a paired comparison between the modded world and a mod-free world under the same environment. IGMBench contains 225 regression checks across 30 tasks (14 Minecraft and 16 Terraria), primarily at L1. We report pass rates over checks reached by each configuration; runs that fail to build or load do not execute regression checks.

Open-weight
	Minecraft								Terraria
	L1		L2		L3		L4		L1		L2		L3		L4
Model	Token	Time	Token	Time	Token	Time	Token	Time	Token	Time	Token	Time	Token	Time	Token	Time
DeepSeek-V4-Pro	1.3M	8m	2.5M	14m	3.8M	19m	4.4M	19m	0.4M	2m	1.8M	10m	2.7M	13m	5.4M	20m
GLM-5.3	2.0M	11m	12.7M	27m	11.3M	27m	12.9M	31m	4.3M	15m	10.9M	25m	11.3M	27m	10.6M	27m
Kimi-K3	1.1M	14m	4.8M	44m	6.6M	47m	6.8M	38m	2.3M	18m	4.5M	25m	9.5M	32m	9.6M	35m

Appendix

Table 8: Cost efficiency of evaluated agents across games and intervention levels. We report the median processed input tokens and agent runtime, where M denotes millions of tokens and m denotes minutes. The lowest value in each column is bolded.

Figure 9: Failure modes in agent-produced world edits. The outer ring shows broad categories and the inner ring shows specific failure types; labels report counts and percentages of categorized failures.

Model	Build	Load	Behavior
DeepSeek-V4-Pro	2	13	60
GLM-5.3	11	3	44
Kimi-K3	5	1	39
Gemini 3.5 Flash	4	2	42
Claude Opus 4.8	0	2	27
GPT-5.6 Luna	0	4	35

Appendix

Table 9: Observed failure stage by model across audited unsuccessful runs. Counts indicate whether failures occur during build, load, or behavioral validation.

Observed domain	Count
Interaction / progression	101
Mechanic semantics	84
Resources / registration	53
API / loader	33
Incomplete delivery	18
Evaluation attribution	18

Appendix

Table 10: Observed failure domains across audited unsuccessful runs. These categories describe where failures are observed and do not always identify the underlying root cause.

	GPT-5.6 Sol	GPT-5.6 Luna	Gemini 3.5 Flash	Claude Opus 4.8
Read game sources	97%	99%	52%	91%
Read vanilla data	56%	52%	24%	37%
Searched the Web	9%	9%	72%	3%
Used provided skills	100%	96%	93%	58%
Rebuilt after initial build	85%	90%	83%	76%
Ran game server	15%	19%	0%	6%

Appendix

Table 11: Agent development behavior across all IGMBench runs. Values are the percentage of runs exhibiting each behavior. Trajectory-level behaviors are reported only for harnesses whose execution traces are observable.

	L1		L2		L3		L4		Overall
Configuration	CPR	WSR	CPR	WSR	CPR	WSR	CPR	WSR	CPR	WSR
Luna (w/ skills)	96.3	92.6	90.2	58.6	90.4	44.4	80.0	63.0	88.5	64.5
Luna (w/o skills)	97.8	92.6	89.8	44.8	93.3	59.3	78.5	33.3	89.2	57.3

Appendix

Table 12: Effect of host-provided skills on GPT-5.6 Luna. We report Criterion Pass Rate (CPR) and World-Editing Success Rate (WSR) across intervention levels, pooled over both Minecraft and Terraria. All values are percentages.

Game	Configuration	$\mathrm{Pass}_{\mathrm{style}}$	$\mathrm{Pass}_{\mathrm{sem}}$	$\mathrm{Pass}_{\mathrm{both}}$
Minecraft	Luna (w/ skills)	0.16	0.20	0.14
	Luna (w/o skills)	0.22	0.36	0.18
Terraria	Luna (w/ skills)	0.18	0.36	0.12
	Luna (w/o skills)	0.21	0.38	0.19

Appendix

Table 13: Effect of host-provided skills on visual asset quality for GPT-5.6 Luna. We report art-style, semantic, and joint visual pass rates for Minecraft and Terraria.

	CSD	TPIPS art style
Majority vote	0.54	0.66

Appendix

Table 14: Agreement with majority human judgments of whether agent-produced textures match the game’s art style, measured by threshold-free AUC. Nine raters with prior gaming experience evaluated 40 blind-sampled textures; unsure responses were excluded before majority voting.

Open-weight
Model	Harness	Effort	Image Tool	Released
DeepSeek-V4-Pro ( DeepSeek-AI et al., 2026 )	Hermes Agent	high	Nano Banana	2026-08
GLM-5.3 ( GLM-5-Team et al., 2026 )	Hermes Agent	high	Nano Banana	2026-08
Kimi-K3 ( Team et al., 2026 )	Hermes Agent	high	Nano Banana	2026-07
Closed-source
Gemini 3.5 Flash ( Google DeepMind, 2026a )	Gemini-CLI	high	Nano Banana	2026-05

Appendix

Table 15: Agent configurations evaluated on IGMBench . Each configuration specifies a backbone model, coding harness, inference effort, and image-generation backend. All models support approximately 1M-token context windows.

	Minecraft	Terraria
Game / server	Minecraft 1.21.1	tModLoader v2026.05.3.0
Mod loader	Fabric loader 0.19.3	tModLoader v2026.05.3.0
Build tool	Gradle 8.12	tML-build
Runtime	Java 21	bundled .NET 8.0
Mod API	Fabric API 0.116.14	tModLoader API
Probe channel	RCON+mineflayer	tModLoader ModSystems

Appendix

Table 16: The two game docker environments used at IGMWorld .

Game	Phase	L1	L2	L3	L4	All levels
	Fresh-world load	46.2	44.4	45.2	44.2	44.8
Minecraft	Behavior checks	236.1	582.8	421.1	1,738.1	632.9
	Combined	269.6	615.9	463.8	1,770.2	670.2
	Fresh-world load	35.3	35.1	34.5	35.7	35.2
Terraria	Behavior checks	80.9	79.0	68.0	68.0	78.3
	Combined	116.0	110.5	102.5	101.4	112.1

Appendix

Table 17: Evaluation runtime per task, reported as the 95th percentile in seconds. Measurements were collected on a machine with two AMD EPYC 7302 processors (32 cores total) and 503 GiB of RAM, excluding build and orchestration overhead. Using the overall combined p95 of 508.1 s gives a conservative sequential budget of 15.52 hours for all 110 tasks. In practice, evaluating the full benchmark with five parallel workers takes approximately 6 hours.

Figure 10: Interface used for the preliminary human perceptual study. Raters judged whether agent-produced assets were stylistically compatible with the shown vanilla references and semantically plausible for the requested object category.

	Art Style vs. style votes			Semantic vs. object votes
$\alpha$	Acc.	Rej.	Bal.	Acc.	Rej.	Bal.	Mean Bal. $\uparrow$	Vanilla
0.01	0.92	0.09	0.51	0.90	0.22	0.56	0.53	0.98
0.02	0.88	0.27	0.58	0.80	0.33	0.57	0.57	0.97
0.05	0.73	0.45	0.59	0.63	0.33	0.48	0.54	0.92
0.10	0.58	0.64	0.61	0.53	0.56	0.54	0.58	0.85
0.15	0.38	0.82	0.60	0.47	0.78	0.62	0.61	0.78

Appendix

Table 18: Calibration of the pass threshold

\alpha

using the preliminary human study. Nine raters judged 40 agent-produced textures blind to metric scores. Acc. is the fraction of majority-accepted textures passed by the metric, Rej. the fraction of majority-rejected textures rejected by the metric, and Bal. their mean. Vanilla is the fraction of original game assets passing both checks. The shaded row is the setting used in the main experiments.

Metric	$k$	Pass	Rank corr.	Same verdict	Model order
Art Style	1	0.32	0.948	0.91	0.95
	3	0.32	0.993	0.97	1.00
	5	0.33	1.000	1.00	1.00
	10	0.37	0.989	0.94	0.98
	20	0.42	0.969	0.88	1.00
Semantic	1	0.37	0.943	0.88	0.93

Appendix

Table 19: Sensitivity to the neighbour count

k

\alpha=0.15

over 1,760 scored agent textures in sprite-like categories. Thresholds are recalibrated by leave-one-out for each

k

. Pass is the agent pass rate, Rank corr. is the Spearman correlation of texture rankings with

k=5

, Same verdict is the fraction retaining the same pass/fail decision, and Model order is the Spearman correlation of model-level pass rates with those at

k=5

. The shaded rows indicate the setting used in the main experiments.

Panel of Figure 11 — Figure 11: Example prompt provided to the agent for an example world-editing task. The prompt specifies the requested modification, evaluation-facing behavioral requirements, project location, build command, and generic execution rules.

Panel of Figure 12 — Figure 12: Sample task specification from IGMBench . The task specification records benchmark metadata, the intervention level, the natural-language request, and the evaluation criteria describing the desired world edit.

Panel of Figure 13 — Figure 13: Sample evaluation specification from IGMBench . The verification view defines the controlled game setup and executable checks used to determine whether the requested world edit is correctly realized.

Figure 14: Task genre distribution across all 110 tasks in IGMBench . Percentages indicate the share of tasks in each genre.

Panel of Figure 15 — Figure 15: Task genre distributions in IGMBench by game. (a) Minecraft (57 tasks); (b) Terraria (53 tasks). Percentages are computed within each game.

Interactive World Models

Interactive world models predict and generate how environments change in response to actions, supporting simulation, planning, and control.

See all 122 papers

Precise Editing and Flexible Referencing for Interactable Worlds
Sep 28, 2026Xinyao Liao, Xianfang Zeng, Zhu Liang +5Instruction-Based Video Editing Autoregressive Video Generation
Nanyang Technological University · StepFun
HappyWorld-Bench
Sep 21, 2026Zhiqi Bai, Junai Cai, Yixin Chen +33World Models World Model Evaluation
Alibaba Token Hub, Alibaba Group · State Key Laboratory of General Artificial Intelligence, BIGAI · Tsinghua University +2
MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Engines
Mar 3, 2026Ryan Po, David Junhao Zhang, Amir Hertz +3Interactive World Models Video World Models
Stanford University · Google

Sep 28, 2026J/K move · Enter open · S save

Precise Editing and Flexible Referencing for Interactable Worlds

Xinyao Liao, Xianfang Zeng, Zhu Liang, Zhoujie Fu+4

Nanyang Technological University · StepFun

Instruction-Based Video Editing Autoregressive Video Generation Interactive World Models Video World Models

Open paper AskPDF arXiv

We present EditWorld, a video world model for precise editing and flexible referencing in interactable worlds. Existing video world models primarily focus on navigation, letting users explore generated worlds but offering limited control over how existing world content is modified. EditWorld extends world modeling from exploration to precise modification by streaming editing instructions and reference images during autoregressive generation. To support these capabilities, EditWorld introduces Gated Causal Attention for temporally varying editing conditions and reference images, together with a Sparse Context mechanism that maintains a bounded historical context for long-horizon inference. We further adopt joint autoregressive and bidirectional training with annealed self-resampling, and construct a dedicated data synthesis and annotation pipeline that provides supervision for world editing. We also present WBench-Editing to systematically evaluate streaming world editing capabilities. EditWorld achieves the best overall performance on WBench-Editing with an overall score of 73.8 and an editing score of 80.0, substantially outperforming existing methods on editing-related metrics. https://github.com/leoisufa/EditWorld

Abstract

Interactive World Models

Explore similar work

Precise Editing and Flexible Referencing for Interactable Worlds

HappyWorld-Bench

MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Engines