Moving a scene generator to a new game engine requires more than exporting its final output: planning, construction tools, and stored experience may also depend on the original backend. We present SceneAgent, a multi-agent framework for reusing the construction process and its experience across backends. A canonical SceneContext maintains the evolving world, while backend adapters isolate coordinate conventions, asset bindings, and export. Structured handoffs let fresh Builder sessions query this persistent state without inheriting the full construction transcript. Across generations, human-approved traces are distilled into asset-ID-free Layout skills and semantic Content skills that can be retrieved and recombined. We introduce GameSceneBench, comprising 69 indoor and outdoor prompts. Under a common asset and evaluation protocol, SceneAgent obtains human tie-adjusted preference scores of 78.2-89.4% against five baselines. Ablations show that reusable skills and the multi-agent protocol primarily improve prompt alignment and scene completeness, with additional runtime. Deployment in a second engine preserves the shared construction protocol and skill library while replacing backend integration components. Together, these results support separating persistent scene state, task-local reasoning, and reusable experience for portable scene construction.
Figures & tables
Figure 1: SceneAgent separates within-run construction from cross-run experience reuse. Fresh Builders coordinate through canonical scene state. Backend adapters realize that state in the target engine. The optional human-gated update path retains approved traces as composable Layout and Content skills.
Component
Responsibility
Shared across backends
Agent protocol
Task decomposition and handoffs.
SceneContext
Canonical state and spatial queries.
Reusable skills
Layout and Content guidance.
Backend-specific integration
Asset configuration
Asset catalogs and semantic mappings.
Table 1: Shared and backend-local components in SceneAgent .
Figure 2: Qualitative comparison on representative GameSceneBench prompts. Rows share an input prompt, and columns show different methods under the common Engine A asset and rendering protocol.
Figure 3: Representative generations in Engine B.
Figure 4: Cross-LLM comparison on the night-market prompt.
Figure 5: Composition of Layout and Content skills. Rows fix the Layout skill, while columns fix the Content skill.
Figure 6: Visual experience acquisition and reuse. A reference image guides scene construction, and the human-approved trace is factorized into Layout and Content skills for reuse.
Metric
Single agent
Multi-agent
CLIPScore ↑
0.210±0.032
0.216±0.034
Runtime (s) ↓
39.3±24.2
203.2±97.8
VLM-Align. ↑
42.5
57.5
VLM-Plaus. ↑
49.0
51.0
VLM-Comp. ↑
37.7
62.4
Table 4: Single- versus multi-agent construction on GameSceneBench. Results report mean ± standard deviation and tie-adjusted VLM preferences.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Subset
Scene families
Outdoor (56)
Natural landscapes, settlements, sports and recreational spaces, historical and stylized environments, and urban infrastructure.
Indoor (13)
Residential, workplace, commercial, hospitality, entertainment, and utility spaces.
Table 6: Builder-visible construction tools. The interface is shared across backends. Backend-specific configuration supplies compatible asset categories and export conventions.
update_conventions , report_subtask , plus the construction tools in Table 6
Appendix
Table 7: Role-specific control interfaces. Area-budget estimation is enabled for area-aware planning, and room-size estimation is enabled for indoor planning.
Figure 7: Instructions presented to participants before the human pairwise evaluation. The page describes the blind protocol, response options, evaluation dimensions, and minimum completion requirement.
Figure 8: Human pairwise-evaluation interface. Each trial presents the scene prompt and an anonymous, randomly ordered 2×5 composite containing five views per method. Participants select the better row or a tie, may record optional notes, and navigate through the assigned trials.
Figure 9: Additional qualitative results in Engine A (set 1). From top to bottom: Amusement Park, Ancient Siege, Ancient Ruins, Basketball Court, and Beach Resort. Each row shows four views of the same generated scene.
Figure 10: Additional qualitative results in Engine A (set 2). From top to bottom: Chinese Garden, Chinese Temple, City Street, Desert Camp, and Egypt Adventure. Each row shows four views of the same generated scene.
Figure 11: Additional qualitative results in Engine A (set 3). From top to bottom: Japanese Onsen, Ice Wonderland, European Garden, Forest Town, and Graveyard. Each row shows four views of the same generated scene.
Figure 12: Additional qualitative results in Engine A (set 4). From top to bottom: Pond Stream, Pine Forest, Military Base, Military March, and Night Market. Each row shows four views of the same generated scene.
Figure 13: Additional qualitative results in Engine A (set 5). From top to bottom: Tomb Raider, Tang Dynasty, Primitive Tribe, Sakura Grove, and Parking Lot. Each row shows four views of the same generated scene.
Figure 14: Additional qualitative results in Engine A (set 6). From top to bottom: Winter Town, Western Town, Snow Camp, Subway Station, and Villa Town. Each row shows four views of the same generated scene.
Figure 15: Additional qualitative results in Engine A (set 7). From top to bottom: Venice, Hotel Lobby, Laundry Room, Bedroom, and Kitchen. Each row shows four views of the same generated scene.
Figure 16: Additional qualitative results in Engine A (set 8). From top to bottom: Fruit Store, Meeting Room, Nursery Room, and Dining Room. Each row shows four views of the same generated scene.
Figure 17: Additional qualitative results in Engine B (set 1). From top to bottom: Ancient Ruins, Bamboo Forest, Creek Canyon, European Garden, and Ice Wonderland. Each row shows four views of the same generated scene.
Figure 18: Additional qualitative results in Engine B (set 2). From top to bottom: Creek Canyon, Tang Dynasty, Villa Town, Japanese Onsen, and Winter Town. Each row shows four views of the same generated scene.