SceneAgent: A Multi-Agent Framework for Portable Game Scene Generation
Organizations: Tencent · Fudan University
Abstract
Moving a scene generator to a new game engine requires more than exporting its final output: planning, construction tools, and stored experience may also depend on the original backend. We present SceneAgent, a multi-agent framework for reusing the construction process and its experience across backends. A canonical SceneContext maintains the evolving world, while backend adapters isolate coordinate conventions, asset bindings, and export. Structured handoffs let fresh Builder sessions query this persistent state without inheriting the full construction transcript. Across generations, human-approved traces are distilled into asset-ID-free Layout skills and semantic Content skills that can be retrieved and recombined. We introduce GameSceneBench, comprising 69 indoor and outdoor prompts. Under a common asset and evaluation protocol, SceneAgent obtains human tie-adjusted preference scores of 78.2-89.4% against five baselines. Ablations show that reusable skills and the multi-agent protocol primarily improve prompt alignment and scene completeness, with additional runtime. Deployment in a second engine preserves the shared construction protocol and skill library while replacing backend integration components. Together, these results support separating persistent scene state, task-local reasoning, and reusable experience for portable scene construction.
Figures & tables
| Component | Responsibility |
|---|---|
| Shared across backends | |
| Agent protocol | Task decomposition and handoffs. |
| SceneContext | Canonical state and spatial queries. |
| Reusable skills | Layout and Content guidance. |
| Backend-specific integration | |
| Asset configuration | Asset catalogs and semantic mappings. |
| Metric | Single agent | Multi-agent |
|---|---|---|
| CLIPScore | ||
| Runtime (s) | ||
| VLM-Align. | ||
| VLM-Plaus. | ||
| VLM-Comp. |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Subset | Scene families |
|---|---|
| Outdoor (56) | Natural landscapes, settlements, sports and recreational spaces, historical and stylized environments, and urban infrastructure. |
| Indoor (13) | Residential, workplace, commercial, hospitality, entertainment, and utility spaces. |
| Tool family | Tools |
|---|---|
| Asset discovery and grounding | search_modules , get_module_info , web_search , view_thumbnails , plan_assets , get_planned_assets |
| Direct and relation-aware placement | place_module , place_facing , place_against_wall , place_on_ceiling |
| Parametric geometry and structures | create_floor , build_room , create_line , create_rect , create_circle , create_arc , build_column , build_board , build_stairs , create_shape |
| Procedural layout and population | region_define , get_landscape_data , surface_sampler , transform_points , remove_points_by_region , batch_place , pcg_scatter |
| Scene-state queries and handoff | get_scene_summary , update_conventions , report_subtask |
| Targeted editing and rollback | undo_last , clear_scene , remove_by_type , remove_by_region , undo_subtask |
| Agent | Control tools |
|---|---|
| Orchestrator | load_skill , plan_scene , revise_plan , build_scene , refine_scene , export_scene |
| Planner | search_modules , get_module_info , web_search , view_thumbnails , estimate_area_budget , estimate_room_size , commit_subtasks |
| Builder | update_conventions , report_subtask , plus the construction tools in Table 6 |
3D Scene Generation
3D scene generation creates structured, explorable environments from text, images, maps, or other inputs.
See all 198 papers