cs.CVOct 16, 2026
SaveScene-Designer: Learning Design Intelligence for Agentic 3D Scene Generation
Organizations: Zhejiang University · Universal Identification Technology (Hangzhou) Co.,Ltd.
Abstract
Generating complete, coherent, and usable 3D indoor scenes from natural language requires strong design intelligence: the ability to design high-quality scenes and critique their quality. However, existing approaches either rely on general-purpose models lacking domain-specific design experience or learn from static scenes. In this work, we introduce Scene-Designer, an agentic framework that learns design intelligence through complementary design and critique experiences. We construct two datasets: Scene-OTA-35K, which provides Observation--Thought--Action supervision for learning scene design and efficient error correction, and Scene-OTC-15K, which provides Observation--Thought--Critic supervision for learning scene quality critique across physical validity, functional correctness, and layout coherence. Based on these experiences, we propose Scene-GRPO, a Critic-guided GRPO framework that improves design decisions and reduces unnecessary trial-and-error corrections. Extensive experiments demonstrate that Scene-Designer achieves superior performance in physical validity, functional correctness, and layout coherence, while the learned Scene-Critic significantly outperforms general-purpose models in scene critique. Moreover, test-time Critic guidance further improves scene generation, showing that the Critic can provide useful guidance beyond training. Additional analysis suggests a trend of mutual enhancement between scene design and critique abilities.
Figures & tables
Figure 1: Overview of Design Intelligence. We define design intelligence as the ability to both design a scene and critique its quality. It consists of two complementary capabilities: scene design , which constructs physically valid, functionally correct, and layout-coherent scenes while making effective corrections, and scene critique , which assesses scene quality along the same dimensions and identifies better designs.
Figure 2: Analysis of construction errors and correction behavior in 100 scenes generated by GPT-5.2. (a) Left: Distribution of construction errors across physical errors and semantic and layout errors . (b) Right: Distribution of corrective steps for each error category, showing that even straightforward errors often involve unnecessary trial and error before being resolved.
Figure 3: Analysis of scene critique ability. (a) Left: Ours vs. GPT-5.2 on physical validity , functional correctness , and layout coherence : two-scene ranking accuracy ( ) and adjacent-pair ordering accuracy within four-scene rankings ( ). (b) Right: A representative failure case of GPT-5.2 in scene critique.
Figure 4: Overview of Scene-Designer. The framework learns Design Intelligence from two complementary experiences: Scene-OTA Construction extracts effective design experience from construction trajectories, while Scene-OTC Construction generates scene preference comparisons for learning critique. Together, they enable Scene-Designer to learn both scene design and critique abilities. Consolidation Statistics & Performance presents (a) the scene categories, asset counts, and asset classes in Scene-OTA-35K; (b) trajectory statistics before and after consolidation; (c) overall scene quality comparisons across methods, with scores averaged over Real., Func., Lay., and Comp.
Figure 5: Overview of Scene-GRPO. At each editing step, the policy expands multiple candidate actions from the current scene and selects the best branch to continue the trajectory. A frozen Critic provides local rewards through branch ranking and parent–child comparison, while global rewards encourage efficient and complete scene design.
| Method | Bedroom | Living Room | ||||||||||||
| Physics | Visual & Semantics | Physics | Visual & Semantics | |||||||||||
| #Obj | #OB | #CN | Real. | Func. | Lay. | Comp. | #Obj | #OB | #CN | Real. | Func. | Lay. | Comp. | |
| ATISS | 3.9 | 0.5 | 0.6 | 7.4 | 7.1 | 6.6 | 4.2 | 7.8 | 0.1 | 0.7 | 5.8 | 5.3 | 6.4 | 3.7 |
| DiffuScene | 3.5 | 0.1 | 1.1 | 6.5 | 7.0 | 6.7 | 3.6 | 6.9 | 0.5 | 1.2 | 5.5 | 4.9 | 5.2 | 3.5 |
| PhyScene | 3.3 | 0.1 | 0.3 | 5.7 | 6.3 | 5.7 | 4.0 | 8.0 | 0.0 | 0.7 | 5.2 | 5.3 | 5.1 | 3.3 |
| LayoutGPT | 5.4 | 1.0 | 1.3 | 7.5 | 8.1 | 6.7 | 4.2 | 8.4 | 1.1 | 2.8 | 6.4 | 5.8 | 5.2 | 3.6 |
Table 1: Quantitative comparison on common room types. Bold and underlined numbers indicate the best and second-best performance, respectively. Shaded rows highlight our models, with green cells indicating improvements from test-time Critic guidance. Green and red superscripts denote performance improvements and degradations relative to the preceding model variant, respectively.
| Method | Bathroom | Office | ||||||||||||
| Physics | Visual & Semantics | Physics | Visual & Semantics | |||||||||||
| #Obj | #OB | #CN | Real. | Func. | Lay. | Comp. | #Obj | #OB | #CN | Real. | Func. | Lay. | Comp. | |
| LayoutGPT | 7.7 | 1.3 | 1.0 | 8.3 | 9.3 | 7.7 | 6.0 | 7.3 | 0.3 | 0.0 | 6.7 | 7.7 | 6.3 | 4.0 |
| Holodeck | 12.0 | 0.0 | 1.7 | 7.7 | 6.7 | 7.0 | 5.3 | 27.0 | 0.0 | 4.7 | 7.0 | 6.3 | 4.3 | 4.0 |
| I-Design | 9.7 | 0.0 | 0.0 | 7.4 | 7.2 | 7.4 | 5.4 | 11.7 | 0.0 | 0.0 | 8.0 | 9.0 | 6.8 | 5.4 |
| SceneWeaver | 19.7 | 0.0 | 0.0 | 9.0 | 10.0 | 8.0 | 9.0 | 40.0 | 0.0 | 0.0 | 9.0 | 10.0 | 8.0 | 8.7 |
Table 2: Quantitative comparison on other scene types. Bold and underlined numbers indicate the best and second-best performance, respectively. Shaded rows highlight our models, with green cells indicating improvements from test-time Critic guidance. Green and red superscripts denote performance improvements and degradations relative to the preceding model variant, respectively.
Figure 6: Comparison of error correction efficiency across different error categories. Compared with (a) Scene-Builder, (b) Scene-Designer generally requires fewer corrective moves, reducing unnecessary trial and error.
Figure 10: Evaluation of the Scene-OTC test sets. (a) compares overall preference accuracy. (b) analyzes ranking consistency with Pos- and Top- metrics.
Figure 11: Image-order robustness of the learned Critic. We independently shuffle the input image-label order five times. Left: accuracy on and across five randomized runs. Right: Pos- and Top- accuracy on .
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 20: A representative asset-level editing trajectory of a GPT-5.2-generated scene. (a) The editing trajectory of each major asset over the generation process, where different colors indicate different editing operations, including initial placement and subsequent reworks. (b) The final generated scene corresponding to the trajectory in (a), with major assets highlighted.
| Severity | Degradation Type | Perturbation Functions |
|---|---|---|
| Tier 1 | Physical violation | move_blocking_door, move_into_wall, move_overlap, rotate_out_of_room, rotate_into_collision |
| Tier 2 | Functional degradation | face_wall, rotate_180, remove_object, swap_two |
| Tier 3 | Layout deviation | move_off_wall, move_far_from_focal, move_to_corner, rotate_45_offset |
Table 4: Overview of perturbation categories in Scene-OTC.
| Severity | Perturbation | Description |
| Tier 1 | move_into_wall | Move an object across the room boundary, causing partial or complete out-of-bound placement. |
| move_blocking_door | Move an object into the door clearance region, reducing accessibility and blocking movement paths. | |
| move_overlap | Move an object to overlap with another furniture item, creating physical collision. | |
| rotate_out_of_room | Rotate an object so that its geometry exceeds the room boundary. | |
| rotate_into_collision | Rotate an object to introduce collisions with surrounding objects. | |
| Tier 2 | face_wall | Rotate an oriented object toward the wall, violating its intended functional orientation. |
Table 5: Detailed perturbation strategies used in Scene-OTC.
Figure 24: Representative perturbation strategies in Scene-OTC. (a) Accumulating perturbations progressively degrade scene quality by introducing additional defects. (b) Different-strength perturbations apply the same modification with varying severity. (c) The same perturbation applied to different objects reveals the influence of object functional importance.
Figure 28: The evaluated instruction is: “Create a spa-like bathroom Create a spa-like bathroom with a separate shower, a sleek vanity with a large mirror, and calm, minimal finishes.” Qualitative comparison across SFT, RL, and test-time Critic guidance. (a,d), (b,e), and (c,f) correspond to Scene-Builder, Scene-Designer, and Scene-Designer with Critic, respectively.
Figure 29: Test-time Critic guidance during scene construction. The Scene-Designer samples multiple candidate branches at each step, and the Scene-Critic ranks intermediate layouts based on physical validity, functional correctness, and layout coherence. The selected branches progressively refine the scene toward the final design.
Figure 30: A representative failure case of GPT-5.2 as a test-time Critic during scene construction. GPT-5.2 often provides plausible rationales but fails to consistently evaluate physical validity, functional correctness, and layout coherence when selecting among candidate branches.
Figure 31: Qualitative comparison between Scene-Designer and existing scene generation methods. Existing methods often generate incomplete scenes or suffer from physical violations, poor functional organization, and incoherent layouts. Scene-Builder produces physically valid scenes, while Scene-Designer further improves functional correctness and layout coherence through learned design strategies. Adding the Critic during inference provides additional guidance and further enhances overall scene quality.
Figure 34: Prompt template for correction-effect evaluation. The verifier receives structured trajectory evidence and outputs operation-level causal judgments.
| Method | [email protected] | [email protected] | [email protected] | mRR | [email protected] | [email protected] | DAR | WAR |
|---|---|---|---|---|---|---|---|---|
| Scene-Builder | 0.92 | 0.98 | 0.89 | 0.96 | 0.72 | 0.53 | 1.0 | 0.96 |
| Scene-Designer | 0.97 | 1.0 | 0.93 | 0.96 | 0.83 | 0.53 | 1.0 | 0.96 |
| Scene-Designer w/ Critic | 1.0 | 1.0 | 0.94 | 0.98 | 1.0 | 0.54 | 1.0 | 1.0 |
Table 6: Quantitative evaluation of scene accessibility and spatial organization. Higher values are better for RR, mRR, FR, DAR, and WAR. NAR measures the proportion of navigable area and is not a strictly monotonic quality metric.
More in this topic
3D Scene Generation
3D scene generation creates structured, explorable environments from text, images, maps, or other inputs.
See all 189 papers