cs.ROAug 19, 2026
SaveBeyond Placement and Articulation: Usage-Driven Code Scenes for Embodied Interaction
Organizations: Shanghai Jiao Tong University · Meituan
Abstract
Indoor scene synthesis provides essential environments for embodied AI, robotic manipulation, and simulation-based policy learning. Recent code-based scene generation methods produce editable and extensible environments, yet they remain focused on visual construction and object-level articulation, leaving the functional usage of scenes largely unmodeled. To address this problem, we present RoomWright, an agentic usage-driven framework for generating 3D scenes represented entirely as code for embodied interaction. RoomWright performs usage-driven object reasoning, which treats each anchor as a task centre and admits task-required objects and their affordances. A code agent further enables multi-part interaction by compiling each interaction into a trigger, condition, effect rule that updates structured object states, capturing causal dependencies across objects. Moreover, since manipuland orientation is ambiguous and hard to recover from pixels, RoomWright alleviates this via annotation-informed usage-guided orientation. Extensive experiments demonstrate the effectiveness of our method. The resulting scenes are executable, editable, and simulation-ready, providing interactive environments for embodied AI and policy learning.
Figures & tables
Figure 1: The scenes automatically generated by RoomWright. Each object in the scene is independently separable and possesses its own articulation structure, while different parts of an object, and even multiple objects, exhibit rich coordinated interactions. The objects, the scene layout, and all interaction behaviors are represented entirely as code.
Figure 2: Pipeline overview of RoomWright. Starting from a natural-language room description, RoomWright build an interactive 3D scene through three stages: (1) scene planning infers a usage-driven symbolic scene state with object, support, and interaction relations; (2) simulation-ready asset generation produces simulation-ready articulated assets with semantic parts, physical properties, and executable interactions; (3) layout generation optimizes object placement through geometric reasoning, visual critique, and physical simulation. The generated scene is directly executable in embodied environments.
Figure 3: Qualitative comparison of the four methods. RoomWright produces layouts with better plausibility than the baselines, and achieves highly realistic visual quality even though it relies entirely on code, without any 3D generation or asset retrieval.
| Method | Collision-free | Navig. | In-bound | Opening-clear | Support | Access. | ObjCount | ObjAttr | OO-Rel | OA-Rel |
|---|---|---|---|---|---|---|---|---|---|---|
| SceneCode | 91.2 | 99.0 | 81.2 | 96.4 | 26.9 | 26.1 | 38.1 | 30.7 | 16.1 | 55.6 |
| Code-as-Room | 45.5 | 89.5 | 73.3 | 47.2 | 51.9 | 63.1 | 37.0 | 56.7 | 20.5 | 22.2 |
| SceneSmith | 95.0 | 99.6 | 99.7 | 98.0 | 58.4 | 39.8 | 48.8 | 42.5 | 27.0 | 42.2 |
| Ours | 95.9 | 97.3 | 99.7 | 95.7 | 53.6 | 65.6 | 57.0 | 64.6 | 22.4 | 40.6 |
Table 1: SceneEval comparison across four scene-generation methods (mean over 28 scenes). Higher is better ( ) for all metrics. The best result in each column is shown in bold , and the second-best result is underlined .
| Method | IO | AO | CHI | MPC |
|---|---|---|---|---|
| Code-as-Room | – | – | – | – |
| SceneSmith | 2.4 | 2.4 | – | – |
| SceneCode | 4.4 | 4.4 | – | – |
| Ours | 16.4 | 13.2 | 6.4 | 3.6 |
Table 2: Interactive objects and behaviors in generated scenes. ’–’ indicates that the method does not provide the corresponding attribute.
Figure 4: Simulation of generated interactions. (a)(b) The gripper grasps the handle and closes the revolute door. (c) The arm approaches the button while the display remains off. (d) Pressing the button lights up the display, demonstrating a causal interaction.
Figure 5: Qualitative comparison with and without usage-driven object reasoning (UR in the figure).
Figure 6: Comparison of manipuland placement with and without usage-guided orientation (UO in the figure).
| ID | Room type | Prompt (verbatim) |
|---|---|---|
| laundry_room | laundry room | A laundry room with a washing machine and a dryer. |
| bathroom1 | bathroom | A bathroom with a vanity, a bathtub, and a toilet. |
| office1 | home office | A home office with a desk, an office chair, and a bookshelf. |
| gym1 | home gym | A home gym with a treadmill and an exercise bike, and a wall mirror. |
| library | home library | A home library with a bookshelf, a reading armchair, and a floor lamp. |
| art_studio | art studio | An art studio with an easel, a stool, and a worktable. |
Table 3: The 16 easy-tier evaluation prompts (verbatim).
| ID | Room type | Prompt (verbatim) |
|---|---|---|
| kitchen | kitchen | A kitchen with a counter along the main wall holding a microwave and a knife block, a refrigerator at one end of the counter, and a built-in oven with a stovetop, with an island in the middle with two bar stools tucked underneath. |
| bedroom1 | bedroom | A bedroom with a queen bed against the main wall, a nightstand with a table lamp on each side of the bed, a tall wardrobe with sliding doors, a low dresser with a wide mirror above it, and a cushioned bench at the foot of the bed. |
| living_room | living room | A living room with a three-seat sofa facing a wall-mounted TV, a coffee table with a stack of magazines on a wide area rug, a media console beneath the TV flanked by two tall floor speakers, and a bookshelf along the side wall. |
| dining_room1 | dining room | A dining room with a long rectangular table surrounded by six chairs and set with tableware, a sideboard against the wall holding serving dishes, and a pendant chandelier above the table. |
| garage_workshop | garage workshop | A garage workshop with a heavy workbench along the main wall fitted with a bench vise and a toolbox, a tall pegboard above the bench holding hanging tools, a rolling tool chest with drawers, and a metal shelving unit against the side wall. |
| playroom | children’s playroom | A children’s playroom with a low activity table with four small chairs, a tall toy shelf filled with bins of toys, a cozy reading corner with a bean bag chair and a floor lamp, and a play tent near the window. |
Table 4: The 7 medium-tier evaluation prompts (verbatim).
| Family | Metrics | What it measures |
|---|---|---|
| Physical plausibility | COL, NAV, OOB, OPC | realistic, walkable, in-room layout |
| VLM-judged physical | SUP, ACC | objects properly supported & reachable |
| Prompt fidelity | CNT, ATR, OO-Rel, OA-Rel | scene matches the text prompt |
Table 5: The ten evaluation metrics and their families.
Figure 7: Method × effect distribution of the non-contact interaction edges across 23 scenes.
Figure 8: Effect-channel share over all inferred edges.
Figure 9: Most frequent induced relation verbs over the 194 edges.
Figure 10: Demo of Objects with Complex Interactions
| Layout parameters | Resulting asset |
|---|---|
| rows=2, cols=1 | nightstand |
| rows=2, cols=3 | wide dresser |
| double doors top + rows=2 bottom | cupboard with drawers |
| rows=1 top + door bottom | nightstand with cabinet |
| door column + rows=3 column | wardrobe with drawers |
| rows=3 column + door-over-drawer column | combo dresser |
Table 6: The eight cabinet variants shown in Figure 11 , all generated from the single template of Listing 4 . Each row is a parameter change.
Figure 11: Parametric Editing of Code-Represented 3D Assets
| Cost item | Tokens (M) |
|---|---|
| Total token count | 1,204.5 |
| Average token count | 43.0 |
| Maximum token count | 72.0 |
| Minimum token count | 27.2 |
Table 7: Token consumption statistics for the evaluation runs.
| Statistic | Wall-clock time | Seconds |
|---|---|---|
| Average time cost | 9:10:38 | 33,038.0 s |
| Maximum time cost | 12:21:11 | 44,471.0 s |
| Minimum time cost | 6:34:49 | 23,689.0 s |
Table 8: Wall-clock time cost statistics for the evaluation runs, reported both in hours:minutes:seconds and seconds.
Figure 12: Failure Cases in Generated Scenes.
Figure 13: A children’s playroom with a low activity table with four small chairs, a tall toy shelf filled with bins of toys, a cozy reading corner with a bean bag chair and a floor lamp, and a play tent near the window.
Figure 14: A sewing room with a worktable, a sewing machine, and a fabric shelf.
Figure 15: A home library with a bookshelf, a reading armchair, and a floor lamp.
Figure 16: A photography studio with a backdrop, two studio lights, and a camera on a tripod.
Figure 17: A music room with a piano and a piano bench, and a drum kit.
More in this topic
3D Scene Generation
3D scene generation creates structured, explorable environments from text, images, maps, or other inputs.
See all 189 papers