cs.GROct 20, 2026
SaveGALA: Geometry-Aware Language Model for Controllable Object Arrangement
Organizations: University of Science and Technology of China, Hefei, China · Tencent, Shenzhen, China · Tsinghua University, Shenzhen, China · Hong Kong University of Science and Technology, Hong Kong, China
Abstract
Reasoning about fine-grained placement constraints, such as support and containment, is essential for realistic 3D scene creation. It remains a significant challenge for existing methods that rely on simplified object representations like oriented bounding box (OBB), which are blind to the detailed geometry. To address this limitation, we introduce GALA, a geometry-aware language model with explicit point cloud perception for controllable object placement. Our model performs autoregressively by predicting the 6D pose of one target object at a time conditioned on the scene context and placement instruction. We utilize a two-stage training strategy: the first stage aligns geometric and textual modalities through pretraining tasks, and the second stage fine-tunes the model for placement. To facilitate this task, we also curate a multi-source scene dataset covering diverse room types and fine-grained furniture arrangements, which will be open-sourced for community benefit. Experiments on our benchmark show our model significantly outperforms baselines without explicit geometric reasoning. We further develop a multi-stage plan-place-verify agent pipeline that uses the model for text-driven scene generation. Project Page: https://xccelephant.github.io/paper/gala/
Figures & tables
Figure 1. Left: input objects and fine-grained placement instructions. Middle: plausible layouts that follow the instructions. Right: zoom-in views show how GALA goes beyond oriented bounding boxes by perceiving geometry to produce correct support, containment, and collision-free placements. Teaser figure showing input placement goals, instruction-conditioned layout results from a bounding-box baseline and GALA, and zoom-in views of GALA placements with correct support, containment, and collision-free geometry.
Figure 2. Overview of the geometry-aware placement model. Given the scene boundary, the already-placed objects (with poses, sizes, semantic categories, and point clouds), the new object (with size, semantic category, and point cloud), and a placement instruction, the multimodal LLM outputs a numeric 6D pose. The model performs placement autoregressively, adding each predicted object to the scene context before predicting the next one. Overview of the geometry-aware placement model.
Figure 3. Dataset construction pipeline. Multi-source scenes are first cleaned by physical plausibility checks and VLM-based quality assessment. Each retained scene is parsed into a scene graph containing absolute room-location edges ( abs ), dependency edges ( on , within , under ), and proximity edges ( next to ). We form a floor-rooted scene tree by selecting dependency edges with priority on within under abs . We obtain the placement order by traversing from the floor root, sorting nodes at the same depth by edge priority and footprint area. For each placement step, the tree edge and nearby proximity relations are verbalized into an instruction paired with per-object point clouds and the target 6D pose. Pipeline for dataset filtering and scene data structuring, showing multi-source datasets, physical and VLM-based filtering, scene graph construction with absolute, dependency, and proximity relations, floor-rooted scene tree extraction with relation priority, BFS placement order generation, and instruction-target sample construction.
Figure 4. Dataset statistics. (a) Distribution of object counts per scene across data sources. (b) Proportion of placement instructions generated from each source. (c) Distribution of instruction relation types, including absolute , on , within , and under . Dataset statistics showing objects per scene, instruction source proportions across Imaginarium, artist-crafted scenes, 3D-FRONT, and SpatialLM-PCG, and instruction relation distributions for absolute, on, within, and under relations.
Figure 5. Agent pipeline for automated scene generation, where a planner turns a user prompt into object-level steps, GALA places each object, and a verifier feeds back failures for correction. The agent pipeline converts a user prompt into object plans, generates placement instructions, uses GALA to predict poses, and verifies each updated scene before continuing.
Figure 6. Qualitative comparisons of instruction-conditioned single-object placement results across instruction-aware baselines and our method. Qualitative comparisons of instruction-conditioned single-object placement results across baselines and our method.
| Method | Input | Within. | Instruction. | Float. | Support. |
|---|---|---|---|---|---|
| ATISS | OBB+Cat. | – | – | 0.93% | 99.20% |
| ReSpace | OBB+Cat. | – | – | 5.79% | 94.20% |
| LayoutVLM | OBB+Instr. | 10.96% | 37.71% | 17.05% | 82.91% |
| GPT-OSS-120B | OBB+Instr. | 22.53% | 42.29% | 7.64% | 93.54% |
| Ours (0.7B) | PC+Instr. | 65.62% | 78.59% | 4.63% | 96.03% |
Table 1. Quantitative comparisons for single-object inference on the held-out test split. Cat., Instr., and PC denote category, instruction, and point cloud, respectively. Dashes indicate instruction-dependent metrics that are not applicable.
Figure 7. Qualitative ablation results on fine-grained placement. Each column shows one instruction, the predicted scene, and a zoomed crop around the target placement. Blue boxes mark inserted target objects. Qualitative ablation results on fine-grained placement.
| Table 2. User-study scores from 30 participants. Dashes denote unavailable single-object scores for methods without instruction input. Method Single-object placement Whole-scene result Physical. Instruction. Physical. Aesthetics. ATISS – – 1.9 1.9 ReSpace – – 2.7 2.9 LayoutVLM 1.9 2.0 3.0 3.2 GPT-OSS-120B 2.7 2.4 2.5 2.5 Ours (0.7B) 4.5 4.5 3.9 4.1 | Table 3. Controlled ablations on the held-out test split. Within / Instruction report valid rates on containment/all samples. Arrows show preferred directions; bold/underlined values mark the best/second-best results. Method Within. Instruction. Float. Support. w/o point cloud 62.40% 78.81% 5.19% 95.35% w/o pretraining 62.69% 78.37% 5.20% 95.42% w/o artist scenes 60.32% 76.69% 5.02% 95.88% Full method 65.62% 78.59% 4.63% 96.03% |
Table 2. User-study scores from 30 participants. Dashes denote unavailable single-object scores for methods without instruction input.
Figure 8. Qualitative comparison with baselines on whole-scene inference results. Each method receives the object set and the per-step placement instructions of a held-out test scene and places the objects one at a time, so the comparison isolates placement quality from planning. Qualitative comparison with baselines on whole-scene inference results, where every method places the objects of a held-out test scene one at a time from the same per-step placement instructions.
Figure 9. More autoregressive inference results of our model on held-out test scenes. Starting from scene-level object sets and placement instructions, our model sequentially predicts object poses and produces complete arrangements across different room types and layout complexity. Gallery of twelve autoregressive inference results produced by our model on held-out test scenes, including bedrooms, living rooms, a dental office, and a dining room with varying layout complexity.
Figure 10. Gallery results from the agent pipeline for scene generation. Given text prompts and polygon floor boundary, the planner decomposes the request into object-level placement steps, then GALA predicts poses for the target objects to produce complete 3D scene arrangements. Gallery of generated 3D scenes produced by the agent pipeline for scene generation from text prompts and polygon floor boundary.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 11. Procedural expansion of SpatialLM layouts. Starting from a 3D-box semantic layout, we replace large proxy objects with compatible assets from a curated library and then enrich the scene with small objects placed according to the geometry and affordances of the instantiated furniture. Pipeline for SpatialLM-PCG construction, showing an input 3D bounding-box semantic layout, replacement of large furniture assets, and enrichment with small objects on or near furniture.
Figure 12. Overview of the hand-crafted scene dataset used for geometry-aware object arrangement. The gallery illustrates the diversity of room types, object categories, and fine-grained support and containment configurations. Dataset gallery showing diverse indoor scenes with varied room types, furniture categories, and fine-grained object arrangements.
| Relation type | Relation | Geometric condition | Thresholds |
|---|---|---|---|
| Scene boundary | against east wall against west wall against north wall against south wall | The object is floor-supported ( ). The distance from to the corresponding boundary edge is below . Directional wall names are assigned from the edge outward normal. | |
| Scene boundary | at northeast corner at northwest corner at southeast corner at southwest corner | The object is floor-supported and its footprint is within the wall threshold of two adjacent convex boundary edges. The corner name is determined from the two outward normals, with a centroid-based quadrant fallback for irregular boundaries. | |
| Scene region | in front region in back region in left region in right region in center region | Used when no wall or corner relation is emitted. The object center is compared with the scene AABB center: front/back use the sign of the Y offset and left/right use the sign of the X offset. A directional region requires the corresponding offset magnitude to exceed times the scene depth or width; otherwise the object is in the center region. | |
| Horizontal proximity | next to | The footprint distance satisfies . Horizontal relations are suppressed when an on , under , or within relation already holds, and require compatible bottom heights: , where . | |
| Horizontal direction | left right in front behind | The footprint distance satisfies and the same height and vertical-relation exclusions as above hold. The relation is selected by : left for , right for , in front for , and behind for . | |
| Support | on | The target center projected to XY lies inside , and the target bottom is close to the reference top: . |
Table 4. Definitions of spatial relations used for instruction generation.
Figure 13. Validation loss curves for the text-only and point-cloud variants used in the internal comparison. Both variants reach their minimum validation loss at step 6240 (epoch 2.67), the checkpoint used for the reported test metrics; the markers annotate the corresponding validation losses. Line plot showing validation loss curves for the text-only and point-cloud variants, with the selected minimum-loss checkpoint marked at step 6240 and the minimum validation losses annotated.
| Method | Within. | Instruction. | Floating. | Support. |
|---|---|---|---|---|
| PlaceIt3D | 0.32% | 1.39% | 67.53% | 35.96% |
| DeepSeek-V4-Flash | 11.53% | 30.02% | 17.89% | 84.75% |
| Qwen3.5-397B-A17B | 17.35% | 37.28% | 12.87% | 89.11% |
| Ours (0.7B) | 65.62% | 78.59% | 4.63% | 96.03% |
Table 5. Additional single-object placement baselines on the full held-out test split.
| Method | Within. | Instruction. | Floating. | Support. |
|---|---|---|---|---|
| Nano-Banana + FoundationPose | 12.82% | 15.00% | 69.00% | 31.21% |
| Ours | 66.67% | 79.00% | 6.00% | 92.02% |
Table 6. Image-pipeline comparison on the same 100-case subset using four views per case and the best recovered pose. The first row abbreviates the Nano-Banana + SAM + FoundationPose pipeline.
| Test subset | Method | Within. | Instruction. | Float. | Support. |
|---|---|---|---|---|---|
| high-quality | Full method | 52.82 | 71.86 | 10.34 | 91.41 |
| w/o high-quality | 39.47 | 64.59 | 13.34 | 89.56 | |
| 3D-FRONT & SpatialLM-PCG | Full method | 68.65 | 80.06 | 3.38 | 97.05 |
| w/o high-quality | 65.44 | 79.57 | 3.17 | 97.25 |
Table 7. Stratified ablation of high-quality training scenes on two held-out test subsets. The high-quality subset contains Imaginarium and artist-crafted scenes; the other contains 3D-FRONT and SpatialLM-PCG scenes. All entries are percentages.
| Metric | w/ point cloud | w/o point cloud |
|---|---|---|
| Height error (m) | 0.0747 | 0.0868 |
| User-study score (1–5) | 4.14 | 1.91 |
Table 8. Point-cloud analysis on within samples and in a separate ablation user study.
| Group | Within. | Floating. | Support. |
|---|---|---|---|
| Seen container, w/ point cloud | 61.72% | 16.41% | 84.21% |
| Unseen container, w/ point cloud | 59.38% | 31.25% | 68.34% |
Table 9. Asset-level generalization on two size-matched sets of 128 within -relation samples.
| Component | Frequency | Runtime | Execution | Local memory/hardware |
|---|---|---|---|---|
| Scene planner | Once per scene | 100 s | API | – |
| Instruction generator | Once per object | 70 s | API | – |
| GALA placement | Once per object | 1 s | Local inference | 3 GB, 1 NVIDIA H20 |
Table 10. Runtime breakdown of the end-to-end agent pipeline. API timings depend on the remote service, whereas GALA placement is measured locally.
More in this topic
3D Scene Layout Generation
3D scene layout generation arranges objects and structures into coherent, controllable scenes from text, images, plans, or functional goals.
See all 80 papers