Inter-object relations underpin spatial intelligence, yet existing representations -- linguistic prepositions or object-level scene graphs -- are too coarse to specify which regions actually support, contain, or contact one another, leading to ambiguous and physically inconsistent layouts. To address these ambiguities, a part-level formulation is needed; therefore, we introduce PARSE, a framework that explicitly models how object parts interact to determine feasible and spatially grounded scene configurations. PARSE centers on the Part-centric Assembly Graph (PAG), which encodes geometric relations between specific object parts, and a Part-Aware Spatial Configuration Solver that converts these relations into geometric constraints to assemble collision-free, physically valid scenes. Using PARSE, we build PARSE-10K, a dataset of 10,000 3D indoor scenes constructed from real-image layout priors and a curated part-annotated shape database, each with dense contact structures and a part-level contact graph. With this structured, spatially grounded supervision, fine-tuning Qwen3-VL on PARSE-10K yields stronger object-level layout reasoning and more accurate part-level relation understanding; furthermore, leveraging PAGs as structural priors in 3D generation models leads to scenes with substantially improved physical realism and structural complexity. Together, these results show that PARSE significantly advances geometry-grounded spatial reasoning and supports the generation of physically consistent 3D scenes.
Figures & tables
Figure 1: Overview of the capabilities of PARSE. Using a real image (a) as reference, we first construct Part-centric Assembly Graphs (PAGs) (b) that capture its spatial organization of objects. Then, by combining PARSE with physics simulation, we generate physically plausible 3D scenes (c) from these PAGs, featuring diverse inter-object relationships and rich part-level contacts. Furthermore, we introduce PARSE-10K, a collection of high-quality 3D indoor scenes with fully part-segmented object instances, which effectively supports downstream tasks such as fine-tuning VLMs for spatial reasoning and enhancing 3D scene generation.
Figure 2: An illustrative example of our Part-centric Assembly Graph (PAG). Left: A 3D scene generated by PARSE, with specific local regions highlighted. Middle: The corresponding global PAG, emphasizing the sub-PAGs that match the highlighted regions on the left. In the graph, large labeled circles represent object nodes, while small dark circles attached to them represent part nodes (unrelated part nodes are omitted for clarity). Right: Zoomed-in views of three sub-PAGs. These panels explicitly annotate the specific surfaces used to define relational constraints, alongside the corresponding Object-Level Spatial Edges and Part-Level Geometric Edges.
Figure 3: Controllable Scene Synthesis via Part-Aware Spatial Configuration Solver. (a) Coarse Localization: The solver first prunes occupied regions (red) from the 2D support surface, then further contracts the feasible space using object-level spatial relations (orange). (b) Part-Level Alignment: Precise geometric alignment is achieved by enforcing constraints ( e.g . , coplanarity) between specific surfaces identified by the solver. This drastically reduces the feasible pose space for final pose sampling. (c) Fine-Grained Relational Control: Specifying different part-level geometric relations in the PAG results in distinct and predictable arrangements, showcasing the framework’s fine-grained controllability.
Dataset
# Scenes
# Objects
# Avg.Objects
Layout Generation
Physical Optimization
Part Annots.
Part-Level Contact Annots.
HSSD-200 [ 27 ]
211
18656
329.7
Human-designed
✗
✗
✗
3D-FRONT [ 15 ]
18968
13151
6.9
Human-designed
✗
✗
✗
FurniScene [ 67 ]
111698
39691
14.4
Human-designed
✗
✗
✗
METASCENES [ 65 ]
706
15366
-
Real-world Scanned
✓
✗
✗
PARSE-10K (Ours)
10000
17372
49.9
Real-image Guided
✓
✓
✓
Table 1: Comparison of indoor 3D scene datasets. Columns from left to right denote: dataset name, number of scenes, number of objects, average objects per scene, layout generation method, whether physics simulation or optimization is applied, whether object parts are annotated, and whether part–part contact annotations are provided.
Figure 4: Gallery of PARSE-10K.
Models
Visual Relation ↑
Part-level Contact ↑
Scene Graph Generation
Recall ↑
Precision ↑
F1 Score ↑
Avg.
GPT-5
82.1
75.2
13.7/40.9
13.9/41.3
13.8/41.1
15.3
Gemini-2.5-Pro
85.0
75.6
40.5/43.4
48.6/52.0
44.2/47.3
12.9
Claude-Opus-4
80.3
73.2
8.0/33.7
12.7/53.7
9.8/41.4
9.7
Robobrain2.0
60.8
37.2
9.2/11.3
26.7/32.8
13.7/16.9
5.6
Qwen3-VL
86.2
60.4
26.0/29.6
46.0/52.4
33.2/37.9
8.7
Table 2: Quantitative comparison with baselines. We evaluate models across three tasks: visual relation MCQ, part-level contact MCQ, and scene graph generation (SGG). For SGG, each metric is reported in the format With BBox Matching / No BBox Matching . The Avg. column indicates the average number of relations generated per scene by each model.
Figure 5: Visualization of model-predicted graphs. Green boxes indicate objects correctly matched by both label and grounding; red boxes indicate failed matches; gray boxes denote missed detections. Green arrows denote relations judged correct under the grounding-agnostic metric, red arrows denote incorrect relations.
Figure 6: Scene generation comparison . Left column: scenes generated by InstructScene trained on 3D-FRONT. Middle column: scenes generated by our method trained on PARSE-10K without PAG control. Right column: scenes generated by the model with PAG control.
Method
Complexity ↑
Realism ↑
Contact Fidelity ↑
InstructScene
7.5%
33.8%
28.8%
Ours(uncond)
45.0%
27.5%
26.3%
Ours(cond)
47.5%
38.8%
45.0%
Table 3: User study. The table reflects the percentage of user votes for scenes generated from the corresponding model.
Figure 7: Overview of PARSE Framework
Relation ( r )
Semantic Description
Support?
Default Surfaces ( SA→SB )
Geometric Constraints & Solver Logic
1. Part-Level: Support Relations
These relations define the primary parent-child dependency, organizing the scene assembly order.
Table 4: Dictionary of Spatial Relations in PARSE. We categorize relations into two primary granularities: Part-Level (fine-grained geometric control) and Object-Level (coarse layout). Specific Part-Level relations function as Support Relations, defining the scene’s spanning tree. To handle unspecified inputs, we define Default Surfaces for implicit resolving. Notation: A is Child, B is Parent; S denotes surface, n denotes normal, x denotes centroid.
Figure 8: Overview of the part-level contact detection pipeline. After simulating the scene, we perform contact detection on collision meshes by uniformly sampling surface points on every pair of convex hulls across object pairs. Each detected convex-hull contact pair is then mapped back to its corresponding visual-mesh parts, producing a set of part-level contact pairs (each row maps its hull-level pairs to the corresponding final part-level pair on the right). These part contacts are aggregated to construct the part-level contact graph (the red highlight indicating the contacts between the glasses and the book).
Figure 9: Distribution of Object Categories
Figure 12: Distribution of Object Density Across Rooms
Spatial reasoning studies how systems represent geometry, relations, viewpoints, and motion to understand scenes, plan actions, and solve spatial tasks.
Automatically generating interactive 3D indoor scenes from natural language is crucial for virtual reality, gaming, and embodied AI. However, existing LLM-based approaches often suffer from spatial errors and collisions, in part because common scene representations-raw coordinates or verbose code-are difficult for models to reason about 3D spatial relationships and physical constraints. We propose SpatialGrammar, a domain-specific language that represents gravity-aligned indoor layouts as BEV grid placements with deterministic compilation to valid 3D geometry, enabling verifiable constraint checking. Building on this representation, we develop (1) SG-Agent, a closed-loop system that uses compiler feedback to iteratively refine scenes and enforce collision constraints, and (2) SG-Mini, a 104M-parameter model trained entirely on compiler-validated synthetic data. Across 159 test scenes spanning five scenarios of different complexity, SG-Agent improves spatial fidelity and physical plausibility over prior methods, while SG-Mini performs competitively against larger LLM-based baselines on single-shot generation scenarios.
Song Tang, Kaiyong Zhao, Yuliang Li +5
The Hong Kong University of Science and Technology (Guangzhou) · XGRIDS · Harbin Institute of Technology (Shenzhen)
Relative spatial relations provide a compact representation of spatial structure and are fundamental to relative spatial reasoning in 3D layout generation. Recent works leverage Multimodal Large Language Models (MLLMs) to infer such relations, but the inferred relations are often unreliable and are typically handled with post-hoc heuristics. In this paper, we propose R3L, a general framework that improves the reliability and consistency of relative spatial reasoning for 3D layout generation. Our key motivation is that multi-hop reasoning requires repeated reference-frame transformations, which accumulate errors in inferred relations and lead to semantic and metric drift. To mitigate this, we propose invariant spatial decomposition to break coupled relation chains, and consistent spatial imagination to promote self-consistency through an imagine-and-revise loop. We further introduce supportive spatial optimization to ease pose optimization via global-to-local coordinate re-parameterization. Extensive experiments across diverse scene types and instructions demonstrate that R3L produces more physically feasible and semantically consistent layouts. Notably, our analysis shows that resolving frame-induced inconsistencies is crucial for reliable multi-hop relative spatial reasoning. The code is available at https://github.com/Neal2020GitHub/R3L.
Zhifeng Gu, Yuqi Wang, Bing Wang
Spatial Intelligence Group, The Hong Kong Polytechnic University.
Recent advances in 3D multimodal large language models (3D-MLLMs) have enabled unified solutions for 3D scene understanding tasks, including visual question answering, captioning, and referring segmentation. However, existing 3D-MLLMs remain largely object-centric, limiting their ability to model fine-grained part structures that are essential for embodied interaction with 3D environments. In this work, we present PAR3D, a unified part-aware 3D-MLLM framework that enables models to understand, reason about, and ground both objects and their parts in 3D scenes. To enable training and evaluation of part-aware 3D scene understanding, we introduce ScenePart, a synthetic 3D scene dataset with part-level annotations and language instructions. We further develop Part-Aware 3D Representation Learning to enrich 3D visual representations with fine-grained part-level semantics, and propose Hierarchical Segmentation Query Generation to ground part targets via hierarchical object-part queries. Extensive experiments show that our method substantially improves part-level question answering and referring segmentation, while also achieving strong performance across object-level vision-language tasks.
Shaohui Dai, Yansong Qu, You Shen +2
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University