PARSE: Part-Aware Relational Spatial Modeling

Yinuo Bai; Peijun Xu; Kuixiang Shao; Yuyang Jiao; Jingxuan Zhang; Kaixin Yao; Jiayuan Gu; Jingyi Yu

cs.CVMar 8, 2026

PARSE: Part-Aware Relational Spatial Modeling

Authors: Yinuo Bai, Peijun Xu, Kuixiang Shao, Yuyang Jiao, Jingxuan Zhang, Kaixin Yao, Jiayuan Gu, Jingyi Yu

Organizations: ShanghaiTech University · Deemos Technology

Abstract

Inter-object relations underpin spatial intelligence, yet existing representations -- linguistic prepositions or object-level scene graphs -- are too coarse to specify which regions actually support, contain, or contact one another, leading to ambiguous and physically inconsistent layouts. To address these ambiguities, a part-level formulation is needed; therefore, we introduce PARSE, a framework that explicitly models how object parts interact to determine feasible and spatially grounded scene configurations. PARSE centers on the Part-centric Assembly Graph (PAG), which encodes geometric relations between specific object parts, and a Part-Aware Spatial Configuration Solver that converts these relations into geometric constraints to assemble collision-free, physically valid scenes. Using PARSE, we build PARSE-10K, a dataset of 10,000 3D indoor scenes constructed from real-image layout priors and a curated part-annotated shape database, each with dense contact structures and a part-level contact graph. With this structured, spatially grounded supervision, fine-tuning Qwen3-VL on PARSE-10K yields stronger object-level layout reasoning and more accurate part-level relation understanding; furthermore, leveraging PAGs as structural priors in 3D generation models leads to scenes with substantially improved physical realism and structural complexity. Together, these results show that PARSE significantly advances geometry-grounded spatial reasoning and supports the generation of physically consistent 3D scenes.

Figures & tables

Figure 1: Overview of the capabilities of PARSE. Using a real image (a) as reference, we first construct Part-centric Assembly Graphs (PAGs) (b) that capture its spatial organization of objects. Then, by combining PARSE with physics simulation, we generate physically plausible 3D scenes (c) from these PAGs, featuring diverse inter-object relationships and rich part-level contacts. Furthermore, we introduce PARSE-10K, a collection of high-quality 3D indoor scenes with fully part-segmented object instances, which effectively supports downstream tasks such as fine-tuning VLMs for spatial reasoning and enhancing 3D scene generation.

Figure 2: An illustrative example of our Part-centric Assembly Graph (PAG). Left: A 3D scene generated by PARSE, with specific local regions highlighted. Middle: The corresponding global PAG, emphasizing the sub-PAGs that match the highlighted regions on the left. In the graph, large labeled circles represent object nodes, while small dark circles attached to them represent part nodes (unrelated part nodes are omitted for clarity). Right: Zoomed-in views of three sub-PAGs. These panels explicitly annotate the specific surfaces used to define relational constraints, alongside the corresponding Object-Level Spatial Edges and Part-Level Geometric Edges.

Figure 3: Controllable Scene Synthesis via Part-Aware Spatial Configuration Solver. (a) Coarse Localization: The solver first prunes occupied regions (red) from the 2D support surface, then further contracts the feasible space using object-level spatial relations (orange). (b) Part-Level Alignment: Precise geometric alignment is achieved by enforcing constraints ( e.g . , coplanarity) between specific surfaces identified by the solver. This drastically reduces the feasible pose space for final pose sampling. (c) Fine-Grained Relational Control: Specifying different part-level geometric relations in the PAG results in distinct and predictable arrangements, showcasing the framework’s fine-grained controllability.

Dataset	# Scenes	# Objects	# Avg.Objects	Layout Generation	Physical Optimization	Part Annots.	Part-Level Contact Annots.
HSSD-200 [ 27 ]	211	18656	329.7	Human-designed	✗	✗	✗
3D-FRONT [ 15 ]	18968	13151	6.9	Human-designed	✗	✗	✗
FurniScene [ 67 ]	111698	39691	14.4	Human-designed	✗	✗	✗
METASCENES [ 65 ]	706	15366	-	Real-world Scanned	✓	✗	✗
PARSE-10K (Ours)	10000	17372	49.9	Real-image Guided	✓	✓	✓

Table 1: Comparison of indoor 3D scene datasets. Columns from left to right denote: dataset name, number of scenes, number of objects, average objects per scene, layout generation method, whether physics simulation or optimization is applied, whether object parts are annotated, and whether part–part contact annotations are provided.

Models	Visual Relation $\uparrow$	Part-level Contact $\uparrow$	Scene Graph Generation
Models	Visual Relation $\uparrow$	Part-level Contact $\uparrow$	Recall $\uparrow$	Precision $\uparrow$	F1 Score $\uparrow$	Avg.
GPT-5	82.1	75.2	13.7/40.9	13.9/41.3	13.8/41.1	15.3
Gemini-2.5-Pro	85.0	75.6	40.5/43.4	48.6/52.0	44.2/47.3	12.9
Claude-Opus-4	80.3	73.2	8.0/33.7	12.7/53.7	9.8/41.4	9.7
Robobrain2.0	60.8	37.2	9.2/11.3	26.7/32.8	13.7/16.9	5.6
Qwen3-VL	86.2	60.4	26.0/29.6	46.0/52.4	33.2/37.9	8.7

Table 2: Quantitative comparison with baselines. We evaluate models across three tasks: visual relation MCQ, part-level contact MCQ, and scene graph generation (SGG). For SGG, each metric is reported in the format With BBox Matching / No BBox Matching . The Avg. column indicates the average number of relations generated per scene by each model.

Figure 5: Visualization of model-predicted graphs. Green boxes indicate objects correctly matched by both label and grounding; red boxes indicate failed matches; gray boxes denote missed detections. Green arrows denote relations judged correct under the grounding-agnostic metric, red arrows denote incorrect relations.

Figure 6: Scene generation comparison . Left column: scenes generated by InstructScene trained on 3D-FRONT. Middle column: scenes generated by our method trained on PARSE-10K without PAG control. Right column: scenes generated by the model with PAG control.

Method	Complexity $\uparrow$	Realism $\uparrow$	Contact Fidelity $\uparrow$
InstructScene	7.5%	33.8%	28.8%
Ours(uncond)	45.0%	27.5%	26.3%
Ours(cond)	47.5%	38.8%	45.0%

Table 3: User study. The table reflects the percentage of user votes for scenes generated from the corresponding model.

Relation ( $r$ )	Semantic Description	Support?	Default Surfaces ( $S_{A}\to S_{B}$ )	Geometric Constraints & Solver Logic
1. Part-Level: Support Relations
These relations define the primary parent-child dependency, organizing the scene assembly order.
ON	Object rests stably on parent.	Yes	Bottom $\to$ Top	Contact: $\text{dist}(S_{A},S_{B})\approx 0$ . Orientation: $\mathbf{n}_{A}\cdot\mathbf{n}_{B}\approx-1$ (Anti-parallel). Stability: $Proj_{xy}(\text{CoM}_{A})\in\text{Polygon}(S_{B})$ .
IN	Object is contained inside parent.	Yes	Bottom $\to$ Top(internal support surface)	Contact: $\text{dist}(S_{A},S_{B})\approx 0$ . Inclusion: $\text{RayCastEnclosure}(A,B)>\tau_{contain}$ .
HANGING_BELOW	Object suspended under parent.	Yes	Top $\to$ Bottom	Contact: $\text{dist}(S_{A},S_{B})\approx 0$ . Orientation: $\mathbf{n}_{A}\cdot\mathbf{n}_{B}\approx-1$ .
SECURED_TO	Rigidly attached to side/wall.	Yes	Back $\to$ Front	Contact: $\text{dist}(S_{A},S_{B})\approx 0$ . Orientation: $\mathbf{n}_{A}\cdot\mathbf{n}_{B}\approx-1$ .

Table 4: Dictionary of Spatial Relations in PARSE. We categorize relations into two primary granularities: Part-Level (fine-grained geometric control) and Object-Level (coarse layout). Specific Part-Level relations function as Support Relations, defining the scene’s spanning tree. To handle unspecified inputs, we define Default Surfaces for implicit resolving. Notation:

A

is Child,

B

is Parent;

S

denotes surface,

\mathbf{n}

denotes normal,

\mathbf{x}

denotes centroid.

Figure 8: Overview of the part-level contact detection pipeline. After simulating the scene, we perform contact detection on collision meshes by uniformly sampling surface points on every pair of convex hulls across object pairs. Each detected convex-hull contact pair is then mapped back to its corresponding visual-mesh parts, producing a set of part-level contact pairs (each row maps its hull-level pairs to the corresponding final part-level pair on the right). These part contacts are aggregated to construct the part-level contact graph (the red highlight indicating the contacts between the glasses and the book).

Panel of Figure 9 — Figure 9: Distribution of Object Categories

Panel of Figure 12 — Figure 12: Distribution of Object Density Across Rooms

Figure 14: User Interface for Designing PAGs

Spatial Reasoning

Spatial reasoning studies how systems represent geometry, relations, viewpoints, and motion to understand scenes, plan actions, and solve spatial tasks.

See all 159 papers

SpatialGrammar: A Domain-Specific Language for LLM-Based 3D Indoor Scene Generation
Apr 30, 2026Song Tang, Kaiyong Zhao, Yuliang Li +53D Scene Generation 3D Scene Representation
The Hong Kong University of Science and Technology (Guangzhou) · XGRIDS · Harbin Institute of Technology (Shenzhen)
R $^3$ L: Reasoning 3D Layouts from Relative Spatial Relations
May 7, 2026Zhifeng Gu, Yuqi Wang, Bing Wang3D Spatial Reasoning 3D Scene Generation
Spatial Intelligence Group, The Hong Kong Polytechnic University.
PAR3D: A Unified 3D-MLLM with Part-Aware Representation for Scene Understanding
Jun 4, 2026Shaohui Dai, Yansong Qu, You Shen +23D Part Segmentation 3D Representation Learning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University

Apr 30, 2026J/K move · Enter open · S save

SpatialGrammar: A Domain-Specific Language for LLM-Based 3D Indoor Scene Generation

Song Tang, Kaiyong Zhao, Yuliang Li, Qingsong Yan+4

The Hong Kong University of Science and Technology (Guangzhou) · XGRIDS · Harbin Institute of Technology (Shenzhen)

3D Scene Generation 3D Scene Representation 3D Scene Layout Generation Indoor Scene Generation

Open paper AskPDF arXiv

Automatically generating interactive 3D indoor scenes from natural language is crucial for virtual reality, gaming, and embodied AI. However, existing LLM-based approaches often suffer from spatial errors and collisions, in part because common scene representations-raw coordinates or verbose code-are difficult for models to reason about 3D spatial relationships and physical constraints. We propose SpatialGrammar, a domain-specific language that represents gravity-aligned indoor layouts as BEV grid placements with deterministic compilation to valid 3D geometry, enabling verifiable constraint checking. Building on this representation, we develop (1) SG-Agent, a closed-loop system that uses compiler feedback to iteratively refine scenes and enforce collision constraints, and (2) SG-Mini, a 104M-parameter model trained entirely on compiler-validated synthetic data. Across 159 test scenes spanning five scenarios of different complexity, SG-Agent improves spatial fidelity and physical plausibility over prior methods, while SG-Mini performs competitively against larger LLM-based baselines on single-shot generation scenarios.

Abstract

Spatial Reasoning

Explore similar work

SpatialGrammar: A Domain-Specific Language for LLM-Based 3D Indoor Scene Generation

R3^33L: Reasoning 3D Layouts from Relative Spatial Relations

PAR3D: A Unified 3D-MLLM with Part-Aware Representation for Scene Understanding

R $^3$ L: Reasoning 3D Layouts from Relative Spatial Relations