Pair2Scene: Learning Local Object Relations for Procedural Scene Generation

Xingjian Ran; Shujie Zhang; Weipeng Zhong; Li Luo; Bo Dai

cs.CVApr 13, 2026

Pair2Scene: Learning Local Object Relations for Procedural Scene Generation

Authors: Xingjian Ran, Shujie Zhang, Weipeng Zhong, Li Luo, Bo Dai

Organizations: The University of Hong Kong · Tsinghua University · Shanghai Jiao Tong University · Shenzhen Loop Area Institute

Abstract

Generating high-fidelity 3D indoor scenes remains a significant challenge due to data scarcity and the complexity of modeling intricate spatial relations. Current methods often struggle to scale beyond training distribution to dense scenes or rely on LLMs/VLMs that lack the ability for precise spatial reasoning. Building on top of the observation that object placement relies mainly on local dependencies instead of information-redundant global distributions, in this paper, we propose Pair2Scene, a novel procedural generation framework that integrates learned local rules with scene hierarchies and physics-based algorithms. These rules mainly capture two types of inter-object relations, namely support relations that follow physical hierarchies, and functional relations that reflect semantic links. We model these rules through a network, which estimates spatial position distributions of dependent objects conditioned on position and geometry of the anchor ones. Accordingly, we curate a dataset 3D-Pairs from existing scene data to train the model. During inference, our framework can generate scenes by recursively applying our model within a hierarchical structure, leveraging collision-aware rejection sampling to align local rules into coherent global layouts. Extensive experiments demonstrate that our framework outperforms existing methods in generating complex environments that go beyond training data while maintaining physical and semantic plausibility.

Figures & tables

Figure 2: Architecture of Pair2Scene Model. The pipeline processes the dependent asset $\mathcal{A}_{dep}$ and anchor objects (support $\mathcal{O}_{sup}$ and functional $\mathcal{O}_{fnc}$ ) to predict the spatial distribution of the dependent object. Point cloud encoders extract latent geometric features $\mathbf{z}^{geo}$ , while the bounding box encoder transforms spatial configurations into embeddings $\mathbf{e}^{bbox}$ . These are processed through cascaded Transformer-like blocks consisting of Relational Self-Attention, which captures global relations using learnable query tokens $\mathbf{x}_{m}$ , and Geometry-Aware Cross-Attention for fine-grained geometric reasoning. Finally, the MLP head outputs the parameters for a MoL distribution, providing a multi-modal probabilistic framework for object placement.

Figure 3: Scene Assembly Pipeline. The framework parses text or room type into support and functional Trees, serialized as an ordered relation sequence. The Pair2Scene model then predicts spatial distributions based on these relations. Finally, rejection sampling resolves global collisions to produce the final 3D scene.

Figure 4: Data Curation Pipeline. Our data curation workflow involves Physical Validation to filter unstable objects, Support Extraction to identify support dependencies, and LLM-Driven Functional Distillation to extract functional relations. This process results in the 3D-Pair Dataset, which captures fine-grained object interactions.

Figure 5: Qualitative comparison under the 3D-Front only setting.

Figure 6: Qualitative comparison under the multi-source setting.

Method	FID $\downarrow$	KID ( $\times 1e^{-3}$ ) $\downarrow$	Objects
ATISS	71.24 $\pm$ 1.55	42.18 $\pm$ 1.42	7.65
DiffuScene	67.45 $\pm$ 0.92	31.72 $\pm$ 1.08	6.75
LayoutVLM	120.87 $\pm$ 2.05	138.54 $\pm$ 3.12	-
FactoredScenes	104.12 $\pm$ 2.78	129.45 $\pm$ 2.64	8.53
Ours-Fit	65.92 $\pm$ 1.14	22.14 $\pm$ 0.85	6.98
Ours-Beyond	75.88 $\pm$ 1.22	69.05 $\pm$ 1.58	14.15

Table 1: Comparison of FID and KID on the 3D-Front only setting.

Method	SA $\uparrow$	PP $\uparrow$	SC $\uparrow$	MQ $\uparrow$	CFS $\uparrow$
ATISS	3.14	3.27	2.27	3.21	1.21
DiffuScene	4.45	4.14	2.86	4.30	2.05
FactoredScenes	1.73	2.27	2.82	2.00	0.94
LayoutVLM	2.27	2.14	3.91	2.21	1.44
Ours-Fit	4.18	4.18	3.91	4.18	2.73
Ours-Beyond	5.23	5.00	5.23	5.12	4.46

Table 2: User study results on the 3D-Front only setting.

Method	SA $\uparrow$	PP $\uparrow$	SC $\uparrow$	MQ $\uparrow$	CFS $\uparrow$
FactoredScenes	1.50	2.05	1.91	1.78	0.68
Holodeck	3.36	3.59	2.09	3.48	1.45
Infinigen-Indoors	2.50	2.73	3.09	2.62	1.62
LayoutVLM	3.09	2.32	3.18	2.71	1.72
Ours	4.55	4.32	4.73	4.44	4.20

Table 3: User study results on the multi-source setting.

Variant	FID $\downarrow$	KID ( $\times 1e^{-3}$ ) $\downarrow$
w/o relation	92.34	82.74
w/o pretrain	81.14	73.91
w/o rejection	78.26	68.56
Class-only	68.45	35.23
w/o gravity	66.82	28.48
Full	65.15	22.32

Table 4: Ablation study results on the 3D-Front only setting.

Figure 7: Qualitative ablation study on the multi-source setting.

Figure 8: More visualization results on new scene types.

Method	3D-Front only	Multi-source
ATISS ( Paschalidou et al., 2021 )	9.45s	-
DiffuScene ( Tang et al., 2024 )	28.16s	-
Holodeck ( Yang et al., 2024b )	-	210.23s
Infinigen-Indoors ( Raistrick et al., 2024 )	-	686.44s
LayoutVLM ( Sun et al., 2025a )	247.92s	373.69s
FactoredScenes ( Hsu et al., 2025 )	85.26s	85.26s

Table 5: Comparison of Inference Time (Seconds per Scene).

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Acc. Rate $\uparrow$	Resample/Obj. $\downarrow$
Ours-Fit	75.63%	0.322
Ours-Beyond	57.12%	0.751

Appendix

Table 6: Comparison of Accuracy Rate and Resampling per Object.

Datasource	Precision $\uparrow$	Prec. (w/o $k$ )	Recall $\uparrow$	Rec. (w/o $k$ )	F1 $\uparrow$	F1 (w/o $k$ )
3D-Front	0.924 $\pm$ 0.065	0.909 $\pm$ 0.062	0.921 $\pm$ 0.059	0.927 $\pm$ 0.070	0.922	0.918
MesaTask	0.846 $\pm$ 0.041	0.777 $\pm$ 0.058	0.933 $\pm$ 0.085	0.967 $\pm$ 0.073	0.887	0.862
InternScenes	0.800 $\pm$ 0.095	0.763 $\pm$ 0.073	0.807 $\pm$ 0.110	0.819 $\pm$ 0.067	0.803	0.790

Appendix

Table 7: Quantitative validation of LLM-based extraction and ablation study of the

k

-check mechanism. We report mean and standard deviation across 30 random scenes per source.

Figure 9: Screenshot of the user study interface.

Variant	Floating Rate $\downarrow$
w/o gravity	$0.27\pm 0.12$
Full (Ours)	$0$

Appendix

Table 8: Ablation analysis of the floating rate.

Comparison with Baselines		Ablation Study
Method	Collision Rate $\downarrow$	Variant	Collision Rate $\downarrow$
ATISS	$0.31\pm 0.14$	w/o relation	$0.33\pm 0.18$
DiffuScene	$0.24\pm 0.11$	w/o pretrain	$0.07\pm 0.06$
LayoutVLM	$0.27\pm 0.06$	w/o rejection	$0.25\pm 0.08$
FactoredScenes	$0.25\pm 0.09$	Class-only	$0.03\pm 0.04$
Ours-Fit	$0.02\pm 0.05$	w/o gravity	$0.02\pm 0.05$

Appendix

Table 9: Collision rate results: comparison with baselines (left) and ablation study (right).

Figure 10: Distribution of dependents for floor as support anchor.

Figure 11: Distribution of dependents for cabinet as support anchor.

Figure 12: Distribution of dependents for floor as support anchor and bed as functional anchor.

Figure 13: Distribution of dependents for floor as support anchor and cabinet as functional anchor.

Figure 14: Distribution of dependents for floor as support anchor and table as functional anchor.

Figure 15: Distribution of dependents for table as support anchor.

Figure 16: Distribution of dependents for table as support anchor and book as functional anchor.

Figure 17: Distribution of dependents for table as support anchor and bowl as functional anchor.

Figure 18: Distribution of dependents for table as support anchor and computer as functional anchor.

3D Scene Generation

3D scene generation creates structured, explorable environments from text, images, maps, or other inputs.

See all 190 papers

ThinkBLOX: 3D Indoor Scene Generation with Progressive Reasoning
Jul 15, 2026Yuan Xiao, Can Wang, Xiangyu Kong +13D Scene Editing 3D Scene Generation
Department of Computer Science, City University of Hong Kong, Hong Kong · School of Computing and Data Science, University of Hong Kong · School of Computer Science, Beijing Information Science and Technology University
DistScene: Object-to-Scene Distillation for 3D Scene Generation
Oct 3, 2026Kunming Luo, Hongyu Yan, Ken Deng +63D Scene Generation Image-to-3D Scene Generation
The Hong Kong University of Science and Technology · TeleAI, China Telecom
Tetris3D: 3D Scene Generation With Objects That Fit Together
Oct 7, 2026Jaeyeong Kim, Jinhyuk Jang, Jongmin Lee +23D Scene Generation Image-to-3D Scene Generation
KAIST AI

Jul 15, 2026J/K move · Enter open · S save

ThinkBLOX: 3D Indoor Scene Generation with Progressive Reasoning

Yuan Xiao, Can Wang, Xiangyu Kong, Jing Liao

Department of Computer Science, City University of Hong Kong, Hong Kong · School of Computing and Data Science, University of Hong Kong · School of Computer Science, Beijing Information Science and Technology University

3D Scene Editing 3D Scene Generation 3D Scene Layout Generation Indoor Scene Generation

Open paper AskPDF arXiv

While traditional graphics methods often synthesize 3D indoor scenes autoregressively or hierarchically, recent vision-language model (VLM)-based generators predominantly adopt a one-shot paradigm where the full layout is planned at once. This one-shot approach often requires global re-optimization or complete reconstruction during interactive editing (e.g., inserting or moving objects) and can lead to physically or semantically poorly organized arrangements. To address these challenges, we propose ThinkBLOX, a VLM-based progressive reasoning framework that iteratively designs and refines 3D scenes. ThinkBLOX treats layout generation as a state-conditioned, step-by-step reasoningand-action process. To power this, we construct the ThinkBLOX-Data-200K dataset, containing 224,757 procedural placement pairs annotated with multi-view scene context, explicit Chain-of-Thought (CoT) rationales, and structured JSON layouts. Through supervised fine-tuning (SFT) on this dataset, the VLM learns to bridge the reasoning-action gap under incremental updates. Furthermore, recognizing that scene synthesis is inherently a multisolution task where SFT suffers from reward conflict, we introduce Tier-Decoupled GDPO. This reinforcement learning scheme organizes heterogeneous rewards into distinct tiers, stabilizing policy optimization across physical validity, semantic plausibility, and reasoning-action consistency. Extensive experiments show that ThinkBLOX significantly outperforms recent one-shot and iterative baselines in physical plausibility, semantic alignment, and interactive editability. Additionally, we show that it supports diverse applications, including both global and local generation and rearrangement of 3D scenes.

Abstract

3D Scene Generation

Explore similar work

ThinkBLOX: 3D Indoor Scene Generation with Progressive Reasoning

DistScene: Object-to-Scene Distillation for 3D Scene Generation

Tetris3D: 3D Scene Generation With Objects That Fit Together