Functional 3D Scene Generation

Follow this topic Ask about this topic

Functional 3D scene generation studies how to build environments that are not only visually plausible but also usable for activities such as navigation, object manipulation, and simulation. Methods include language- or vision-guided agents, procedural scene programs, learned object-relation models, and generative asset and layout systems, often combined with geometric constraints and physics-based checking or repair. Work spans indoor layouts, tabletop interactions, and larger simulated worlds for robotics, with evaluation based on measures such as collisions, reachability, relation correctness, task completion, and human preference. A central challenge is balancing visual quality and diversity with physical validity and functional completeness, while keeping generation efficient, editable, and reliable beyond familiar settings.

Latest papers

34 papers9 papers in the last four weeks, up 29% on the four weeks before. 0.1% of all new papers.

Pair2Scene: Learning Local Object Relations for Procedural Scene Generation
Apr 13, 2026Xingjian Ran, Shujie Zhang, Weipeng Zhong +23D Scene Generation 3D Scene Layout Generation
The University of Hong Kong · Tsinghua University · Shanghai Jiao Tong University +1
Pano3DComposer: Feed-Forward Compositional 3D Scene Generation from Single Panoramic Image
Mar 6, 2026Zidian Qiu, Ancong Wu3D Scene Generation Image-to-3D Scene Generation
Sun Yat-sen University
RoboLayout: Differentiable 3D Scene Generation for Embodied Agents
Feb 18, 2026Ali Shamsaddinlou3D Scene Layout Generation 3D Scene Generation
PhyScensis: Physics-Augmented LLM Agents for Complex Physical Scene Arrangement
Feb 16, 2026Yian Wang, Han Yang, Minghao Guo +53D Scene Layout Generation Functional 3D Scene Generation
UMass Amherst · Genesis AI · MIT +1

Apr 13, 2026J/K move · Enter open · S save

Pair2Scene: Learning Local Object Relations for Procedural Scene Generation

Xingjian Ran, Shujie Zhang, Weipeng Zhong, Li Luo+1

The University of Hong Kong · Tsinghua University · Shanghai Jiao Tong University+1

3D Scene Generation 3D Scene Layout Generation Indoor Scene Generation Functional 3D Scene Generation

Open paper AskPDF arXiv

Generating high-fidelity 3D indoor scenes remains a significant challenge due to data scarcity and the complexity of modeling intricate spatial relations. Current methods often struggle to scale beyond training distribution to dense scenes or rely on LLMs/VLMs that lack the ability for precise spatial reasoning. Building on top of the observation that object placement relies mainly on local dependencies instead of information-redundant global distributions, in this paper, we propose Pair2Scene, a novel procedural generation framework that integrates learned local rules with scene hierarchies and physics-based algorithms. These rules mainly capture two types of inter-object relations, namely support relations that follow physical hierarchies, and functional relations that reflect semantic links. We model these rules through a network, which estimates spatial position distributions of dependent objects conditioned on position and geometry of the anchor ones. Accordingly, we curate a dataset 3D-Pairs from existing scene data to train the model. During inference, our framework can generate scenes by recursively applying our model within a hierarchical structure, leveraging collision-aware rejection sampling to align local rules into coherent global layouts. Extensive experiments demonstrate that our framework outperforms existing methods in generating complex environments that go beyond training data while maintaining physical and semantic plausibility.

Latest papers

Pair2Scene: Learning Local Object Relations for Procedural Scene Generation

Pano3DComposer: Feed-Forward Compositional 3D Scene Generation from Single Panoramic Image

RoboLayout: Differentiable 3D Scene Generation for Embodied Agents

PhyScensis: Physics-Augmented LLM Agents for Complex Physical Scene Arrangement