Functional 3D Scene Generation

Ask about this topic

Overview

Functional 3D scene generation studies how to build environments that are not only visually plausible but also usable for activities such as navigation, object manipulation, and simulation. Methods include language- or vision-guided agents, procedural scene programs, learned object-relation models, and generative asset and layout systems, often combined with geometric constraints and physics-based checking or repair. Work spans indoor layouts, tabletop interactions, and larger simulated worlds for robotics, with evaluation based on measures such as collisions, reachability, relation correctness, task completion, and human preference. A central challenge is balancing visual quality and diversity with physical validity and functional completeness, while keeping generation efficient, editable, and reliable beyond familiar settings.

Latest papers

CardsList

34 papers9 papers in the last four weeks, up 29% on the four weeks before. 0.1% of all new papers.

  1. ScIntBench: Physical Interaction Plausibility in 3D Scene Generation

    Oct 27, 2026Wendi Ying, Daochang Liu, Leonid Sigal +13D Scene GenerationIndoor Scene Generation

    The University of Western Australia · University of British Columbia · Canada CIFAR AI Chair

  2. SceneBloom: Agentic Indoor Environment Generation with Realistic and Diverse Layouts for Embodied Learning

    Oct 23, 2026Yuhan Liu, Lianhui Qin, Shenji WanIndoor Scene Generation3D Scene Layout Generation

    New York University · University of California, San Diego

  3. Scene-Designer: Learning Design Intelligence for Agentic 3D Scene Generation

    Oct 16, 2026Xuanyu Liu, Hongyan Feng, Miao Pan +53D Scene GenerationIndoor Scene Generation

    Zhejiang University · Universal Identification Technology (Hangzhou) Co.,Ltd.

  4. PhySGA: A Physics-Space Grounded Agentic Framework for Generalizable and Interactive 3D Composition

    Oct 14, 2026Weichen Lyu, Chong Xia, Mingyu Dong +33D Scene GenerationRobotic Assembly

    Tsinghua University · Zhejiang University

  5. EmbodiedSmith: Scaling Embodied Data through Recursive Self-Improvement Flywheel in Simulation

    Oct 6, 2026Yikai Qin, Yifei Deng, Mingjian Liang +13Robot Foundation ModelsRobotics Simulation

    The Hong Kong University of Science and Technology (Guangzhou) · Xiaomi Robotics · Southeastern University +7

  6. DistScene: Object-to-Scene Distillation for 3D Scene Generation

    Oct 3, 2026Kunming Luo, Hongyu Yan, Ken Deng +63D Scene GenerationImage-to-3D Scene Generation

    The Hong Kong University of Science and Technology · TeleAI, China Telecom

  7. LMBuild: Evaluating LLM Agents for Generating Buildable and Functional Structures

    Oct 3, 2026Jiateng Liu, Rushi Wang, Cheng Qian +9LLM Agent EvaluationAI Agent Benchmarks

    University of Illinois Urbana-Champaign

  8. SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RL

    Oct 1, 2026Hyeonmin Lee, Zheng Wei, Kyungmin Kwon +3Human-in-the-Loop AI3D Scene Generation

    Sungkyunkwan University · The Hong Kong University of Science and Technology

  9. Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation

    Oct 1, 2026Awomo-PhysicalRSI Team, Danjiao Ma, Enhui Ma +21Robot Demonstration GenerationSynthetic Data Generation for Robotics

    Awomo-PhysicalRSI Team

  10. SPOON: Towards Coherent Compositional 3D Scene Generation from Uncalibrated Multi-view Images

    Sep 30, 2026Guibiao Liao, Mochu Xiang, Heng Li +6Object Pose Estimation3D Scene Generation

    The University of Hong Kong · Shenzhen Loop Area Institute · TranscEngram +2

  11. WorldAgent: Verification-Guided Agentic Physical World Construction

    Sep 27, 2026Caoliwen Wang, Mengdi Wang, Yige Chen +133D Scene GenerationFunctional 3D Scene Generation

    University of British Columbia · VAST · Zhejiang University +3

  12. Fysiverse-3D-Vision Technical Report: Generating Executable 3D Worlds from Images through Unified Spatial Reasoning

    Sep 22, 2026Dingkang Yang, Yizhou Liu, Wendong Cheng +53D Scene GenerationFunctional 3D Scene Generation

    Physical Superintelligence Lab, Fysics AI · Multimedia Laboratory (MMLab), The Chinese University of Hong Kong · College of Intelligent Robotics and Advanced Manufacturing, Fudan University

  13. ESG: Generating Physically Consistent Dynamic 3D Scenes from Text Descriptions

    Sep 14, 2026Xintong Fang, Zhiyuan Fang, Rengan Xie +63D SGG3D Scene Generation

    Zhejiang University, Hangzhou, China · State Key Laboratory of CAD&CG, Zhejiang University, Hangzhou, China · Korea Advanced Institute of Science and Technology (KAIST) +1

  14. GIF: Agentic Generation of Interactive and Functional Object Compositions for Robot Learning

    Sep 5, 2026Long Xu, Zhiqi Zhang, Mi Yan +8Synthetic Data Generation for Robotics3D Scene Generation

    Galbot · Zhejiang University · Peking University +3

  15. WorldSculpt: Generating Compositional Worlds from Grounded Videos

    Sep 4, 2026Muyao Niu, Jixuan He, Ruihan Yu +93D Scene Generation3D Scene Representation

    Alaya Lab · The University of Tokyo

  16. SceneMosaic: Efficient and Diverse Simulation-Ready Scene Generation via Hybrid Agentic Layout Evolution

    Sep 4, 2026Xingjian Ran, Xiaoye Mo, Sihao Liu +33D Scene Generation3D Scene Layout Generation

    The University of Hong Kong · University of Electronic Science and Technology of China

  17. ScenePilot: Grow-and-Repair Policy for Text-Driven 3D Indoor Scene Generation

    Aug 31, 2026Jiawei Zhang, Hongsong Wang, Pan Zhou3D Scene EditingFunctional 3D Scene Generation

    Southeast University · Singapore Management University

  18. FuncRoom-Agent: Sequential Feed-Forward 3D Functional Indoor Scene Generation

    Aug 30, 2026Hao Feng, Zhi Zuo, MingJian Liang +6Process Reward Models3D Scene Generation

  19. SpatialCrafter: Single Image World Modeling with Generative 3D Proxies

    Aug 27, 2026Chuan Fang, Lingteng Qiu, Yixun Liang +8Image-to-3D Scene Generation3D Scene Generation

    Hong Kong University of Science and Technology, Hong Kong · Tongyi Lab, Alibaba Group, Hang Zhou, China · ManyCore Tech Inc., Hang Zhou, China +1

  20. Beyond Placement and Articulation: Usage-Driven Code Scenes for Embodied Interaction

    Aug 19, 2026Zijian Xiao, Zipeng Ye, Jinkun Hao +33D Scene GenerationFunctional 3D Scene Generation

    Shanghai Jiao Tong University · Meituan

  21. D3D-GEN: Robot-Aware Domain-Grounded Interactive 3D World Generation for Social Robotics

    Aug 12, 2026Anh Duc Do, Volodymyr Shcherbyna, Tai Duc Nguyen +5Robotics SimulationFunctional 3D Scene Generation

    Singapore Management University · Max Planck Institute · Technical University Braunschweig

  22. iARCS: Iterative Agentic RL for Controllable 3D Scene Generation

    Aug 6, 2026Saugat Adhikari, Ashok Prasad Neupane, Pramish Paudel +2Constrained Generative ModelingAgentic RL

    Pulchowk Campus, IOE, Tribhuvan University, Lalitpur, Nepal · INSAIT, Sofia University “St. Kliment Ohridski”, Bulgaria · NAAMII, Kathmandu, Nepal

  23. WorldClaw: Agentic 3D Open-World Generation at Scale

    Aug 5, 2026Chunchao Guo, Jinpeng Li, Yang Li +1Open-Ended Generation3D Asset Generation

    Tencent Hunyuan

  24. EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AI

    Jul 8, 2026Xinjie Wang, Liu Liu, Taojun Ding +93D Asset GenerationRobotics Simulation

    Horizon Robotics · WuwenAI

  25. CP4D: Compositional Physics-aware 4D Scene Generation

    Jun 8, 2026Hanxin Zhu, Cong Wang, Tianyu He +4Physical Consistency in Video Generation3D Scene Generation

    School of Information Science and Technology, University of Science and Technology of China · Zhongguancun Academy · the State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences. +2

  26. PhyScene3D: Physically Consistent Interactive 3D Tabletop Scene Generation

    Jun 1, 2026Weixing Chen, Zhuoqian Feng, Yang Liu +63D Scene GenerationFunctional 3D Scene Generation

    Sun Yat-sen University, China · Guangdong Key Laboratory of Big Data Analysis and Processing · Huawei +1

  27. Function2Scene: 3D Indoor Scene Layout from Functional Specifications

    May 29, 2026Ruiqi Wang, Qimin Chen, Daniel Ritchie +43D Scene GenerationFunctional 3D Scene Generation

    Simon Fraser University, Canada · Brown University, USA · Simon Fraser University, Canada and ShanghaiTech University, China

  28. SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objects

    May 19, 2026Puyi Wang, Yuhao Wang, Linjie Li +43D Asset GenerationLLM-Based Program Synthesis

    1The Chinese University of Hong Kong · 2Shanghai Jiao Tong University · 3Shanghai AI Laboratory +2

  29. TelePhysics: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interaction

    May 19, 2026Xin Zhang, Yabo Chen, Yijie Fang +5Physical Consistency in Video GenerationReal-Time Interactive Video Generation

    Fudan University · Institute of Artificial Intelligence, China Telecom (TeleAI)

  30. PG-3DGS: Optimizing 3D Gaussian Splatting to Satisfy Physics Objectives

    May 11, 2026Zachary Lee, Maxwell Jacobson, Yexiang Xue3D Gaussian Splatting3D Shape Generation

    Department of Computer Science, Purdue University, West Lafayette, 47907, IN, United States