3D Scene Generation

Ask about this topic

Overview

3D scene generation studies how to turn prompts and visual observations into environments with coherent layouts, geometry, appearance, and sometimes motion or interaction. Methods range from procedural rules and language-model agents to diffusion, flow-matching, and autoregressive models that generate meshes, Gaussian splats, voxels, or occupancy representations. Work targets indoor design, large-scale worlds, driving scenes, and simulation for robotics, using constraints, rendering feedback, and physics checks to improve control and usability. A central challenge is balancing visual realism and scalable generation with consistent geometry, physical plausibility, editability, and reliable evaluation.

Latest papers

CardsList

191 papers25 papers in the last four weeks, up 39% on the four weeks before. 0.1% of all new papers.

  1. LucidLyra: Feed-Forward Generative 3D Scene Reconstruction and Joint Understanding via Multi-Task Distillation

    Oct 29, 2026Oliver Hahn, Rodrigo Marcuzzi, Sangeetha Grama Srinivasan +73D Scene Understanding3D Scene Generation

    NVIDIA · TU Darmstadt · Università degli Studi di Modena e Reggio Emilia +1

  2. ScIntBench: Physical Interaction Plausibility in 3D Scene Generation

    Oct 27, 2026Wendi Ying, Daochang Liu, Leonid Sigal +13D Scene GenerationIndoor Scene Generation

    The University of Western Australia · University of British Columbia · Canada CIFAR AI Chair

  3. CityLift: Compositional 3D Urban Generation from a Single Image

    Oct 23, 2026Haoran Gong, Jie Feng, Chenxu Wang3D Scene GenerationImage-to-3D Scene Generation

    School of Software Engineering, Xi’an Jiaotong University · Zhongguancun Academy, Beijing, China

  4. WorldStream: Streaming Interactive 3D Worlds with Hierarchical Planning and Persistent State

    Oct 20, 2026Xiaoxiao Ma, Weiyang Jin, Yanwen Ma +11Interactive World Models3D Scene Generation

    University of Science and Technology of China · Joy Future Academy, JD

  5. CityPaint: A Free Lunch from 2D Diffusion for City-Scale 3D Texturing

    Oct 20, 2026Yuchen Zou, Lihuang Fang, Siyu Zhu +13D Texture Generation3D Scene Generation

    Xi’an Jiaotong University Xi’an, China · Southern University of Science and Technology Shenzhen, China · International Digital Economy Academy Shenzhen, China +1

  6. InfiniDream: Open-Ended Photorealistic 3D Worlds Through Procedural Generation

    Oct 19, 2026Yi-Chuan Huang, Hao-Jen Chien, Zheng-Hui Huang +3Procedural 3D Generation3D Scene Generation

    Alaya Lab · National Yang Ming Chiao Tung University · National Taiwan University

  7. Scene-Designer: Learning Design Intelligence for Agentic 3D Scene Generation

    Oct 16, 2026Xuanyu Liu, Hongyan Feng, Miao Pan +53D Scene GenerationIndoor Scene Generation

    Zhejiang University · Universal Identification Technology (Hangzhou) Co.,Ltd.

  8. TriFold: Folding irregular 3D Gaussians into structured latents for 3D Scene Generation

    Oct 16, 2026Yun Wang, Chiyu Wang, Fan Jiang +53D Scene Generation3D Representation Learning

    City University of Hong Kong · AMAP, Alibaba Group · Hong Kong Polytechnic University

  9. PhySGA: A Physics-Space Grounded Agentic Framework for Generalizable and Interactive 3D Composition

    Oct 14, 2026Weichen Lyu, Chong Xia, Mingyu Dong +33D Scene GenerationRobotic Assembly

    Tsinghua University · Zhejiang University

  10. BlenderWorld: Benchmarking Visual Agents for 3D Creation from Real-World References

    Oct 14, 2026Xinyu Fang, Yuchi Wang, Weihao Yu +123D Scene Generation3D Reconstruction Benchmarks

    Zhejiang University · Alibaba Token Hub, Alibaba Group · The Chinese University of Hong Kong +2

  11. SceneAE: Scene Generation and Reconstruction in Queryable 3D-Anchored Latent Space

    Oct 14, 2026Shuofeng Sun, Wenzhao Zheng, Yunpeng Zhang +33D Scene Generation3D Scene Representation

    Beijing University of Posts and Telecommunications · University of California, Berkeley · PhiGent Robotics +1

  12. CityCode: Building Agent-Operable 3D Cities from Sparse Aerial Observations

    Oct 13, 2026Jiahui Sun, Ran Bi, Yifei Liu +8Aerial 3D Reconstruction3D Scene Representation

    Nanjing University · Shanghai Artificial Intelligence Laboratory · Beihang University +3

  13. ExeWorldBench: Benchmarking Code-Based Executable 3D World Modeling

    Oct 10, 2026Xinge Peng, Zhen Fang, Ruibing Wang +8World ModelsWorld Model Evaluation

    University of Science and Technology of China · The Chinese University of Hong Kong

  14. Tetris3D: 3D Scene Generation With Objects That Fit Together

    Oct 7, 2026Jaeyeong Kim, Jinhyuk Jang, Jongmin Lee +23D Scene GenerationImage-to-3D Scene Generation

    KAIST AI

  15. WorldBench: Evaluating LLMs on Three.js Voxel World Generation

    Oct 7, 2026Krish BakshiLLM Code Generation3D Scene Generation

  16. Building Rome from a Single Image

    Oct 6, 2026Jiraphon Yenphraphai, Fang Li, Tianshuo Xu +53D Scene GenerationSingle-View 3D Reconstruction

    Applied Intuition · Purdue University · University of Illinois at Urbana-Champaign +1

  17. Controllable and Photorealistic Pedestrian Risky Motion Generation for End-to-End Driving Safety Evaluation

    Oct 5, 2026Siyuan Liu, Miao Li, Haibao Yu +4Pedestrian Behavior ModelingEnd-to-End Autonomous Driving

    School of Vehicle and Mobility, Tsinghua University · Carnegie Mellon University · State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University

  18. Code2Games: Enabling Coding Agents for Gaming World Generation

    Oct 4, 2026Wei Wu, Ziyang Xu, Zeyu Zhang +2Agentic Code Generation3D Scene Generation

    School of Computer Science, Peking University · La Trobe University

  19. DistScene: Object-to-Scene Distillation for 3D Scene Generation

    Oct 3, 2026Kunming Luo, Hongyu Yan, Ken Deng +63D Scene GenerationImage-to-3D Scene Generation

    The Hong Kong University of Science and Technology · TeleAI, China Telecom

  20. EnvDreamer: Large-Scale Multimodal-to-Environment Generation for Embodied AI

    Oct 3, 2026Kabir Swain, Sijie Han, Antonio TorralbaSynthetic Data Generation for Robotics3D Scene Generation

    MIT · University of Toronto

  21. Generative Cinematographer: Composing Camera and Object Motion in 3D

    Oct 1, 2026Jiahan Zhang, Chaohao Yang, Namitha Guruprasad +4Camera-Conditioned Video GenerationCamera-Controlled Video Generation

    Johns Hopkins University

  22. SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RL

    Oct 1, 2026Hyeonmin Lee, Zheng Wei, Kyungmin Kwon +3Human-in-the-Loop AI3D Scene Generation

    Sungkyunkwan University · The Hong Kong University of Science and Technology

  23. Task-Adaptive Grounded 3D-Programmers Using 2D VLMs

    Oct 1, 2026Arman Raayatsanati, Sombit Dey, Anna-Maria Halacheva +33D Scene Generation3D Scene Understanding

    TextQL · INSAIT, Sofia University “St. Kliment Ohridski”

  24. SPOON: Towards Coherent Compositional 3D Scene Generation from Uncalibrated Multi-view Images

    Sep 30, 2026Guibiao Liao, Mochu Xiang, Heng Li +6Object Pose Estimation3D Scene Generation

    The University of Hong Kong · Shenzhen Loop Area Institute · TranscEngram +2

  25. Audible World Models: Spatially Aware Sound Generation for 3D Worlds

    Sep 29, 2026Duowen Chen, Jinjin He, Gouthaman KV +2Sound Effects Generation3D Scene Generation

    Georgia Institute of Technology · Dolby Laboratories

  26. Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes

    Sep 29, 2026Xiaokang Ye, Siddhant Hitesh Mantri, Zimeng Chen +6AI Coding Agents3D Scene Editing

    UC San Diego · UC Berkeley

  27. WorldWeave: Growing Persistent Geometric Worlds for Video Generation

    Sep 28, 2026Yifan Huang, Lifan Jiang, Qingyue Hao +5Memory-Augmented Video Generation3D Scene Generation

    Zhejiang University · Daerwen AI

  28. WorldAgent: Verification-Guided Agentic Physical World Construction

    Sep 27, 2026Caoliwen Wang, Mengdi Wang, Yige Chen +133D Scene GenerationFunctional 3D Scene Generation

    University of British Columbia · VAST · Zhejiang University +3

  29. Fysiverse-3D-Vision Technical Report: Generating Executable 3D Worlds from Images through Unified Spatial Reasoning

    Sep 22, 2026Dingkang Yang, Yizhou Liu, Wendong Cheng +53D Scene GenerationFunctional 3D Scene Generation

    Physical Superintelligence Lab, Fysics AI · Multimedia Laboratory (MMLab), The Chinese University of Hong Kong · College of Intelligent Robotics and Advanced Manufacturing, Fudan University

  30. AgriGen: Large-Scale Scene Generation Framework for Photorealistic Agricultural Robotics Simulation

    Sep 22, 2026Utkarsh Bajpai, Serge Tleiji, Cédric Pradalier +1Robotics SimulationSynthetic Data Generation for Robotics

    Georgia Tech-CNRS IRL2958, France · Notre Dame University, Lebanon; work done during internship at 1