3D Scene Generation

Ask about this topic

Overview

3D scene generation studies how to turn prompts and visual observations into environments with coherent layouts, geometry, appearance, and sometimes motion or interaction. Methods range from procedural rules and language-model agents to diffusion, flow-matching, and autoregressive models that generate meshes, Gaussian splats, voxels, or occupancy representations. Work targets indoor design, large-scale worlds, driving scenes, and simulation for robotics, using constraints, rendering feedback, and physics checks to improve control and usability. A central challenge is balancing visual realism and scalable generation with consistent geometry, physical plausibility, editability, and reliable evaluation.

Latest papers

CardsList

191 papers25 papers in the last four weeks, up 39% on the four weeks before. 0.1% of all new papers.

  1. Pose-Aware Diffusion for 3D Generation

    May 1, 2026Zihan Zhou, Luxi Chen, Jingzhi Zhou +43D Asset Generation3D Scene Generation

    Gaoling School of AI, Renmin University of China · VCIP, School of Computer Science, Nankai University · THU-Bosch MLCenter, Tsinghua University +1

  2. HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation

    Apr 30, 2026Xin Zhou, Dingkang Liang, Xiwu Chen +4World Model Learning3D Scene Generation

    Huazhong University of Science and Technology · Mach Drive · the University of Hong Kong

  3. SpatialGrammar: A Domain-Specific Language for LLM-Based 3D Indoor Scene Generation

    Apr 30, 2026Song Tang, Kaiyong Zhao, Yuliang Li +53D Scene Generation3D Scene Representation

    The Hong Kong University of Science and Technology (Guangzhou) · XGRIDS · Harbin Institute of Technology (Shenzhen)

  4. CasLayout: Cascaded 3D Layout Diffusion for Indoor Scene Synthesis with Implicit Relation Modeling

    Apr 30, 2026Yingrui Wu, Youkang Kong, Mingyang Zhao +33D Spatial Reasoning3D Scene Generation

    MAIS, Institute of Automation, Chinese Academy of Sciences and School of Artificial Intelligence, University of Chinese Academy of Sciences, China · Tsinghua University, China · SKLMS, Academy of Mathematics and Systems Science, Chinese Academy of Sciences and University of Chinese Academy of Sciences, China +1

  5. 3D Generation for Embodied AI and Robotic Simulation: A Survey

    Apr 29, 2026Tianwei Ye, Yifan Mao, Minwen Liao +63D Asset GenerationRobotics Simulation

    the Hong Kong University of Science and Technology · Wuhan University · Harbin Institute of Technology +2

  6. Cutscene Agent: An LLM Agent Framework for Automated 3D Cutscene Generation

    Apr 28, 2026Lanshan He, Haozhou Pang, Qi Gan +12LLM Agent Orchestration3D Scene Generation

    Kuaishou GameMind Lab

  7. Seed3D 2.0: Advancing High-Fidelity Simulation-Ready 3D Content Generation

    Apr 22, 2026Diandian Gu, Jing Lin, Gaohong Liu +253D Mesh Generation3D Asset Generation

    ByteDance Seed

  8. Dream-Cubed: Controllable Generative Modeling in Minecraft by Training on Billions of Cubes

    Apr 22, 2026Tim Merino, Sam Earle, Ryunosuke Iwai +23D Scene Generation3D Shape Generation

    New York University · Sakana AI

  9. SceneOrchestra: Efficient Agentic 3D Scene Synthesis via Full Tool-Call Trajectory Generation

    Apr 21, 2026Yun He, Kelin Yu, Matthias ZwickerLLM Agent OrchestrationTool-Use Planning

    University of Maryland, College Park

  10. MetaEarth3D: Unlocking World-scale 3D Generation with Spatially Scalable Generative Modeling

    Apr 19, 2026Jinqi Cao, Zhiping Yu, Baihong Lin +3Geospatial Foundation ModelsRemote Sensing

    School of Astronautics, Beihang University

  11. ScenarioControl: Vision-Language Controllable Vectorized Latent Scenario Generation

    Apr 18, 2026Lili Gao, Yanbo Xu, William Koch +8VLMs for Autonomous Driving3D Scene Generation

    1Torc Robotics · 2Princeton University · 3Mila

  12. Repurposing 3D Generative Model for Autoregressive Layout Generation

    Apr 17, 2026Haoran Feng, Yifan Niu, Zehuan Huang +43D Scene GenerationAutoregressive Generation

    School of Software, Beihang University · 2Tsinghua University · University of Hong Kong +2

  13. From Seeing to Simulating: Generative High-Fidelity Simulation with Digital Cousins for Generalizable Robot Learning and Evaluation

    Apr 17, 2026Jasper Lu, Zhenhao Shen, Yuanfei Wang +8Robotics SimulationSynthetic Data Generation for Robotics

    1Peking University · 2Lightwheel

  14. Co-generation of Layout and Shape from Text via Autoregressive 3D Diffusion

    Apr 17, 2026Zhenggang Tang, Yuehao Wang, Yuchen Fan +9Autoregressive Diffusion3D Scene Generation

    Meta Reality Labs · University of Illinois Urbana-Champaign · University of Texas at Austin

  15. Geometrically Consistent Multi-View Scene Generation from Freehand Sketches

    Apr 15, 2026Ahmed Bourouis, Savas Ozkan, Andrea Maracani +23D Scene GenerationMulti-View Consistency

    Samsung Research, UK · SketchX, University of Surrey, UK

  16. HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds

    Apr 15, 2026Team HY-World, Chenjie Cao, Xuhui Zuo +42World Models3D Scene Generation

  17. Rethinking Image-to-3D Generation with Sparse Queries: Efficiency, Capacity, and Input-View Bias

    Apr 15, 2026Zhiyuan Xu, Jiuming Liu, Yuxin Chen +3Image-to-3D Scene Generation3D Scene Generation

    UC Berkeley · University of Cambridge · UT Austin

  18. Lyra 2.0: Explorable Generative 3D Worlds

    Apr 14, 2026Tianchang Shen, Sherwin Bahmani, Kai He +123D Scene Generation3D-Aware Video Generation

    NVIDIA

  19. SceneCritic: A Symbolic Evaluator for 3D Indoor Scene Synthesis

    Apr 14, 2026Kathakoli Sengupta, Kai Ao, Paola Cascante-Bonilla3D Scene Generation3D Scene Layout Generation

    Stony Brook University

  20. Pair2Scene: Learning Local Object Relations for Procedural Scene Generation

    Apr 13, 2026Xingjian Ran, Shujie Zhang, Weipeng Zhong +23D Scene Generation3D Scene Layout Generation

    The University of Hong Kong · Tsinghua University · Shanghai Jiao Tong University +1

  21. Any 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale

    Apr 13, 2026Dongxu Wei, Qi Xu, Zhiqi Li +63D Scene Generation3D Diffusion Models

    Westlake University · Afari Intelligent Drive · Zhejiang University

  22. ReplicateAnyScene: Zero-Shot Video-to-3D Composition via Textual-Visual-Spatial Alignment

    Apr 12, 2026Mingyu Dong, Chong Xia, Mingyuan Jia +43D Reconstruction3D Scene Generation

    Tsinghua University · Zhejiang University

  23. HOG-Layout: Hierarchical 3D Scene Generation, Optimization and Editing via Vision-Language Models

    Apr 12, 2026Haiyan Jiang, Deyu Zhang, Dongdong Weng +23D Scene Generation3D Scene Layout Generation

    The Hong Kong Polytechnic University · Beijing Institute of Technology

  24. PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit--Explicit Optimization

    Apr 11, 2026Dongli Wu, Jingyu Hu, Ka-Hei Hui +4Image-to-3D Scene Generation3D Scene Generation

    1Lingnan University · 2The Hong Kong University of Science and Technology (Guangzhou) · 3The Chinese University of Hong Kong +2

  25. Genie Sim PanoRecon: Fast Immersive Scene Generation from Single-View Panorama

    Apr 8, 2026Zhijun Li, Yongxin Su, Di Yang +43D Scene GenerationImage-to-3D Scene Generation

  26. SEM-ROVER: Semantic Voxel-Guided Diffusion for Large-Scale Driving Scene Generation

    Apr 7, 2026Hiba Dahmani, Nathan Piasco, Moussab Bennehar +53D Scene GenerationAutonomous Driving Scenario Generation

    Noah’s Ark, Huawei Paris Research Center, France · COSYS, Gustave Eiffel University, France · LASTIG, IGN-ENSG, Gustave Eiffel University, France

  27. VoxelCodeBench: Benchmarking 3D World Modeling Through Code Generation

    Apr 2, 2026Yan Zheng, Florian BordesCode Generation Benchmarks3D Spatial Reasoning

    FAIR at Meta · The University of Texas at Austin

  28. GardenDesigner: Encoding Aesthetic Principles into Jiangnan Garden Construction via a Chain of Agents

    Apr 2, 2026Mengtian Li, Fan Yang, Ruixue Xiong +33D Scene Generation3D Scene Layout Generation

    Shanghai University · Shanghai Engineering Research Center of Motion Picture Special Effects · The Hong Kong University of Science and Technology (Guangzhou)

  29. LivingWorld: Interactive 4D World Generation with Environmental Dynamics

    Apr 2, 2026Hyeongju Mun, In-Hwan Jin, Sohyeong Kim +13D Scene GenerationInteractive World Models

    Department of Electrical and Electronic Engineering, Pusan National University

  30. RecGen3D: Reconstruction-Guided 3D Generation in a Shared Canonical Space

    Apr 1, 2026Zhisheng Huang, Jiahao Chen, Cheng Lin +103D Asset Generation3D Scene Generation

    Texas A&M University, College Station, USA · Macau University of Science and Technology, Macau, Macau · HKUST, Hong Kong, Hong Kong +2