3D Scene Understanding

Ask about this topic

Overview

3D scene understanding studies how to recover the geometry, objects, relationships, and often motion of environments from images, depth, and LiDAR. Common approaches include point-cloud and voxel networks, occupancy prediction, Gaussian splatting, and 3D scene graphs, increasingly combined with vision-language models for grounding and reasoning. These representations support tasks such as robot navigation and manipulation, autonomous driving, scene reconstruction, and open-vocabulary object retrieval. A central challenge is making systems both spatially reliable and efficient in large, changing scenes, while benchmarks continue to expose gaps in metric reasoning, cross-view consistency, and generalization.

Latest papers

CardsList

295 papers34 papers in the last four weeks, up 17% on the four weeks before. 0.2% of all new papers.

  1. TopoField: Technical Report for ECCV 2026 SceneFun3D Challenge

    Oct 27, 2026Zhiwei Chen, Zhiheng Fu, Zixu Li +33D Point Cloud Segmentation3D Instance Segmentation

    Shandong University

  2. 3D-SceneQ: Empowering 3D LLM With Query-Guided Adaptive Pruning and Multi-Modal Feature Enhancement

    Oct 27, 2026Yongqi Shan, Lu Zhang, Jiazuo Yu +23D Scene Understanding3D VLMs

    School of Artificial Intelligence, Dalian University of Technology, Dalian 116081, China. · School of Information and Communication Engineering, Dalian University of Technology, Dalian 116081, China.

  3. OVM-Occ: Open-Vocabulary Monocular Occupancy Prediction via Geometry-Aware Vision-Language Distillation

    Oct 26, 2026Shuyang Yu, Haibo Qin, Shaohan Hu +43D Occupancy Prediction3D Scene Understanding

    Zhejiang University. · AgiBot.

  4. RelateGS: Reliable Semantic and Relational Evidence for 3D Scene Understanding with Gaussian Splatting

    Oct 24, 2026Seongjun Choi, Sangyoun Lee3D Scene Understanding3D Visual Grounding

    Yonsei University, School of Electrical and Electronic Engineering

  5. Point-Level Refinable Semantic Mapping for Spatial Understanding

    Oct 22, 2026Taimeng Fu, Yi Du, Zitong Zhan +43D Semantic MappingSemantic Mapping

    Spatial AI & Robotics Lab, Department of Computer Science and Engineering, University at Buffalo, USA.

  6. GeoMem-VLA: Internalizing 3D Scene Understanding into Temporal Memory for Vision-Language-Action Models

    Oct 21, 2026Qinfan Zhang, Yifan Luo, Haoyuan Shi +6Vision-Language-Action ModelsVision-Based Robot Control

    BUAA · X-Humanoid · RUC +3

  7. OMAR: Open-World Multi-Attribute Reasoning for Scene-Conditioned Shelf Organization

    Oct 21, 2026Benedikt Kreis, Rohit Menon, Benno Wingender +3Robotic ManipulationMultimodal Spatial Reasoning

    Humanoid Robots Lab, University of Bonn, Germany. · Center for Robotics, University of Bonn, Germany. · Lamarr Institute for ML and AI, Bonn, Germany. +1

  8. CoGaussian: Open-Vocabulary Gaussian-based Collaborative 3D Scene Understanding

    Oct 20, 2026Om Shivam Verma, Shaya Farahmand, Lingjun Zhao +13D Scene Understanding3D Occupancy Prediction

    Lunar Lab, Georgia Institute of Technology, Atlanta.

  9. Learning from Automatic 3D Gaussian Splatting Edits for Online Bayesian Scene Change Detection

    Oct 20, 2026Changrui Zhu, Simon Julier3D Scene Change DetectionChange-Point Detection

    University College London

  10. PanoGraphGS: Unifying Panoptic Segmentation and Hierarchical 3D Scene Graphs with Gaussian Splatting

    Oct 20, 2026Jinghan Gao, Siyu Wang, Dongyu Feng +23D SGG3D Scene Understanding

    Zhejiang Humanoid Robot Innovation Center, Ningbo, China.

  11. Where Do the Gaussians Go? Foreground Attraction for Camera-Based Weakly Supervised 3D Occupancy

    Oct 19, 2026Jeagon Song, Jhonghyun An3D Occupancy Prediction3D Scene Understanding

    School of Computing, Gachon University, Seongnam 13120, Republic of Korea.

  12. From Views to Worlds: Active Exploration over 3D Worlds for Vision-Language Models

    Oct 17, 2026Qijian Tian, Jiayu Ying, Ke Fan +2Visual Spatial Reasoning3D Scene Understanding

    Shanghai Jiao Tong University, Shanghai, China · East China Normal University, Shanghai, China

  13. O-SCD++: Replay-Free Online Scene Change Detection via Recursive Evidence Fusion

    Oct 17, 2026Chamuditha Jayanga Galappaththige, Jason Lai, Lloyd Windrim +43D Scene Change Detection3D Scene Understanding

    QUT Centre for Robotics, Queensland University of Technology, Australia · Australian Research Council Research Hub in Intelligent Robotic Systems for Real-Time Asset Management (ARIAM) · Australian Centre for Robotics, The University of Sydney, Australia +1

  14. Learning to Discriminate Scene Structures Makes Self-Supervised Depth Learning Scalable

    Oct 16, 2026Mengtan Zhang, Yuchuan Cui, Yu Ma +3Self-Supervised Monocular Depth EstimationDomain Generalization

    Tongji University

  15. Grounding Generative Verification in 3D Content

    Oct 15, 2026Shuanglin Li, Jiaying Lin, Shuquan Ye3D Scene Understanding

    Southeast University · Shenzhen Loop Area Institute · The Hong Kong University of Science and Technology

  16. FALCON: FPGA-Accelerated Latency-Certifiable Occupancy Networks for Camera-Based Autonomous Driving

    Oct 15, 2026Haohua Que, Mingkai Liu, Handong YaoAutonomous Driving PerceptionFPGA-Based Neural Network Acceleration

    College of Engineering, University of Georgia, Athens, Georgia, USA

  17. DeSpatial: The devil of Agentic Spatial Reasoning Lies in 3D Itself

    Oct 14, 2026Yifei Liu, Xiaolong Zhou, Ziyang Gong +83D Spatial Reasoning3D Scene Understanding

    Shanghai Artificial Intelligence Laboratory · Beihang University · Shanghai Jiaotong University +3

  18. Holi-Spatial 360: Generative Reconstruction from Sparse Panoramas for Spatial Rendering and Intelligence

    Oct 14, 2026Yuanyuan Gao, Zhirui Liu, Yuning Gong +9Indoor Scene ReconstructionPanoramic 3D Reconstruction

    The Hong Kong University of Science and Technology · ShanghaiTech University · Tencent +4

  19. LAMP: 3D Source-Aware Lighting Estimation at Multiple Points

    Oct 13, 2026Yanqiao Wang, Jin SunInverse Rendering3D Scene Understanding

    School of Computing, University of Georgia, Athens, GA 30602, USA

  20. GHOST: Geometry-Constrained Hybrid Diffusion for Open-Vocabulary 3D Scene Graph Generation

    Oct 12, 2026Changqun Feng, Wangxiandi Yin, Xin Hu +43D SGG3D Scene Understanding

    University of Electronic Science and Technology of China Chengdu, Sichuan, China · Jilin University Changchun, Jilin, China

  21. FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning

    Oct 12, 2026Junhyeong Go, Mingyu Jung, Woobin Im +1Scene Flow EstimationVision-Language-Action Models

    Ajou University · Samsung Electronics

  22. CoCam4D: Geometry-Aware Cooperative 4D Perception for Camera-Only Autonomous Driving

    Oct 8, 2026Soham Pahari, Sudip Das, Arindam Das +1Autonomous Driving PerceptionCooperative Perception

    University of Petroleum and Energy Studies, Dehradun · Indian Statistical Institute, Kolkata · anSWer India, Valeo Brain

  23. AffordDrive3D: Affordance-Aware World-Action Modeling with Spatial Understanding

    Oct 8, 2026Tianhui Cai, Xinglong Sun, Chao Fang +6Autonomous DrivingWorld Models for Autonomous Driving

    University of California, Los Angeles · NVIDIA · 42dot by Hyundai +1

  24. ECHO: Embodied Camera Observations of Human Object Carrying

    Oct 7, 2026Xuefei Sun, Lorin Achey, Kali Hamilton +4Human-Object Interaction3D Scene Understanding

    Intelligent Robotics Laboratory, Department of Computer Science, University of Colorado Boulder, Boulder, CO 80309, USA · Lockheed Martin, Advanced Technology Laboratories, USA · Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA

  25. M3SunAgent: Monocular 3D Spatial Understanding Agent for Metric Depth Estimation and 3D Visual Grounding

    Oct 6, 2026Jinsong Zhang, Kejun Wu, Ming Zhu +3Monocular Depth EstimationTool-Using Agents

    School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China · College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China · Institute of Advanced Intelligence and Computation, Agency for Science, Technology and Research (A*STAR), Singapore

  26. GS-Pool: Object-Level Change Detection in 3D Gaussian Splatting

    Oct 5, 2026Boaz Keren-Gil, James Gain, Patrick Marais3D Gaussian Splatting3D Scene Change Detection

    University of Cape Town

  27. ArticuTable: Generating Instance-Level Interactive Rigid-Articulated 3D Tabletop Scenes from a Single Image

    Oct 4, 2026Kai Lv, Yibo Yin, Lijun Guo +3Articulated Object Reconstruction3D Reconstruction

    School of Computer Science, Wuhan University · Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Switzerland · Department of Computer Science and Engineering, University of North Texas +1

  28. ExStereo: Lifting 2D Vision-Language-Action Models to 3D with Explicit Stereo Representations

    Oct 3, 2026I-Chun Arthur Liu, Jason Chen, Gaurav S. Sukhatme +1Vision-Language-Action ModelsRobotic Manipulation

    Viterbi School of Engineering, University of Southern California

  29. ForeAct3D: Policy-Grounded Future World Modeling for VLA Policies

    Oct 3, 2026Zhe Tao, Feiran Wang, Gaowen Liu +2World Models for RoboticsVision-Language-Action Models

    University of Illinois Urbana-Champaign · University of Illinois Chicago · Cisco

  30. SelectOccFlow: Selective Spatiotemporal Aggregation for 3D Occupancy and Scene Flow Prediction

    Oct 3, 2026Yuhang Wang, Kai Luo, Yuanfan Zheng +1Scene Flow Estimation3D Occupancy Prediction

    School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China · National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China