3D Scene Understanding

Ask about this topic

Overview

3D scene understanding studies how to recover the geometry, objects, relationships, and often motion of environments from images, depth, and LiDAR. Common approaches include point-cloud and voxel networks, occupancy prediction, Gaussian splatting, and 3D scene graphs, increasingly combined with vision-language models for grounding and reasoning. These representations support tasks such as robot navigation and manipulation, autonomous driving, scene reconstruction, and open-vocabulary object retrieval. A central challenge is making systems both spatially reliable and efficient in large, changing scenes, while benchmarks continue to expose gaps in metric reasoning, cross-view consistency, and generalization.

Latest papers

CardsList

295 papers34 papers in the last four weeks, up 17% on the four weeks before. 0.2% of all new papers.

  1. Lookalike3D: Seeing Double in 3D

    Mar 25, 2026Chandan Yeshwanth, Angela Dai3D Shape Matching3D Scene Understanding

    Technical University of Munich

  2. LightSplat: Fast and Memory-Efficient Open-Vocabulary 3D Scene Understanding in Five Seconds

    Mar 25, 2026Jaehun Bang, Jinhyeok Kim, Minji Kim +2Open-Vocabulary 3D Segmentation3D Scene Understanding

    AIGS, UNIST · GSAI, POSTECH

  3. UniFunc3D: Unified Active Spatial-Temporal Grounding for 3D Affordance Segmentation

    Mar 24, 2026Jiaying Lin, Dan Xu3D Scene Understanding3D Visual Grounding

    The Hong Kong University of Science and Technology

  4. 3DCity-LLM: Empowering Multi-modality Large Language Models for 3D City-scale Perception and Understanding

    Mar 24, 2026Yiping Chen, Jinpeng Li, Wenyu Ke +63D VLMs3D Scene Understanding

    School of Geospatial Engineering and Science, Sun Yat-sen University, Zhuhai, 519082, China. · College of Electronic Science, National University of Defense Technology, Changsha, 410000, China. · Department of Civil and Environmental Engineering, Norwegian University of Science and Technology, Trondheim, Norway. +1

  5. Predictive Photometric Uncertainty in Gaussian Splatting for Novel View Synthesis

    Mar 24, 2026Chamuditha Jayanga Galappaththige, Thomas Gottwald, Peter Stehr +43D Scene Change Detection3D Gaussian Splatting

    QUT Centre for Robotics, Australia · ARIAM Hub, Australia · University of Wuppertal, Germany +1

  6. SpatialBoost: Enhancing Visual Representation through Language-Guided Reasoning

    Mar 23, 2026Byungwoo Jeon, Dongyoung Kim, Huiwon Jang +2Visual Representation Learning3D Spatial Reasoning

    KAIST · RLWRLD · NAVER Cloud

  7. GaussianSSC: Triplane-Guided Directional Gaussian Fields for 3D Semantic Completion

    Mar 23, 2026Ruiqi Xian, Jing Liang, He Yin +2Tri-Plane Representations3D Scene Completion

  8. KLDrive: Fine-Grained 3D Scene Reasoning for Autonomous Driving based on Knowledge Graph

    Mar 22, 2026Ye Tian, Jingyi Zhang, Zihao Wang +4Question AnsweringLLM Reasoning with Graphs

    University of California San Diego, La Jolla, California, USA · University of California Merced, Merced, California, USA

  9. Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding

    Mar 19, 2026Xianjin Wu, Dingkang Liang, Tianrui Feng +53D Spatial Reasoning3D Scene Understanding

    Huazhong University of Science and Technology, China · Baidu Inc., China

  10. DriveTok: 3D Driving Scene Tokenization for Unified Multi-View Reconstruction and Understanding

    Mar 19, 2026Dong Zhuo, Wenzhao Zheng, Sicheng Zuo +43D Scene Representation3D Scene Understanding

    Tsinghua University · Yinwang Intelligent Technology Co. Ltd.

  11. OnlinePG: Online Open-Vocabulary Panoptic Mapping with 3D Gaussian Splatting

    Mar 19, 2026Hongjia Zhai, Qi Zhang, Xiaokun Pan +53D Scene UnderstandingPanoptic Segmentation

    State Key Lab of CAD & CG, Zhejiang University · VIVO BlueImage Lab · HKUST

  12. NymeriaPlus: Enriching Nymeria Dataset with Additional Annotations and Data

    Mar 19, 2026Daniel DeTone, Federica Bogo, Eric-Tuan Le +7Egocentric Vision3D Scene Understanding

    Meta Reality Labs Research

  13. Loc3R-VLM: Language-based Localization and 3D Reasoning with Vision-Language Models

    Mar 18, 2026Kevin Qu, Haozhe Qi, Mihai Dusmanu +33D VLMs3D Spatial Reasoning

    Microsoft Spatial AI Lab · ETH Zurich · EPFL

  14. Feeling the Space: Egomotion-Aware Video Representation for Efficient and Accurate 3D Scene Understanding

    Mar 18, 2026Shuyao Shi, Kang G. Shin3D Scene Understanding3D Spatial Reasoning

    Department of Computer Science, University of Michigan, Ann Arbor, MI, USA

  15. S-VGGT: Structure-Aware Subscene Decomposition for Scalable 3D Foundation Models

    Mar 18, 2026Xinze Li, Pengxu Chen, Yiyuan Wang +23D Foundation ModelsEfficient Attention

    Beijing Normal-Hong Kong Baptist University · Jilin University · Hong Kong Baptist University +1

  16. OmniVLN: Omnidirectional 3D Perception and Token-Efficient LLM Reasoning for Visual-Language Navigation across Air and Ground Platforms

    Mar 18, 2026Zhongyuang Liu, Min He, Shaonan Yu +5Vision-Language NavigationRobot Navigation

    CertaintyX. · the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798. · the Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA. +1

  17. x2x^2-Fusion: Cross-Modality and Cross-Dimension Flow Estimation in Event Edge Space

    Mar 17, 2026Ruishan Guo, Ciyu Ruan, Haoyang Wang +3Scene Flow EstimationOptical Flow

    Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology · The University of Hong Kong

  18. OGScene3D: Incremental Open-Vocabulary 3D Gaussian Scene Graph Mapping for Scene Understanding

    Mar 17, 2026Siting Zhu, Ziyun Lu, Guangming Wang +53D Scene Understanding3D SGG

    Shanghai Jiao Tong University, China · University of Cambridge, England · University of Technology Nuremberg, Germany +2

  19. RieMind: Geometry-Grounded Spatial Agent for Scene Understanding

    Mar 16, 2026Fernando Ropero, Erkin Turkoz, Daniel Matos +63D Spatial Reasoning3D Scene Understanding

    Riemann Lab, Huawei Technologies

  20. R3DP: Real-Time 3D-Aware Policy for Embodied Manipulation

    Mar 15, 2026Yuhao Zhang, Wanxi Dong, Yue Shi +13Multi-View Feature Fusion3D Scene Understanding

    Shanghai Jiao Tong University · D-Robotics · Southern University of Science and Technology +5

  21. S2GS: Streaming Semantic Gaussian Splatting for Online Scene Understanding and Reconstruction

    Mar 15, 2026Renhe Zhang, Yuyang Tan, Jingyu Gong +4Streaming 3D Reconstruction3D Scene Understanding

    East China Normal University · Shanghai Artificial Intelligence Laboratory

  22. Learning Geometric and Photometric Features from Panoramic LiDAR Scans for Outdoor Place Categorization

    Mar 13, 2026Kazuto Nakashima, Hojung Jung, Yuki Oto +3Visual Place Recognition3D Scene Understanding

    Graduate School of Information Science and Electrical Engineering, Kyushu University, Fukuoka, Japan · Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA · Faculty of Information Science and Electrical Engineering, Kyushu University, Fukuoka, Japan +1

  23. LR-SGS: Robust LiDAR-Reflectance-Guided Salient Gaussian Splatting for Self-Driving Scene Reconstruction

    Mar 13, 2026ZY Chen, F Zhu, H Zhu +43D Gaussian SplattingAutonomous Driving

  24. O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Embodied Intelligent Robotics

    Mar 12, 2026Mengfei Duan, Hao Shi, Fei Teng +43D Occupancy Prediction3D Scene Understanding

    School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha, China · Ant Group, Hangzhou, China · State Key Laboratory of Extreme Photonics and Instrumentation, Zhejiang University, Hangzhou, China

  25. PanoAffordanceNet: Towards Holistic Affordance Grounding in 360° Indoor Environments

    Mar 10, 2026Guoliang Zhu, Wanjun Jia, Caoyang Shao +3Embodied AI3D Scene Understanding

    School of Artificial Intelligence and Robotics, Hunan University, Changsha 410012, China · National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha 410082, China

  26. X-GS: An Extensible Framework for Perceiving and Thinking with 3D Gaussian Splatting

    Mar 10, 2026Yueen Ma, Zenglin Xu, Irwin King3D Gaussian Splatting3DGS SLAM

    The Chinese University of Hong Kong · Shanghai Academy of AI for Science · Fudan University

  27. UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsing

    Mar 9, 2026Jiaxi Zhang, Yunheng Wang, Wei Lu +83D Visual Grounding3D VLMs

    The Hong Kong University of Science and Technology (Guangzhou) · JD Explore Academy · Zhejiang University

  28. Holi-Spatial: Evolving Video Streams into Holistic 3D Spatial Intelligence

    Mar 8, 2026Yuanyuan Gao, Hao Li, Yifei Liu +143D Scene Understanding3D Spatial Reasoning

    Shanghai AI Lab · Northwestern Polytechnical University · Nanyang Technological University +8

  29. Efficient RGB-D Scene Understanding via Multi-task Adaptive Learning and Cross-dimensional Feature Guidance

    Mar 8, 2026Guodong Sun, Junjie Liu, Gaoyang Zhang +23D Scene UnderstandingSemantic Segmentation

    School of Mechanical Engineering, Hubei University of Technology, Wuhan, 430068, China · Hubei Key Laboratory of Modern Manufacturing Quality Engineering, Hubei University of Technology, Wuhan, 430068, China · Vehicle Measurement, Control and Safety Key Laboratory of Sichuan Province, Xihua University, Chengdu, 610039, China +2

  30. DeepSight: Bridging Depth Maps and Language with a Depth-Driven Multimodal Model

    Mar 6, 2026Hao Yang, Hongbo Zhang, Yanyan Zhao +13D VLMsVision-Language Models

    Harbin Institute of Technology, Harbin, China