3D Scene Understanding

Ask about this topic

Overview

3D scene understanding studies how to recover the geometry, objects, relationships, and often motion of environments from images, depth, and LiDAR. Common approaches include point-cloud and voxel networks, occupancy prediction, Gaussian splatting, and 3D scene graphs, increasingly combined with vision-language models for grounding and reasoning. These representations support tasks such as robot navigation and manipulation, autonomous driving, scene reconstruction, and open-vocabulary object retrieval. A central challenge is making systems both spatially reliable and efficient in large, changing scenes, while benchmarks continue to expose gaps in metric reasoning, cross-view consistency, and generalization.

Latest papers

CardsList

295 papers34 papers in the last four weeks, up 17% on the four weeks before. 0.2% of all new papers.

  1. GenMatter: Perceiving Physical Objects with Generative Matter Models

    Apr 24, 2026Eric Li, Arijit Dasgupta, Yoni Friedman +5Video Object SegmentationMotion Segmentation

    1MIT CSAIL · 2MIT BCS

  2. IMU4D: 4D Human-Object Understanding from Wearable IMUs

    Apr 23, 2026Hao-Yu Hsu, Tianhang Cheng, Jing Wen +24D ReconstructionWearable Sensing

    University of Illinois at Urbana-Champaign

  3. Volume Transformer: Revisiting Vanilla Transformers for 3D Scene Understanding

    Apr 21, 2026Kadir Yilmaz, Adrian Kruse, Tristan Höfer +23D ViTs3D Semantic Segmentation

    RWTH Aachen University · Eindhoven University of Technology

  4. Feasibility of Indoor Frame-Wise Lidar Semantic Segmentation via Distillation from Visual Foundation Model

    Apr 20, 2026Haiyang Wu, Juan J. Gonzales Torres, George Vosselman +1Cross-Modal Knowledge DistillationVision Foundation Model Adaptation

    Department of Earth Observation Science, Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, 7522 NB Enschede, The Netherlands

  5. Geometry-Guided 3D Visual Token Pruning for Video-Language Models

    Apr 20, 2026Han Li, Zehao Huang, Jiahui Fu +2Visual Token PruningVisual Token Compression

    Beihang University · Zhongguancun Academy

  6. PLAF: Pixel-wise Language-Aligned Feature Extraction for Efficient 3D Scene Understanding

    Apr 17, 2026Junjie Wen, Junlin He, Fei Ma +13D Scene Representation3D Representation Learning

    Pengcheng Laboratory, Shenzhen, 518108, China · The School of Intelligent Engineering, Sun Yat-Sen University-ShenzhenCampus, Shenzhen, Guangdong, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China.

  7. Multi-modal panoramic 3D outdoor datasets for place categorization

    Apr 14, 2026Hojung Jung, Yuki Oto, Oscar M. Mozos +23D Scene UnderstandingMultimodal Classification

    Graduate School of Information Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japan · Polytechnic University of Cartagena (UPCT), Spain · Faculty of Information Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japan

  8. GS4City: Hierarchical Semantic Gaussian Splatting via City-Model Priors

    Apr 13, 2026Qilin Zhang, Jinyu Zhu, Olaf Wysocki +23D Scene Representation3D Scene Understanding

    Technical University of Munich (TUM) · Munich Center for Machine Learning (MCML) · Karlsruhe Institute of Technology (KIT) +1

  9. GTASA: Ground Truth Annotations for Spatiotemporal Analysis, Evaluation and Training of Video Models

    Apr 12, 2026Nicolae Cudlenco, Mihai Masala, Marius LeordeanuSynthetic Data GenerationVideo Representation Learning

    Institute of Mathematics of the Romanian Academy, Bucharest, Romania · Büchi Labortechnik AG, Flawil, Switzerland · National University of Science and Technology Politehnica Bucharest, Romania

  10. MAG-3D: Multi-Agent Grounded Reasoning for 3D Understanding

    Apr 10, 2026Henry Zheng, Chenyue Fang, Rui Huang +33D Spatial Reasoning3D Scene Understanding

    Tsinghua University · Pico, ByteDance

  11. OpenSpatial: A Principled Data Engine for Empowering Spatial Intelligence

    Apr 8, 2026Jianhui Liu, Haoze Sun, Wenbo Li +113D Spatial ReasoningSynthetic Data Generation

    Joy Future Academy · The University of Hong Kong · Monash University

  12. Sparsity-Aware Voxel Attention and Foreground Modulation for 3D Semantic Scene Completion

    Apr 7, 2026Yu Xue, Longjun Gao, Yuanqi Su +23D Scene Completion3D Scene Understanding

    Xi’an Jiaotong University, China

  13. Let Geometry GUIDE: Layer-wise Unrolling of Geometric Priors in Multimodal LLMs

    Apr 7, 2026Chongyu Wang, Ting Huang, Chunyu Sun +3Multimodal Large Language Models3D VLMs

    Xi’an Jiaotong University · Peking University · Institute of Automation, Chinese Academy of Sciences +1

  14. Indoor Asset Detection in Large Scale 360° Drone-Captured Imagery via 3D Gaussian Splatting

    Apr 7, 2026Monica Tang, Avideh Zakhor3D Object DetectionObject Detection

    UC Berkeley

  15. PointTPA: Dynamic Network Parameter Adaptation for 3D Scene Understanding

    Apr 6, 2026Siyuan Liu, Chaoqun Zheng, Xin Zhou +33D Scene UnderstandingTest-Time Adaptation

    Huazhong University of Science and Technology

  16. FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning

    Apr 4, 2026Zhengyu Fu, René Zurbrügg, Kaixian Qu +43D SGG3D Scene Understanding

    ETH Zürich · Microsoft · University of Bonn & Lamarr Institute

  17. RGB-Pointmap Pretraining for Unified 3D Scene Understanding

    Apr 2, 2026Ye Mao, Weixun Luo, Ranran Huang +23D Foundation ModelsMulti-View Learning

    Imperial College London

  18. Lifting Unlabeled Internet-level Data for 3D Scene Understanding

    Apr 2, 2026Yixin Chen, Yaowei Zhang, Huangyue Yu +93D Scene Understanding

    State Key Laboratory of General Artificial Intelligence, BIGAI · Beijing University of Posts and Telecommunications · Peking University +2

  19. A3R: Agentic Affordance Reasoning via Cross-Dimensional Evidence in 3D Gaussian Scenes

    Apr 2, 2026Di Li, Jie Feng, Guanbin Li +43D Scene Understanding3D Visual Grounding

    Xidian University, China · Sun Yat-sen University, China · Qinghai Normal University, China

  20. LESV: Language Embedded Sparse Voxel Fusion for Open-Vocabulary 3D Scene Understanding

    Apr 1, 2026Fusang Wang, Nathan Piasco, Moussab Bennehar +3Point Cloud Understanding3D Scene Understanding

    Huawei Noah’s Ark Lab · CAOR, Mines Paris-PSL, France

  21. SceneTeract: Probing and Improving Agent-Aware Activity Reasoning in 3D Indoor Scenes

    Mar 31, 2026Léopold Maillard, Francis Engelmann, Tom Durand +43D Spatial ReasoningEmbodied AI

    Ecole Polytechnique · Dassault Syst`emes · Stanford University +1

  22. TreeGaussian: Tree-Guided Cascaded Contrastive Learning for Hierarchical Consistent 3D Gaussian Scene Segmentation and Understanding

    Mar 31, 2026Jingbin You, Zehao Li, Hao Jiang +83D Semantic Segmentation3D Scene Understanding

    Institute of Computing Technology, Chinese Academy of Sciences, ICT · University of Chinese Academy of Sciences, UCAS · Beihang University, BUAA

  23. Pandora: Articulated 3D Scene Graphs from Egocentric Vision

    Mar 30, 2026Alan Yu, Yun Chang, Christopher Xie +13D Scene Representation3D Scene Understanding

    Laboratory for Information and Decision Systems, Massachusetts Institute of Technology · Meta Reality Labs

  24. Industrial3D: A Water-Treatment TLS Point Cloud Dataset and Cross-Paradigm Benchmark for MEP Scene Understanding

    Mar 30, 2026Chao Yin, Hongzhe Yue, Qing Han +83D Semantic SegmentationPoint Cloud Understanding

  25. AffordMatcher: Affordance Learning in 3D Scenes from Visual Signifiers

    Mar 30, 2026Nghia Vu, Tuong Do, Khang Nguyen +83D Scene Understanding

    AIOZ Ltd., Singapore · University of Liverpool, UK · National Tsing Hua University, Taiwan +4

  26. GS3LAM: Gaussian Semantic Splatting SLAM

    Mar 29, 2026Linfei Li, Lin Zhang, Zhong Wang +13D Semantic Mapping3DGS SLAM

    School of Software Engineering, Tongji University Shanghai, China · Department of Automation, Shanghai Jiaotong University Shanghai, China

  27. Chat-Scene++: Exploiting Context-Rich Object Identification for 3D LLM

    Mar 29, 2026Haifeng Huang, Yilun Chen, Zehan Wang +23D VLMs3D Scene Understanding

    School of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China. · Shanghai AI Laboratory, Shanghai 200032, China.

  28. SpatialStack: Layered Geometry-Language Fusion for 3D VLM Spatial Reasoning

    Mar 28, 2026Jian Zhang, Shijie Zhou, Bangya Liu +23D Spatial Reasoning3D VLMs

    XMU · UCLA · Google +2

  29. 3D-IDE: 3D Implicit Depth Emergent

    Mar 28, 2026Chushan Zhang, Ruihan Lu, Jinguang Tong +23D VLMs3D Scene Understanding

    School of Computing, Australian National University · FreiNexus · School of EECS, The University of Queensland +1

  30. GLASS: Geometry-aware Local Alignment and Structure Synchronization Network for 2D-3D Registration

    Mar 27, 2026Zhixin Cheng, Jiacheng Deng, Xinjun Li +52D-3D Image RegistrationPoint Cloud Registration

    School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China · Meituan Inc, Beijing 100000, China · Institute of Advanced Technology, University of Science and Technology of China, Hefei 230027, China