3D Visual Grounding

Ask about this topic

Overview

Three-dimensional visual grounding studies how to locate and identify the objects or regions described by language or images, often predicting a 3D box, point, or mask. Systems combine language and visual features with point-cloud geometry, scene graphs, or multi-view RGB-D evidence; newer approaches also use vision-language models, Gaussian splatting, and tool-using agents. Benchmarks such as ScanRefer, Nr3D, and SceneFun3D test referring expressions, relational descriptions, and functional regions, while open-vocabulary methods aim to handle unseen objects and scenes. Performance is improving, but reliable grounding remains difficult when viewpoints, ambiguous language, occlusion, or metric spatial constraints matter.

Latest papers

CardsList

142 papers21 papers in the last four weeks, up 5% on the four weeks before. 0.1% of all new papers.

  1. SpaceAct: Benchmarking 3D Spatial and Temporal Reasoning via Closed-Loop Action and Active Perception

    Oct 28, 2026Chenglong Shi, Zekai Yuan, Yue Song +2Spatial Reasoning BenchmarksSpatiotemporal Reasoning

    Tencent HY

  2. 3D-SceneQ: Empowering 3D LLM With Query-Guided Adaptive Pruning and Multi-Modal Feature Enhancement

    Oct 27, 2026Yongqi Shan, Lu Zhang, Jiazuo Yu +23D Scene Understanding3D VLMs

    School of Artificial Intelligence, Dalian University of Technology, Dalian 116081, China. · School of Information and Communication Engineering, Dalian University of Technology, Dalian 116081, China.

  3. ClusterSplat: Semantic Cluster Selection for 3D Visual Grounding in Gaussian Splatting

    Oct 27, 2026Jaesung Lee, Hyeontaek Hwang, Gabriel Manalu +13D Visual Grounding3D Referring Segmentation

    School of Computing, KAIST

  4. RelateGS: Reliable Semantic and Relational Evidence for 3D Scene Understanding with Gaussian Splatting

    Oct 24, 2026Seongjun Choi, Sangyoun Lee3D Scene Understanding3D Visual Grounding

    Yonsei University, School of Electrical and Electronic Engineering

  5. GeoLift: Grounding Video Spatial Reasoning in Explicit 3D Maps

    Oct 24, 2026Xinglin Li, TingTing Long, Jingzhi Zhou +3Multimodal Spatial ReasoningVideo Reasoning

    Nankai University · Peking University · Amazon +1

  6. Gaussian Episodic Memory for Long-Term Spatiotemporal Reasoning

    Oct 23, 2026Ling-Chia Chen, Abdelrhman Werby, Kaan Aytekin +1Spatiotemporal ReasoningLong-Term Memory for Embodied Agents

    Socially Intelligent Robotics Lab, Institute for Artificial Intelligence, University of Stuttgart, Germany

  7. P3^3-VLM: A Point-based Alternative for Grounded 3D Vision-Language Models

    Oct 20, 2026Anna-Maria Halacheva, Jan-Nico Zaech, Sombit Dey +23D VLMs3D Visual Grounding

    INSAIT, Sofia University “St. Kliment Ohridski”

  8. Do 3D Large Language Models Really Use 3D Evidence?

    Oct 17, 2026Yifei Zhao, Fanyu Zhao, Han Zhang +63D VLMs3D VQA

    Fudan University · University of Science and Technology of China

  9. BridgeAfford: Bridging Query-Target Gap for Generalizable 3D Affordance Grounding

    Oct 15, 2026Yijie Su, Yakun Zhu, Yi Bin +53D Affordance Grounding3D Visual Grounding

    Tongji University · University of Electronic Science and Technology of China

  10. Urbex: Agentic Spatial Grounding for City-Scale 3D Scenes

    Oct 13, 2026Yu Wang, Rui Dai, Wei Chen +53D Visual Grounding

    Wuhan University · DreamX Team, Alibaba Group

  11. Agreement versus Softmax under Spatial Disambiguation

    Oct 9, 2026Michał Zakrzewski, Mariia Shpir, Taras Rumezhak +1Vision-Language Grounding3D Visual Grounding

    AGH University of Krakow, Krakow, Poland · SoftServe · ETH Zurich

  12. Token-Budgeted Candidate Retrieval for 3D Scene-Grounded VLMs: Category Names vs. Pairwise Geometry

    Oct 9, 2026Michał Zakrzewski, Mariia Shpir, Taras RumezhakVision-Language Retrieval3D VLMs

    AGH University of Krakow, Krakow, Poland · SoftServe · ETH Zurich

  13. PARC-Loc: Text-to-Point-Cloud Localization with Partial Assignment and Relational Consistency

    Oct 7, 2026Shengkai Ma, Zhenyu Hou, Weihua CaoMap-Based Localization3D Visual Grounding

  14. M3SunAgent: Monocular 3D Spatial Understanding Agent for Metric Depth Estimation and 3D Visual Grounding

    Oct 6, 2026Jinsong Zhang, Kejun Wu, Ming Zhu +3Monocular Depth EstimationTool-Using Agents

    School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China · College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China · Institute of Advanced Intelligence and Computation, Agency for Science, Technology and Research (A*STAR), Singapore

  15. OpenSplatGraph: From Dense Semantic Maps to Structured Scene Graphs for Open-Vocabulary Robot Perception

    Oct 6, 2026Binh Long Nguyen, Kien Nguyen, Clinton Fookes +1Language-Embedded 3DGSOpen-Vocabulary 3D Segmentation

    School of Electrical Engineering and Robotics, Queensland University of Technology (QUT), Brisbane, QLD 4000, Australia · CSIRO Robotics, CSIRO, Brisbane, QLD 4069, Australia

  16. ElasticFit: Fit-Aware 3D Object Insertion via VLM Reasoning and Generative Adaptation

    Oct 5, 2026Tzu-Hsin Hsieh, Ricardo Marroquim3D Scene Editing3D Visual Grounding

    Delft University of Technology Delft, Netherlands

  17. Lens3D: Target-Conditioned Visual Foveation for Fine-Grained 3D Understanding

    Oct 5, 2026Junming Huang, Zini Chen, Shuaiying Hou +3Active View Selection3D VQA

    Zhejiang University · LIGHTSPEED

  18. Have I Scene This Before? Spatially Grounded Conversational Memory for Complex Queries in Egocentric Assistants

    Oct 4, 2026Jiazhou Liang, Liam Gallagher, Kiko Chen +4Conversational MemoryEgocentric Video Understanding

    University of Toronto

  19. Scaling 3D Visual Grounding in Abdominal CT

    Oct 2, 2026Sam Church, Danyal Maqbool, Joshua D. Warner +4Medical Image Grounding3D Visual Grounding

    University of Wisconsin–Madison, Department of Computer Sciences, Madison, WI USA · University of Wisconsin–Madison, Department of Radiology, Madison, WI USA

  20. Task-Adaptive Grounded 3D-Programmers Using 2D VLMs

    Oct 1, 2026Arman Raayatsanati, Sombit Dey, Anna-Maria Halacheva +33D Scene Generation3D Scene Understanding

    TextQL · INSAIT, Sofia University “St. Kliment Ohridski”

  21. RelationVGGT: Visual Geometry Transformers for 3D Spatial Relation Segmentation

    Oct 1, 2026Minsu Kim, Jaesung Choe, Jiwoo Lee +23D ViTs3D Spatial Reasoning

    Yonsei University · NVIDIA

  22. HiWE: Hierarchical World Knowledge Model with Visual Keypoint Enhancement for Zero-Shot 3D Path Planning

    Sep 30, 2026Guoqing Ma, Mingqi Yuan, Chen Gao +2Zero-Shot Robotic ManipulationLanguage-Conditioned Robot Manipulation

    Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Future Technology, University of Chinese Academy of Sciences · State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, CAS +2

  23. HIGS: Hierarchical Implicit Grids for Joint Geometric and Semantic Scene Understanding

    Sep 29, 2026Hanwen Cao, Wenqiang Wu, Kuang-Ting Tu +5Implicit Neural RepresentationsHierarchical Representation Learning

    University of California San Diego, La Jolla, CA, USA · Southern University of Science and Technology, Shenzhen, Guangzhou, China · Norwegian University of Science and Technology, Trondheim, Norway +2

  24. InsightMap: Structured Spatial Modeling for Embodied Multimodal Reasoning

    Sep 29, 2026Hongpei Zheng, Hujun YinAgent MemoryEmbodied Navigation

    University of Manchester

  25. PGL-3D: Towards Progressive Geometric Learning for 3D Visual Query Localization

    Sep 27, 2026Liang Peng, Shizhuo Mu, Bohan Tan +6Geometric Representation Learning3D Representation Learning

    School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University · University of North Texas · Institute of Software, Chinese Academy of Sciences

  26. SceneScaffold: Active Scene-State Construction for Unified 3D Scene Understanding

    Sep 27, 2026Xiangqi Li, Libo Huang, Jiarui Zhao +43D Scene Representation3D VQA

    State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences

  27. Toward Comprehensive 3D Grounding: Orientation Grounding through Vision-Language Models

    Sep 27, 2026Tuo Liang, Disheng Liu, Nengbo Wang +23D Visual Grounding3D Spatial Reasoning

    Case Western Reserve University

  28. Retrieve-to-Localize: Bridging Large Language Models and LiDAR Geometry for Spatial Grounding

    Sep 24, 2026Byounggun Park, Giyong Moon, Jusung Kim +13D Spatial Reasoning3D VLMs

    Department of Automotive Engineering (Automotive-Computer Convergence), Hanyang University, Seoul, South Korea. · Department of Automotive Engineering, Hanyang University, Seoul, South Korea.

  29. From Alignment to Fusion in 3D Vision-Language

    Sep 23, 2026Xueqi Qiu, Xingyu Miao, Jingjing Deng +33D Representation LearningVision-Language Alignment

    School of Engineering Mathematics and Technology, University of Bristol, Bristol, UK · Department of Automation, Tsinghua University, Beijing, China · UCAS-Terminus AI Lab, University of Chinese Academy of Sciences, Beijing 100049, China

  30. MIGU: Multimodal Instruction Grounding under Uncertainty for Manipulation Planning

    Sep 21, 2026Mingke Lu, Anxing Xiao, David HsuMultimodal GroundingRobot Task Planning

    School of Computing and Smart Systems Institute, National University of Singapore, Singapore · Department of Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, CA, USA