3D Visual Grounding

Ask about this topic

Overview

Three-dimensional visual grounding studies how to locate and identify the objects or regions described by language or images, often predicting a 3D box, point, or mask. Systems combine language and visual features with point-cloud geometry, scene graphs, or multi-view RGB-D evidence; newer approaches also use vision-language models, Gaussian splatting, and tool-using agents. Benchmarks such as ScanRefer, Nr3D, and SceneFun3D test referring expressions, relational descriptions, and functional regions, while open-vocabulary methods aim to handle unseen objects and scenes. Performance is improving, but reliable grounding remains difficult when viewpoints, ambiguous language, occlusion, or metric spatial constraints matter.

Latest papers

CardsList

142 papers21 papers in the last four weeks, up 5% on the four weeks before. 0.1% of all new papers.

  1. X-GS: An Extensible Framework for Perceiving and Thinking with 3D Gaussian Splatting

    Mar 10, 2026Yueen Ma, Zenglin Xu, Irwin King3D Gaussian Splatting3DGS SLAM

    The Chinese University of Hong Kong · Shanghai Academy of AI for Science · Fudan University

  2. GST-VLA: Structured Gaussian Spatial Tokens for 3D Depth-Aware Vision-Language-Action Models

    Mar 10, 2026Md Selim Sarowar, Omer Tariq, Sungho KimVision-Language-Action ModelsVLMs for Robotics

    Yeungnam University · Korea Advanced Institute of Science and Technology, KAIST

  3. UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsing

    Mar 9, 2026Jiaxi Zhang, Yunheng Wang, Wei Lu +83D Visual Grounding3D VLMs

    The Hong Kong University of Science and Technology (Guangzhou) · JD Explore Academy · Zhejiang University

  4. TrianguLang: Geometry-Aware Semantic Consensus for Pose-Free 3D Localization

    Mar 9, 2026Bryce Grant, Aryeh Rothenberg, Atri Banerjee +13D Visual GroundingOpen-Vocabulary 3D Segmentation

    Case Western Reserve University, Cleveland, OH, USA

  5. Holi-Spatial: Evolving Video Streams into Holistic 3D Spatial Intelligence

    Mar 8, 2026Yuanyuan Gao, Hao Li, Yifei Liu +143D Scene Understanding3D Spatial Reasoning

    Shanghai AI Lab · Northwestern Polytechnical University · Nanyang Technological University +8

  6. Hierarchical Collaborative Fusion for 3D Instance-aware Referring Expression Segmentation

    Mar 6, 2026Keshen Zhou, Runnan Chen, Mingming Gong +13D Referring SegmentationReferring Expression Segmentation

    The University of Sydney · The University of Melbourne · Mohamed bin Zayed University of Artificial Intelligence

  7. 3D-RFT: Reinforcement Fine-Tuning for Video-based 3D Scene Understanding

    Mar 5, 2026Xiongkun Linghu, Jiangyong Huang, Baoxiong Jia +13D Scene Understanding3D VLMs

    State Key Laboratory of General Artificial Intelligence, BIGAI · Peking University

  8. 3D-DRES: Detailed 3D Referring Expression Segmentation

    Mar 3, 2026Qi Chen, Changli Wu, Jiayi Ji +23D Referring SegmentationReferring Expression Segmentation

    Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Shanghai Innovation Institute · National University of Singapore

  9. HAMMER: Harnessing MLLM via Cross-Modal Integration for Intention-Driven 3D Affordance Grounding

    Mar 2, 2026Lei Yao, Yong Chen, Yuejiao Su +33D Affordance Grounding3D Visual Grounding

    The Hong Kong Polytechnic University · Huazhong University of Science and Technology

  10. Monocular 3D Object Position Estimation with VLMs for Human-Robot Interaction

    Mar 1, 2026Ari Wahl, Dorian Gawlinski, David Przewozny +3VLMs for Robotics3D Visual Grounding

    Fraunhofer HHI, Berlin, Germany · Berliner Hochschule für Technik (BHT), Berlin, Germany

  11. SSR: Pushing the Limit of Spatial Intelligence with Structured Scene Reasoning

    Feb 28, 2026Yi Zhang, Youya Xia, Yong Wang +73D Spatial Reasoning3D VLMs

    Foundation Model Department, Huawei · Central Media Technology Institute, Huawei

  12. VAGNet: Grounding 3D Affordance from Human-Object Interactions in Videos

    Feb 24, 2026Aihua Mao, Kaihang Huang, Yong-Jin Liu +23D Affordance Grounding3D Visual Grounding

    SCUT · THU · UM +1

  13. Direction-aware 3D Large Multimodal Models

    Feb 22, 2026Quan Liu, Weihao Xuan, Junjue Wang +33D VLMs3D Visual Grounding

    Nanyang Technological University · The University of Tokyo · RIKEN AIP +1

  14. SplatReasoner: Enhancing Embodied Reasoning and Grounding by Novel View Synthesis

    Jan 19, 2026Kim Yu-Ji, Dahye Lee, Kim Jun-Seong +6VLM ReasoningVisually Grounded Reasoning

    POSTECH · KAIST · ETRI +1

  15. MedVL-SAM2: A unified 3D medical vision-language model for multimodal reasoning and prompt-driven segmentation

    Jan 14, 2026Yang Xing, Jiong Wu, Savas Ozdemir +4Interactive Medical Image SegmentationUnified Multimodal Models

    Department of Biomedical Engineering, University of Florida, Gainesville, FL, USA · Department of Radiology, University of Florida, Jacksonville, FL, USA · Research Computing, University of Florida, Gainesville, FL, USA +2

  16. OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding

    Dec 28, 2025Wenyuan Huang, Zhenyu Zhang, Zhao Wang +43D VLMs3D Visual Grounding

    Nanjing University, School of Intelligent Science and Technology, Nanjing, China · China Mobile Zijin Innovation Institute, Nanjing, China

  17. PanoGrounder: Bridging 2D and 3D with Panoramic Scene Representations for VLM-based 3D Visual Grounding

    Dec 24, 2025Seongmin Jung, Seongho Choi, Gunwoo Jeon +23D Scene Representation3D Visual Grounding

    Seoul National University, South Korea · Robotics Lab, Hyundai Motor Company, South Korea · Pohang University of Science and Technology (POSTECH), South Korea

  18. Towards Spatial Trace with Reasoning in Vision-Language Models for Robotics

    Dec 15, 2025Enshen Zhou, Yibo Li, Jingkun An +123D Spatial ReasoningVLMs for Robotics

    School of Software, Beihang University · Beijing Academy of Artificial Intelligence · State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University +2

  19. Error-Driven Scene Editing for 3D Grounding in Large Language Models

    Nov 18, 2025Yue Zhang, Zun Wang, Han Lin +53D Scene Editing3D Spatial Reasoning

    UNC Chapel Hill · University of Michigan · Google Research

  20. Diffusion Models are Open-World Affordance Learners: Leveraging Generative Priors for 3D Affordance Learning

    Aug 3, 2025Hanqing Wang, Zhenhao Zhang, Kaiyang Ji +123D Affordance GroundingHuman-Object Interaction

    The Hong Kong University of Science and Technology, Guangzhou, China · ShanghaiTech University, Shanghai, China · Zhejiang University, Hangzhou, China +3

  21. Scene-R1: Video-Grounded Large Language Models for 3D Scene Reasoning with Lightweight Annotations

    Jun 21, 2025Junjie Wang, Zhihao Yuan, Shuyi Jiang +53D Scene Understanding3D VLMs

    FNii-Shenzhen, CUHKSZ · SSE, CUHKSZ · Singapore University of Technology and Design +1

  22. Affogato: Open-Vocabulary Affordance Grounding with Automated Data Generation at Scale

    Jun 13, 2025Junha Lee, Eunha Park, Chunghyun Park +2Synthetic Data Generation for RoboticsEmbodied AI

    Pohang University of Science and Engineering (POSTECH) · SqueezeBits · RLWLRD