3D VLMs

VLM: Vision-Language Model

Overview

Three-dimensional vision-language models (3D VLMs) study how to connect language with point clouds, RGB-D images, multi-view video, and volumetric scans so models can describe, locate, and reason about objects and scenes. Common approaches align 3D features with pretrained image and text models, then add geometric tokens, scene graphs, spatial maps, or active view selection for tasks such as grounding, question answering, captioning, and navigation. Systems are evaluated on benchmarks including ScanRefer, ScanQA, SQA3D, and VSI-Bench, with growing attention to robot interaction and medical imaging. A central challenge is ensuring models use genuine geometric and viewpoint evidence rather than relying on language or 2D visual shortcuts, while keeping inference efficient and performance reliable across scenes and domains.

Latest papers

CardsList

228 papers18 papers in the last four weeks, down 40% on the four weeks before. 0.1% of all new papers.

  1. NativeSeg: Native Vision Language Segmentation of 3D Computed Tomography via Spatial Visual Tokens and Verified Local Zoom

    Nov 18, 2026Chengqian Ma, Gaole Dai, Zijie Meng +53D Medical Image SegmentationText-Guided Image Segmentation

  2. ORGANIC3D: Grounding 3D CT Vision Language Models in Organ Level Image Text Coupling

    Nov 17, 2026Zelong Liu, Hamza Ahmed, Wynn Wingyi Lee +7Medical VLMs3D VLMs

  3. Reference-Guided Data Curation for 3D Medical Vision-Language Pretraining

    Nov 17, 2026Faidra Patsatzi, Ashwin Kumar, Chong Wang +3Training Data CurationMedical VLMs

  4. UNITE: Consolidating Embodied VLM Specialists via Shared-Anchor One-Bit Factorization

    Nov 16, 2026Tao Tao, Anke Tang, Enneng Yang +3VLM MergingVision-Language Models

  5. Persistent Metric-Semantic Maritime Scene Memory from Open-Vocabulary 3D Scene Graphs

    Nov 10, 2026Kim Alexander Christensen, Dina Zimmermann, Martin Steinert +13D Semantic Mapping3D SGG

  6. Anatomy-Activated Mixture-of-Experts for 3D Medical Vision-Language Pre-training

    Nov 7, 2026Szymon Płotka, Gizem Mert, Pedro R. A. S. Bassi +9Medical VLMsMedical Image Representation Learning

  7. Multimodal Hierarchical Mixture-of-Experts for 3D Medical Vision-Language Modeling

    Nov 5, 2026Gizem Mert, Szymon Płotka, Pedro R. A. S. Bassi +5Medical VLMs3D VLMs

  8. TACT-KV: Tri-Axis Cosine Transform for Compressing Volumetric KV Caches in Medical VLMs

    Nov 5, 2026Chongyu Qu, Ritchie Zhao, Yufan He +12KV-Cache CompressionKV-Cache Quantization

  9. Lie Routing Vision Language Models for View-Robust 3D Understanding

    Nov 1, 2026Jingjun Yi, Chen Hu, Qi Bi +73D VLMs3D Scene Understanding

  10. MMCoT3^3: Empowering 3D Scene Understanding with 3D-Text Interleaved Multi-Modal Chain-of-Thought Reasoning

    Nov 1, 2026Yongqi Shan, Yifan Wang, Yunzhi Zhuge +23D Scene Understanding3D VLMs

  11. 3D Occupancy-Language Fields from mmWave Radar

    Oct 27, 2026Haowen Lai, Zhiwei Zheng, Mingmin Zhao3D Semantic MappingSemantic Occupancy Mapping

  12. Zhengrong-Spatial-Aware 1.0: Advancing 3D Multimodal Intelligence

    Oct 27, 2026Yu Zhang, Xiaogang Wang, Yuhang Cheng +23D Visual Question AnsweringMultimodal Spatial Reasoning

  13. 3D-SceneQ: Empowering 3D LLM With Query-Guided Adaptive Pruning and Multi-Modal Feature Enhancement

    Oct 27, 2026Yongqi Shan, Lu Zhang, Jiazuo Yu +23D Scene Understanding3D VLMs

  14. TP-JSCC: Efficient Token Pruning and Joint Source-Channel Coding for LVLMs-based 3D-VQA in Cloud-Edge Collaboration

    Oct 26, 2026Zhengxin Li, Chongzhen Tian, Hui Yuan +2Visual Token Compression3D Visual Question Answering

  15. Beyond Pattern Matching in 3D MLLMs: Part-Level Functional Reasoning for Generalizable Recognition

    Oct 23, 2026Weihong Li, Hao Jiang, Yaohua Zha +23D VLMs3D Spatial Reasoning

  16. Query-Time Calibration of CLIP Features for Open-Vocabulary 3D Segmentation

    Oct 22, 2026Dominik Engel, Yousef Rajeh, Leon Sick +1Open-Vocabulary 3D SegmentationVLM Calibration

  17. Think-Capture: Active 3D Geometry Acquisition for Spatial Reasoning in Vision-Language Models

    Oct 21, 2026Junjie Wang, Zhihao Yuan, Yunhan Tang +7Visual Spatial Reasoning3D VLMs

  18. GALA: Geometry-Aware Language Model for Controllable Object Arrangement

    Oct 20, 2026Chucheng Xiang, Runze Wang, Zhi Deng +103D Scene Layout Generation3D VLMs

  19. P3^3-VLM: A Point-based Alternative for Grounded 3D Vision-Language Models

    Oct 20, 2026Anna-Maria Halacheva, Jan-Nico Zaech, Sombit Dey +23D VLMs3D Visual Grounding

  20. Do 3D Large Language Models Really Use 3D Evidence?

    Oct 17, 2026Yifei Zhao, Fanyu Zhao, Han Zhang +63D VLMs3D Visual Question Answering

  21. FlexPart: Text-Controlled Semantics-Aware 3D Part Segmentation at Flexible Granularity

    Oct 16, 2026Yiqun Zhao, Jiachen Xu, Yihao Zhi +53D Part Segmentation3D Referring Segmentation

  22. DK-VLM: Geometry as Position for Spatial Reasoning in Vision-Language Models

    Oct 15, 2026Han Su, Meng Tian, Tianyu Huang +7Visual Spatial ReasoningSpatial Reasoning

  23. DynThinker: Unifying Static 3D and Dynamic 4D Spatial Reasoning

    Oct 14, 2026Qihang Cao, Haoyuan Li, Zihan Guo +8Spatiotemporal ReasoningSpatial Reasoning

  24. GeoThinker-FF: Endogenous Predictive World Representations for Fast Spatial Intelligence

    Oct 14, 2026Haoyuan Li, Qihang Cao, Huanyuan Zhou +103D VLMs3D Spatial Reasoning

  25. Seeing Depth but Choosing Cues in Vision-Language Models

    Oct 13, 2026Zhengtao Zou, Pekka MarttinenVLM Interpretability3D VLMs