Open-Vocabulary 3D Segmentation

Overview

The field studies how to identify and segment objects, parts, and regions in 3D scenes using free-form language rather than a fixed set of labels, enabling applications such as robotic navigation and object interaction. Common approaches lift features from pretrained 2D vision-language and segmentation models into point clouds, voxel maps, NeRFs, or 3D Gaussian splats, then fuse evidence across views; other methods learn feed-forward 3D representations or use geometric and relational reasoning. Systems are evaluated on indoor benchmarks such as ScanNet, Replica, and LERF-OVS, as well as outdoor LiDAR datasets such as nuScenes and SemanticKITTI. A central tension is making predictions spatially and temporally consistent despite noisy 2D cues and incomplete views, while keeping maps compact and inference fast enough for real-time use and generalizing beyond familiar scenes.

Latest papers

CardsList

77 papers9 papers in the last four weeks, down 18% on the four weeks before. 0.1% of all new papers.

  1. RefineSplat: Reconstruction-Guided Object-Centric Gaussian Splatting for Open-Vocabulary Querying

    Nov 7, 2026Yiren Lu, Disheng Liu, Sreya Srinidhi +7Open-Vocabulary 3D Segmentation3DGS Segmentation

  2. Feature Puzzles: Single-Image Supervision for Multi-View 3D-Aware Segmentation

    Nov 4, 2026Jiahao Ma, Qiang Zhang, Miaomiao Liu +23D Semantic SegmentationMulti-View Consistency

  3. LucidLyra: Feed-Forward Generative 3D Scene Reconstruction and Joint Understanding via Multi-Task Distillation

    Oct 29, 2026Oliver Hahn, Rodrigo Marcuzzi, Sangeetha Grama Srinivasan +73D Scene Understanding3D Scene Generation

  4. RecallGS: Geometry-Grounded Recall for Online Semantic Gaussian Splatting

    Oct 22, 2026Thanh-Khoi Nguyen, Thien-Phuc Tran, Minh-Triet TranOpen-Vocabulary 3D Segmentation3DGS Segmentation

  5. Query-Time Calibration of CLIP Features for Open-Vocabulary 3D Segmentation

    Oct 22, 2026Dominik Engel, Yousef Rajeh, Leon Sick +1Open-Vocabulary 3D SegmentationVLM Calibration

  6. OV-Foam: How 3D Representations Shape Semantic Feature Lifting

    Oct 20, 2026Yousef Rajeh, Alejandra Esparza Pelcastre, Ivan Viola +1Open-Vocabulary 3D Segmentation3D Scene Representation

  7. Training-Free Target-Aware Refinement of 3D Gaussian Language Map

    Oct 15, 2026Sibaek Lee, Kyeongsu Kang, Seongbo Ha +9Language-Embedded 3DGSOpen-Vocabulary 3D Segmentation

  8. ActiveLang: Active Open-Vocabulary 3D Mapping with Semantic-Uncertainty-Guided Exploration

    Oct 7, 2026Liyan Chen, Hairong Yin, Huangying Zhan +33D Semantic Mapping3D Mapping

  9. OpenSplatGraph: From Dense Semantic Maps to Structured Scene Graphs for Open-Vocabulary Robot Perception

    Oct 6, 2026Binh Long Nguyen, Kien Nguyen, Clinton Fookes +1Language-Embedded 3DGSOpen-Vocabulary 3D Segmentation

  10. Lang3DSeg: Annotation-Free Open-Vocabulary 3D Segmentation with Point Transformers

    Oct 1, 2026Cigdem Kokenoz, Amir Salarpour, Alkim Domeke +5Open-Vocabulary 3D SegmentationTransformer

  11. EviSplat: Preserving Multi-View Evidence in 3D Gaussian Splatting for Open-Vocabulary Segmentation

    Sep 28, 2026Sungho Moon, Kota Shimomura, Junwoo Park +43D Gaussian SplattingLanguage-Embedded 3DGS

  12. TRACKGRAPH: Online Open-Vocabulary 3D Scene Graphs via Image-Space Tracking

    Sep 25, 2026Peder Borge Hellesylt, Albert Gassol Puigjaner, Kostas Alexis +1Open-Vocabulary 3D Segmentation3D Semantic Mapping

  13. OREN-X: Octree Residual Network for Real-Time Multi-Modal Mapping

    Sep 24, 2026Zhirui Dai, Qihao Qian, Dinh Minh Nguyen +6Open-Vocabulary 3D SegmentationSemantic Occupancy Mapping

  14. Hybrid Gaussians for Robust Open-Vocabulary 3D Segmentation with Multi-View Object Association and Boundary Refinement

    Sep 23, 2026Xueqi Qiu, Yueming Sun, Tianyu Zhang +2Language-Embedded 3DGSOpen-Vocabulary 3D Segmentation

  15. PanOVOcc: Panoramic Embodied Open-Vocabulary Occupancy Mapping with Long-term Spatial Voxel Memory

    Sep 21, 2026Di Kuang, Mengfei Duan, Yuhang Wang +2Semantic Occupancy MappingOpen-Vocabulary 3D Segmentation

  16. SenseFuse: Label-Free Fusion of Image and Shape Encoders for Open-Vocabulary 3D Instance Segmentation

    Sep 17, 2026Euiseok Han, Tri Ton, Hwanhee Kim +2Open-Vocabulary 3D SegmentationMultimodal Fusion

  17. PerSeM: Persistent Semantic Memory for Long-Horizon Open-Vocabulary UAV Mapping

    Sep 17, 2026Saurbh Singh Jamwal, Ganesh RamakrishnanSemantic MappingOpen-Vocabulary 3D Segmentation

  18. NormLift: From Lifted Features To Semantic Reliability In 3D Gaussian Splatting

    Sep 16, 2026Yihan Zang, Da Li, Dominik Engel +2Open-Vocabulary 3D SegmentationLanguage-Embedded 3DGS

  19. UniPart: Towards Zero-shot Language-Grounded 3D Part Segmentation for Embodied Interaction

    Sep 11, 2026Xinqiang Yu, Zekun qi, Jiawei He +63D ViTsOpen-Vocabulary 3D Segmentation

  20. GoDeep: Annotation-Free Open-Vocabulary 3D Scene Understanding via Language-Space Lifting

    Sep 8, 2026Thodoris Betsas, Anastasios Doulamis, Andreas GeorgopoulosOpen-Vocabulary 3D Segmentation3D Semantic Segmentation

  21. Zero-Shot 3D Plant Organ Segmentation with SAM3 and Semantic NeRFs

    Sep 7, 2026Andreas Gilson, Laura Hennig, Peter PietrzykOpen-Vocabulary 3D Segmentation3D Semantic Segmentation

  22. DSSR-3D: Decoupled Reasoning for View-Dependent Referring in 3D Gaussians

    Sep 3, 2026Thanh-Khoi Nguyen, Thien-Phuc Tran, Minh-Triet Tran3D Referring Segmentation3D Visual Grounding

  23. VOIM: Training-Free Open-Vocabulary 3D Instance Mapping for RGB-D and Monocular SLAM

    Sep 1, 2026Sangmin Song, Sarath Kodagoda, Marc G. Carmichael +4Simultaneous Localization and MappingOpen-Vocabulary 3D Segmentation

  24. Dynamic-Robust Photometric-Semantic Reconstruction for Open-Vocabulary 3D Scene Understanding

    Aug 29, 2026Boyu Cai, Li Yang, Yan Xu +6Dynamic Scene Reconstruction3D Scene Understanding

  25. Understanding Temporal Semantic Stability in Open-Vocabulary UAV Perception through Metric 3D Fusion

    Aug 23, 2026Saurbh Singh JamwalTemporal ConsistencyOpen-Vocabulary 3D Segmentation

  26. Stream3Dv2: Geometric-Semantic Fusion Enhanced Streaming Zero-Shot 3D Scene Understanding

    Aug 21, 2026Jie Xu, Na Zhao3D Scene UnderstandingOpen-Vocabulary 3D Segmentation

  27. Beyond Similarity Matching: Structured Reasoning for Open-Vocabulary Referring Segmentation in 3DGS

    Aug 17, 2026Yizhao Wang, Xinfa Wang, Jingbo Wang +53D Referring SegmentationReferring Expression Segmentation