Open-Vocabulary 3D Segmentation
Overview
The field studies how to identify and segment objects, parts, and regions in 3D scenes using free-form language rather than a fixed set of labels, enabling applications such as robotic navigation and object interaction. Common approaches lift features from pretrained 2D vision-language and segmentation models into point clouds, voxel maps, NeRFs, or 3D Gaussian splats, then fuse evidence across views; other methods learn feed-forward 3D representations or use geometric and relational reasoning. Systems are evaluated on indoor benchmarks such as ScanNet, Replica, and LERF-OVS, as well as outdoor LiDAR datasets such as nuScenes and SemanticKITTI. A central tension is making predictions spatially and temporally consistent despite noisy 2D cues and incomplete views, while keeping maps compact and inference fast enough for real-time use and generalizing beyond familiar scenes.
Latest papers
74 papers10 papers in the last four weeks, down 17% on the four weeks before. 0.1% of all new papers.