P-SLCR: Unsupervised Point Cloud Semantic Segmentation via Prototypes Structure Learning and Consistent Reasoning
Authors: Lixin Zhan, Jie Jiang, Tianjian Zhou, Yukun Du, Yan Zheng, Xuehu Duan
Organizations: College of Systems Engineering, National University of Defense Technology · Laboratory for Big Data and Decision, National University of Defense Technology
Current semantic segmentation approaches for point cloud scenes heavily rely on manual labeling, while research on unsupervised semantic segmentation methods specifically for raw point clouds is still in its early stages. Unsupervised point cloud learning poses significant challenges due to the absence of annotation information and the lack of pre-training. The development of effective strategies is crucial in this context. In this paper, we propose a novel prototype library-driven unsupervised point cloud semantic segmentation strategy that utilizes Structure Learning and Consistent Reasoning (P-SLCR). First, we propose a Consistent Structure Learning to establish structural feature learning between consistent points and the library of consistent prototypes by selecting high-quality features. Second, we propose a Semantic Relation Consistent Reasoning that constructs a prototype inter-relation matrix between consistent and ambiguous prototype libraries separately. This process ensures the preservation of semantic consistency by imposing constraints on consistent and ambiguous prototype libraries through the prototype inter-relation matrix. Finally, our method was extensively evaluated on the S3DIS, SemanticKITTI, and Scannet datasets, achieving the best performance compared to unsupervised methods. Specifically, the mIoU of 47.1% is achieved for Area-5 of the S3DIS dataset, surpassing the classical fully supervised method PointNet by 2.5%.
Figures & tables
Figure 1: P-SLCR architecture for structural learning and consistent reasoning through a learnable prototype-based library. The consistent points are continuously expanding and increasingly resembling the prototype.
Figure 2: An overview of our P-SLCR with features extracted by SparseConv, pseudo-labels and predictions obtained by clustering and segmenting the header. The proposed reliability classifies the scene into consistent and ambiguous points and establishes an updatable prototype library to construct consistent structure learning and semantic relation consistent reasoning based on the prototype library.
Method
OA(%)
mAcc(%)
mIoU(%)
PointNet
77.5
59.1
44.6
Supervised
PointNet++
77.5
62.6
50.1
Methods
SparseConv
88.4
69.2
60.8
semi-supervised (10%)
Jiang et al.
69.1
-
57.7
weakly
MT
-
-
44.4
supervised (1pt)
Zhang et al.
-
-
48.2
Table 1: Quantitative results for semantic segmentation on the S3DIS dataset. Only 12 categories evaluated on the Area-5.
Figure 3: The qualitative results on S3DIS demonstrate the segmentation performance in comparison to GrowSP. Each color represents a semantic class. We highlight the differences with black circles.
Method
OA(%)
mAcc(%)
mIoU(%)
Supervised
PointNet
75.9
67.1
49.4
PointNet++
77.1
74.1
55.1
Methods
SparseConv
89.4
78.1
69.2
GrowSP
76.0
59.4
44.6
P-SLCR (ours)
79.0
62.2
47.5
Table 2: Quantitative results for semantic segmentation on the S3DIS 6-Fold. Only 12 categories evaluated.
Figure 4: The qualitative results on SemanticKITTI demonstrate the segmentation performance in comparison to GrowSP. We zoomed in on local regions to show details and highlighted differences with black circles.
Method
OA(%)
mAcc(%)
mIoU(%)
KMeans
8.2
8.1
2.5
DBSCAN
17.8
7.5
6.8
Unsupervised
IIC-S-PFH
23.4
9.0
4.6
Methods
PICIE-S-PFH
42.7
11.5
6.8
GrowSP
38.3
19.7
13.2
U3DS 3
34.8
23.1
14.2
Table 3: Quantitative results for semantic segmentation on the SemanticKITTI dataset. All 19 categories evaluated on the validation set.
Method
mIoU(%)
PointNet
14.6
Supervised
PointNet++
20.1
Methods
SparseConv
53.2
Unsupervised
GrowSP
14.3
Methods
P-SLCR (ours)
15.9
Table 4: Quantitative results for semantic segmentation on the SemanticKITTI dataset. All 19 categories evaluated on the online test set.
Figure 5: The qualitative results on ScanNet demonstrate the segmentation performance in comparison to GrowSP. Each color represents a semantic class. We highlight the differences with black circles.
Method
OA(%)
mAcc(%)
mIoU(%)
KMeans
10.2
10.4
3.4
DBSCAN
15.3
10.1
6.1
Unsupervised
IIC-S-PFH
18.9
6.3
3.0
Methods
PICIE-S-PFH
23.6
15.1
7.4
GrowSP
57.3
44.2
25.4
U3DS 3
60.1
46.8
27.3
Table 5: Quantitative results for semantic segmentation on the ScanNet dataset. All 20 categories evaluated on the validation set.
OA(%)
mAcc(%)
mIoU(%)
(1)
Remove Reliable Structure Learning
79.36
55.63
45.65
(2)
Remove Consistent Reasoning
79.08
54.09
42.37
(3)
Remove Color Enhancement
79.28
57.50
45.21
(4)
τ = 0.6 for Degree of Reliability
79.19
57.08
43.80
(5)
τ = 0.7 for Degree of Reliability
79.72
57.23
47.08
(6)
τ = 0.8 for Degree of Reliability
78.88
56.76
44.50
Table 6: The OA, mAcc and mIoU scores of all ablated networks on Area-5 of S3DIS based on our full P-SLCR.
Figure 6: Similarity measure matrix visualization results for consistent and ambiguous prototypes. For presentation purposes, we clustered the number of prototypes into categories of visualization results for ScanNet validation sets.
Figure 7: The visualization shows the distance from the prototype for different categories of consistent points. The visualization results for the validation set based on the ScanNet.
3D point cloud segmentation assigns semantic, instance, or part labels to points for scene understanding, mapping, robotics, and scientific measurement.
Unsupervised point cloud segmentation is critical for embodied artificial intelligence and autonomous driving, as it mitigates the prohibitive cost of dense point-level annotations required by fully supervised methods. While integrating 2D pre-trained models such as the Segment Anything Model (SAM) to supplement semantic information is a natural choice, this approach faces a fundamental mismatch between discrete 3D points and continuous 2D images. This mismatch leads to inevitable projection overlap and complex modality alignment, resulting in compromised semantic consistency across 2D-3D transfer. To address these limitations, this paper proposes PointGS, a simple yet effective pipeline for unsupervised 3D point cloud segmentation. PointGS leverages 3D Gaussian Splatting as a unified intermediate representation to bridge the discrete-continuous domain gap. Input sparse point clouds are first reconstructed into dense 3D Gaussian spaces via multi-view observations, filling spatial gaps and encoding occlusion relationships to eliminate projection-induced semantic conflation. Multi-view dense images are rendered from the Gaussian space, with 2D semantic masks extracted via SAM, and semantics are distilled to 3D Gaussian primitives through contrastive learning to ensure consistent semantic assignments across different views. The Gaussian space is aligned with the original point cloud via two-step registration, and point semantics are assigned through nearest-neighbor search on labeled Gaussians. Experiments demonstrate that PointGS outperforms state-of-the-art unsupervised methods, achieving +0.9% mIoU on ScanNet-V2 and +2.8% mIoU on S3DIS.
Yixiao Song, Qingyong Li, Wen Wang +1
Key Laboratory of Big Data & Artificial Intelligence in Transportation (Beijing Jiaotong University), Ministry of Education · Frontiers Science Center for Smart High-speed Railway System, Beijing Jiaotong University
3D point cloud semantic segmentation is essential for real-world spatial understanding, yet the prohibitive cost of human annotations motivates unsupervised approaches that require no labels. Existing superpoint-based methods typically rely on spectral analysis at a fixed granularity, failing to capture the hierarchical semantic structures inherent in complex indoor scenes. To bridge this gap, we present a Multi-Scale Spatially-Constrained Partition (MSSP) framework that combines multi-scale spectral analysis with spatially-constrained clustering. Multi-scale spectral analysis constructs enriched superpoint descriptors across multiple clustering granularities; however, the resulting high-dimensional feature space calls for a structural prior to translate into cleaner segmentation. Spatially-constrained clustering supplies this prior by restricting superpoint merging to physically adjacent regions, imposing the spatial coherence needed for multi-scale features to be effective. Extensive experiments on S3DIS and ScanNet show that MSSP achieves the best mIoU among unsupervised methods on the main benchmarks, with particularly significant gains on S3DIS. Notably, our ablation reveals a regularize-then-enrich interaction: multi-scale features alone do not improve final segmentation, yet become highly effective when coupled with spatial regularization, underscoring that spatial coherence is aprerequisite for multi-scale representations in superpoint clustering.
Zhenghao Zhang, Xinjie Wang, Wei Wang +2
College of Electronic Science and Technology, National University of Defense Technology, Changsha, China · School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen, China
Few-shot 3D semantic segmentation aims to generate accurate semantic masks for query point clouds with only a few annotated support examples. Existing prototype-based methods typically construct compact and deterministic prototypes from the support set to guide query segmentation. However, such rigid representations are unable to capture the intrinsic uncertainty introduced by scarce supervision, which often results in degraded robustness and limited generalization. In this work, we propose UPL (Uncertainty-aware Prototype Learning), a probabilistic approach designed to incorporate uncertainty modeling into prototype learning for few-shot 3D segmentation. Our framework introduces two key components. First, UPL introduces a dual-stream prototype refinement module that enriches prototype representations by jointly leveraging limited information from both support and query samples. Second, we formulate prototype learning as a variational inference problem, regarding class prototypes as latent variables. This probabilistic formulation enables explicit uncertainty modeling, providing robust and interpretable mask predictions. Extensive experiments on the widely used ScanNet and S3DIS benchmarks show that our UPL achieves consistent state-of-the-art performance under different settings while providing reliable uncertainty estimation. The code is available at https://fdueblab-upl.github.io/.