Point cloud segmentation is a fundamental task in 3D data analysis, with applications across robotics, augmented and virtual reality, and remote sensing. However, the inherent sparsity, noise, and unordered nature of point clouds introduce significant challenges, particularly in achieving reliable classification. While the state-of-the-art rely on training models for explicit modeling of uncertainty, there is a gap in exploring different roles that estimated uncertainty using generative approaches can play. This work investigates the impact of deterministic uncertainty in point cloud segmentation by explicitly modeling epistemic uncertainty and using it in different roles, namely, as a feature and in loss function. We evaluate a baseline artificial neural network (ANN) and two uncertainty-aware variants (uncertainty as an additional input feature and as a per-sample confidence weight in the training loss) on four large-scale LiDAR benchmarks spanning airborne and mobile acquisition, namely ISPRS Vaihingen, AHN3, Paris-Lille-3D, and Toronto-3D, together with PointNet++ and RandLA-Net references and their uncertainty-augmented counterparts under canonical spatially disjoint train and test splits. Under this protocol, uncertainty-aware modeling improves mean Intersection over Union, recovers minority classes that the baseline ANN never predicts at all, and in half of the configurations improves the alignment between prediction error and estimated uncertainty as measured by Patch Accuracy versus Patch Uncertainty. Our findings indicate that uncertainty can play multiple roles in the segmentation pipeline rather than as a device for improving accuracy alone.
Figures & tables
Figure 1: Depiction of point cloud segmentation, comparing raw point clouds, RGB data, and segmented outputs (28AN1_16 tile, Lemelerveld region from the AHN3 dataset [ 3 ] ). The legend shows the classes present in the tile; class 0 (never classified) and class 26 (reserved) are excluded.
Descriptor
Definition
Interpretation
Eigenvalue sum
Σλ=λ1+λ2+λ3
Total variance of the neighborhood, independent of how it is distributed among directions. Larger values indicate greater overall spatial extent.
PCA1
PCA1=Σλλ1
Proportion of total variance along the dominant axis.
PCA2
PCA2=Σλλ2
Proportion of total variance along the second axis.
Linearity
L=λ1λ1−λ2
How well the neighborhood aligns along a dominant axis. Values close to 1 indicate linear structure such as edges, wires or poles.
Planarity
P=λ1λ2−λ3
How planar the neighborhood is. High values indicate that most points lie on a surface such as a wall, roof or the ground.
Sphericity
S=λ1λ3
Isotropy of the local point distribution. Larger values indicate a volumetric neighborhood with no dominant direction, as in clutter or foliage.
Table 1: Geometric descriptors computed for each point from the eigenvalues and eigenvectors of the local covariance matrix
Figure 2: Overview of the uncertainty-aware segmentation pipeline. Raw LiDAR point clouds undergo dataset preparation and spatial splitting, geometric feature computation, and epistemic uncertainty estimation (Section 4.3 ); the last of these is computed once and shared, unchanged, across every downstream model. The resulting features and uncertainty estimates are then used to train two parallel model families: the three feed-forward ANN variants M1, M2, and M3 (Section 4.4 ), and the reference architectures, PointNet++ and RandLA-Net, each trained as a baseline and as an uncertainty-augmented counterpart, yielding the final segmented point clouds.
Dataset
Modality
Training files
Test file
Vaihingen
ALS
Area 2
Area 3
AHN3
ALS
28AN1_16 28AZ1_13
32CN2_11
Paris-Lille-3D
MLS
Lille1_1 Lille1_2 Lille2
Paris
Toronto-3D
MLS
L001 L002 L004
L003
Table 2: Canonical file level splits used for all experiments. The test region is disjoint in space from the training region and is never used for training, for fitting the feature scaler, or for fitting the uncertainty model.
Figure 3: Shared feed-forward architecture used by M1, M2, and M3 (Section 4.4 ). All three variants use an identical three-hidden-layer network (128, 64, and 32 units, each followed by ReLU) and a final C -unit softmax layer, where C is the number of semantic classes for the dataset. M1 and M3 take the 17-dimensional feature vector directly as input; M2 additionally concatenates the normalized epistemic uncertainty u~i (Eq. 4 ) as an 18th input dimension, shown here as the uncertainty branch merging into the network input. M3 uses the same 17-dimensional input as M1 and is not distinguished in this forward-pass diagram; it differs only in its training objective, where u~i instead reweights the per-sample cross-entropy loss (Eqs. 5 - 6 ).
Parameter
ANN (M1, M2, M3)
PointNet++
RandLA-Net
Input channels
17 (M2: 18)
3 (+1 with u~i )
3 (+1 with u~i )
Optimizer
Adam
Adam
Adam
Learning rate
10−3
10−3 , cosine schedule
10−2 , decay 0.95 per epoch
Weight decay
none
10−4
none
Batch size
512 points
8 blocks (Vaihingen: 4)
4 blocks
Epochs
30
50 (2000 blocks each, Vaihingen: 1000)
50 (2000 blocks each, Vaihingen: 1000)
Table 3: Training configuration of the three model families. Each baseline shares its configuration with its uncertainty-augmented counterpart.
Dataset
r (m)
Test Acc.
Dropped Pts (%)
Vaihingen
0.10
0.2304
0.13
0.15
0.2263
0.02
0.20
0.2804
0.00
0.25
0.3155
0.00
AHN3
0.10
0.3699
93.47
0.15
0.3128
83.91
Table 4: Effect of the neighborhood radius on test accuracy and on the share of test points without valid descriptors (dropped), per dataset. Each configuration was trained for five epochs. The radius was selected on the training splits, where r=0.25 m also drops the fewest points on every dataset; test values are reported for reference. The most optimal values are in boldface
Figure 4: Segmentation ground truth (left) compared to predictions (M1, M2 and M3, respectively). Visualized using CloudCompare software.
Figure 5: Segmentation ground truth (left) compared to the reference models (PointNet++ baseline and uncertainty-aware variant, RandLA-Net baseline and uncertainty-aware variant, respectively). For class details, refer to the legend in Fig. 4 . Visualized using CloudCompare software.
Dataset
Model
Test Acc.
mIoU
PAvPU
Vaihingen
M1 Baseline ANN
0.3116
0.1166
0.5066
M2 ANN + uncertainty feature
0.2856
0.1095
0.4891
M3 ANN + uncertainty loss
0.3322
0.1216
0.4770
PointNet++
0.5489
0.3450
0.5148
PointNet++ + uncertainty feature
0.6423
0.4275
0.5462
RandLA-Net
0.5302
0.3037
0.5241
Table 5: Test accuracy, mIoU and PAvPU across models and datasets under canonical spatially disjoint splits. Best value per dataset within each model family in boldface, second best within the ANN family underlined.
Class
# test points
M1
M2
M3
PointNet++
PointNet++ + unc.
RandLA-Net
RandLA-Net + unc.
road
20,557,596
0.8943
0.8712
0.8612
0.9037
0.9055
0.9057
0.9057
building
11,676,403
0.7921
0.7236
0.6876
0.9580
0.9610
0.9407
0.9438
car
2,370,844
0.6166
0.4137
0.3630
0.9229
0.9243
0.9258
0.9187
natural
1,939,505
0.4547
0.3848
0.3588
0.9118
0.9248
0.8533
0.8850
road marking
785,591
0.0000
0.0060
0.0123
0.0001
0.0002
0.0000
0.0000
pole
407,843
0.0000
0.2803
0.2613
0.8803
0.8873
0.8309
0.8054
Table 6: Per-class IoU on Toronto-3D (Tested on L003). The results clearly show that the uncertainty aware ANN variants are capable of recovering minority classes at a cost concentrated in the majority classes. Unclassified is excluded from mIoU throughout this table. Best value per class within each model family in boldface, second best within the ANN family underlined.
Class
# test points
M1
M2
M3
PointNet++
PointNet++ + unc.
RandLA-Net
RandLA-Net + unc.
ground
18,103,476
0.3722
0.3828
0.4719
0.9872
0.9871
0.9901
0.9893
building
8,946,816
0.4001
0.3677
0.2207
0.9173
0.9296
0.9171
0.9071
natural vegetation
6,198,638
0.0967
0.1624
0.1500
0.8852
0.8960
0.8777
0.8669
barrier
2,389,847
0.0114
0.0513
0.0013
0.5478
0.5943
0.4360
0.4275
car
1,298,478
0.0111
0.0283
0.0148
0.8770
0.8668
0.8254
0.8727
pole / sign / light
193,166
0.0155
0.0143
0.0109
0.6666
0.6730
0.6114
0.5322
Table 7: Per-class IoU on Paris-Lille-3D (Tested on Paris tile). Unclassified is excluded from mIoU throughout this table. Best value per class within each model family in boldface, second best within the ANN family underlined.
Class
# test points
M1
M2
M3
PointNet++
PointNet++ + unc.
RandLA-Net
RandLA-Net + unc.
Tree
135,449
0.2748
0.2430
0.3250
0.3460
0.4517
0.4155
0.3753
Impervious surfaces
83,372
0.0000
0.0148
0.0235
0.4467
0.5412
0.4818
0.4220
Roof
69,563
0.1917
0.1803
0.1380
0.4626
0.5559
0.2318
0.3132
Low vegetation
35,512
0.0000
0.0000
0.0001
0.1246
0.1613
0.0855
0.1682
mIoU
0.1166
0.1095
0.1216
0.3450
0.4275
0.3037
0.3197
Table 8: Per-class IoU on Vaihingen (trained on Area 2, tested on Area 3). Best value per class within each model family in boldface, second best within the ANN family underlined.
Class
# test points
M1
M2
M3
PointNet++
PointNet++ + unc.
RandLA-Net
RandLA-Net + unc.
Vegetation
9,377,068
0.6720
0.6884
0.6824
0.9257
0.9073
0.9493
0.9524
Ground
7,018,580
0.0984
0.0002
0.0002
0.5467
0.5795
0.0874
0.1346
Building
791,952
0.0126
0.0206
0.0171
0.3323
0.3619
0.3543
0.3971
Reserved
38
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
Water
23
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
mIoU
0.1566
0.1418
0.1399
0.3609
0.3697
0.2782
0.2968
Table 9: Per-class IoU on AHN3 (trained on tiles 28AN1_16 and 28AZ1_13, tested on 32CN2_11). The never-classified category has no points in the test tile and is omitted. Best value per class within each model family in boldface, second best within the ANN family underlined.
Dataset
Model
AC (%)
AU (%)
IC (%)
IU (%)
PAvPU
Vaihingen
M1
15.61
14.95
34.39
35.05
0.5066
M2
13.71
14.80
36.29
35.20
0.4891
M3
15.10
17.40
34.90
32.60
0.4770
PointNet++
28.62
27.14
21.38
22.86
0.5148
PointNet++ + uncertainty
35.41
30.79
14.59
19.21
0.5462
RandLA-Net
28.20
25.79
21.80
24.21
0.5241
Table 10: PAvPU decomposition on the held out split. AC = accurate & certain, AU = accurate & uncertain, IC = inaccurate & certain, IU = inaccurate & uncertain, as percentages of all patches. AC + AU is the share of accurate patches and is therefore not equal to the point-wise test accuracy. PAvPU = AC + IU. The uncertainty threshold is the median patch uncertainty of the test split and is identical across all models. Best value per dataset within each model family in boldface, second best within the ANN family underlined. Higher is better for AC, IU and PAvPU, and lower is better for AU and IC.
Figure 6: Uncertainty u~ on the test tiles. Column 1 shows the ground truth labels. Columns 2 to 4 show subsets of points in the low (blue), intermediate (green) and high (yellow to red) ranges of u~ , and column 5 shows the histogram of u~ with the same color scale, from which the value range of each subset can be read. The histogram tails were clipped per dataset before mapping the color scale, so the ranges follow each dataset’s distribution, and hence, the scales differ between rows. High uncertainty occurs in regions of vegetation, irregular geometry and class boundaries. Visualized using CloudCompare.
Figure 7: Point density variation within and between AHN3 tiles. Top: 28AN1_16, Bottom: 28AZ1_13. Density is the number of neighboring points within a 5 m search radius, min-max normalised per tile to [0, 1]. Left: point clouds colored by normalised density, from blue (low) through green to yellow (high). Right: histograms of the normalised density, colored with the same scale. Within each tile, the bimodal distribution reflects the alternating bands of single flight-strip coverage (lower peak) and strip overlap (higher peak). Between tiles, the peaks lie at different values, so the same color does not represent the same absolute density in both tiles.
In generalized few-shot 3D point cloud segmentation, support annotations are often compromised by sparsity and occlusion. Blindly adapting to such low quality samples risks learning spurious geometric patterns rather than robust novel concepts. Although 3D Vision-Language Models have been introduced to sup plement supervision, the potential of uncertainty to actively govern adaptation dynamics remains underexplored. Existing signal-level methods like loss reweight ing fail to prevent unreliable episodes from steering the optimization trajectory. To address this, we propose Uncertainty-Gated Adaptive (UGA), to the best of our knowledge the first framework that elevates uncertainty to update-level gating for generalized few-shot 3D point cloud segmentation. The core mod ule, Uncertainty-Gated Update (UG-Update), estimates episodic reliability via intra-class dispersion over merged support and pseudo-labeled novel supervision, and selectively gates residual gradient updates. This encourages the model to learn novel semantics from reliable episodes while attenuating updates during unreliable ones. Complementing this, we introduce an Uncertainty-Based Pseudo Label Filter (UB-Filter) to enforce learning from high-confidence dense signals. On ScanNet200 and ScanNet++ (1-/5-shot), UGA sets new state of the art with an average +3.08 HM IoU over the strongest prior baseline.
Yifei Zhao, Fanyu Zhao, Shengtang Wu +3
Fudan University · University of Science and Technology of China
Few-shot 3D semantic segmentation aims to generate accurate semantic masks for query point clouds with only a few annotated support examples. Existing prototype-based methods typically construct compact and deterministic prototypes from the support set to guide query segmentation. However, such rigid representations are unable to capture the intrinsic uncertainty introduced by scarce supervision, which often results in degraded robustness and limited generalization. In this work, we propose UPL (Uncertainty-aware Prototype Learning), a probabilistic approach designed to incorporate uncertainty modeling into prototype learning for few-shot 3D segmentation. Our framework introduces two key components. First, UPL introduces a dual-stream prototype refinement module that enriches prototype representations by jointly leveraging limited information from both support and query samples. Second, we formulate prototype learning as a variational inference problem, regarding class prototypes as latent variables. This probabilistic formulation enables explicit uncertainty modeling, providing robust and interpretable mask predictions. Extensive experiments on the widely used ScanNet and S3DIS benchmarks show that our UPL achieves consistent state-of-the-art performance under different settings while providing reliable uncertainty estimation. The code is available at https://fdueblab-upl.github.io/.
Adverse weather conditions significantly degrade the performance of LiDAR point cloud semantic segmentation networks by introducing large distribution shifts. Existing augmentation-based methods attempt to enhance robustness by simulating weather interference during training. However, they struggle to fully exploit the potential of augmentations due to the trade-off between minor and aggressive augmentations. To address this, we propose A3Point, an adaptive augmentation-aware latent learning framework that effectively utilizes a diverse range of augmentations while mitigating the semantic shift, which refers to the change in the semantic meaning caused by augmentations. A3Point consists of two key components: semantic confusion prior (SCP) latent learning, which captures the model's inherent semantic confusion information, and semantic shift region (SSR) localization, which decouples semantic confusion and semantic shift, enabling adaptive optimization strategies for different disturbance levels. Extensive experiments on multiple standard generalized LiDAR segmentation benchmarks under adverse weather demonstrate the effectiveness of our method, setting new state-of-the-art results.
Wangkai Li, Zhaoyang Li, Yuwen Pan +3
University of Science and Technology of China · National Key Laboratory of Deep Space Exploration, Deep Space Exploration Laboratory