Point cloud segmentation is a fundamental task in 3D data analysis, with applications across robotics, augmented and virtual reality, and remote sensing. However, the inherent sparsity, noise, and unordered nature of point clouds introduce significant challenges, particularly in achieving reliable classification. While the state-of-the-art rely on training models for explicit modeling of uncertainty, there is a gap in exploring different roles that estimated uncertainty using generative approaches can play. This work investigates the impact of deterministic uncertainty in point cloud segmentation by explicitly modeling epistemic uncertainty and using it in different roles, namely, as a feature and in loss function. We evaluate a baseline artificial neural network (ANN) and two uncertainty-aware variants (uncertainty as an additional input feature and as a per-sample confidence weight in the training loss) on four large-scale LiDAR benchmarks spanning airborne and mobile acquisition, namely ISPRS Vaihingen, AHN3, Paris-Lille-3D, and Toronto-3D, together with PointNet++ and RandLA-Net references and their uncertainty-augmented counterparts under canonical spatially disjoint train and test splits. Under this protocol, uncertainty-aware modeling improves mean Intersection over Union, recovers minority classes that the baseline ANN never predicts at all, and in half of the configurations improves the alignment between prediction error and estimated uncertainty as measured by Patch Accuracy versus Patch Uncertainty. Our findings indicate that uncertainty can play multiple roles in the segmentation pipeline rather than as a device for improving accuracy alone.
Figures & tables
Figure 1: Depiction of point cloud segmentation, comparing raw point clouds, RGB data, and segmented outputs (28AN1_16 tile, Lemelerveld region from the AHN3 dataset [ 3 ] ). The legend shows the classes present in the tile; class 0 (never classified) and class 26 (reserved) are excluded.
Descriptor
Definition
Interpretation
Eigenvalue sum
Σλ=λ1+λ2+λ3
Total variance of the neighborhood, independent of how it is distributed among directions. Larger values indicate greater overall spatial extent.
PCA1
PCA1=Σλλ1
Proportion of total variance along the dominant axis.
PCA2
PCA2=Σλλ2
Proportion of total variance along the second axis.
Linearity
L=λ1λ1−λ2
How well the neighborhood aligns along a dominant axis. Values close to 1 indicate linear structure such as edges, wires or poles.
Planarity
P=λ1λ2−λ3
How planar the neighborhood is. High values indicate that most points lie on a surface such as a wall, roof or the ground.
Sphericity
S=λ1λ3
Isotropy of the local point distribution. Larger values indicate a volumetric neighborhood with no dominant direction, as in clutter or foliage.
Table 1: Geometric descriptors computed for each point from the eigenvalues and eigenvectors of the local covariance matrix
Figure 2: Overview of the uncertainty-aware segmentation pipeline. Raw LiDAR point clouds undergo dataset preparation and spatial splitting, geometric feature computation, and epistemic uncertainty estimation (Section 4.3 ); the last of these is computed once and shared, unchanged, across every downstream model. The resulting features and uncertainty estimates are then used to train two parallel model families: the three feed-forward ANN variants M1, M2, and M3 (Section 4.4 ), and the reference architectures, PointNet++ and RandLA-Net, each trained as a baseline and as an uncertainty-augmented counterpart, yielding the final segmented point clouds.
Dataset
Modality
Training files
Test file
Vaihingen
ALS
Area 2
Area 3
AHN3
ALS
28AN1_16 28AZ1_13
32CN2_11
Paris-Lille-3D
MLS
Lille1_1 Lille1_2 Lille2
Paris
Toronto-3D
MLS
L001 L002 L004
L003
Table 2: Canonical file level splits used for all experiments. The test region is disjoint in space from the training region and is never used for training, for fitting the feature scaler, or for fitting the uncertainty model.
Figure 3: Shared feed-forward architecture used by M1, M2, and M3 (Section 4.4 ). All three variants use an identical three-hidden-layer network (128, 64, and 32 units, each followed by ReLU) and a final C -unit softmax layer, where C is the number of semantic classes for the dataset. M1 and M3 take the 17-dimensional feature vector directly as input; M2 additionally concatenates the normalized epistemic uncertainty u~i (Eq. 4 ) as an 18th input dimension, shown here as the uncertainty branch merging into the network input. M3 uses the same 17-dimensional input as M1 and is not distinguished in this forward-pass diagram; it differs only in its training objective, where u~i instead reweights the per-sample cross-entropy loss (Eqs. 5 - 6 ).
Parameter
ANN (M1, M2, M3)
PointNet++
RandLA-Net
Input channels
17 (M2: 18)
3 (+1 with u~i )
3 (+1 with u~i )
Optimizer
Adam
Adam
Adam
Learning rate
10−3
10−3 , cosine schedule
10−2 , decay 0.95 per epoch
Weight decay
none
10−4
none
Batch size
512 points
8 blocks (Vaihingen: 4)
4 blocks
Epochs
30
50 (2000 blocks each, Vaihingen: 1000)
50 (2000 blocks each, Vaihingen: 1000)
Table 3: Training configuration of the three model families. Each baseline shares its configuration with its uncertainty-augmented counterpart.
Dataset
r (m)
Test Acc.
Dropped Pts (%)
Vaihingen
0.10
0.2304
0.13
0.15
0.2263
0.02
0.20
0.2804
0.00
0.25
0.3155
0.00
AHN3
0.10
0.3699
93.47
0.15
0.3128
83.91
Table 4: Effect of the neighborhood radius on test accuracy and on the share of test points without valid descriptors (dropped), per dataset. Each configuration was trained for five epochs. The radius was selected on the training splits, where r=0.25 m also drops the fewest points on every dataset; test values are reported for reference. The most optimal values are in boldface
Figure 4: Segmentation ground truth (left) compared to predictions (M1, M2 and M3, respectively). Visualized using CloudCompare software.
Figure 5: Segmentation ground truth (left) compared to the reference models (PointNet++ baseline and uncertainty-aware variant, RandLA-Net baseline and uncertainty-aware variant, respectively). For class details, refer to the legend in Fig. 4 . Visualized using CloudCompare software.
Dataset
Model
Test Acc.
mIoU
PAvPU
Vaihingen
M1 Baseline ANN
0.3116
0.1166
0.5066
M2 ANN + uncertainty feature
0.2856
0.1095
0.4891
M3 ANN + uncertainty loss
0.3322
0.1216
0.4770
PointNet++
0.5489
0.3450
0.5148
PointNet++ + uncertainty feature
0.6423
0.4275
0.5462
RandLA-Net
0.5302
0.3037
0.5241
Table 5: Test accuracy, mIoU and PAvPU across models and datasets under canonical spatially disjoint splits. Best value per dataset within each model family in boldface, second best within the ANN family underlined.
Class
# test points
M1
M2
M3
PointNet++
PointNet++ + unc.
RandLA-Net
RandLA-Net + unc.
road
20,557,596
0.8943
0.8712
0.8612
0.9037
0.9055
0.9057
0.9057
building
11,676,403
0.7921
0.7236
0.6876
0.9580
0.9610
0.9407
0.9438
car
2,370,844
0.6166
0.4137
0.3630
0.9229
0.9243
0.9258
0.9187
natural
1,939,505
0.4547
0.3848
0.3588
0.9118
0.9248
0.8533
0.8850
road marking
785,591
0.0000
0.0060
0.0123
0.0001
0.0002
0.0000
0.0000
pole
407,843
0.0000
0.2803
0.2613
0.8803
0.8873
0.8309
0.8054
Table 6: Per-class IoU on Toronto-3D (Tested on L003). The results clearly show that the uncertainty aware ANN variants are capable of recovering minority classes at a cost concentrated in the majority classes. Unclassified is excluded from mIoU throughout this table. Best value per class within each model family in boldface, second best within the ANN family underlined.
Class
# test points
M1
M2
M3
PointNet++
PointNet++ + unc.
RandLA-Net
RandLA-Net + unc.
ground
18,103,476
0.3722
0.3828
0.4719
0.9872
0.9871
0.9901
0.9893
building
8,946,816
0.4001
0.3677
0.2207
0.9173
0.9296
0.9171
0.9071
natural vegetation
6,198,638
0.0967
0.1624
0.1500
0.8852
0.8960
0.8777
0.8669
barrier
2,389,847
0.0114
0.0513
0.0013
0.5478
0.5943
0.4360
0.4275
car
1,298,478
0.0111
0.0283
0.0148
0.8770
0.8668
0.8254
0.8727
pole / sign / light
193,166
0.0155
0.0143
0.0109
0.6666
0.6730
0.6114
0.5322
Table 7: Per-class IoU on Paris-Lille-3D (Tested on Paris tile). Unclassified is excluded from mIoU throughout this table. Best value per class within each model family in boldface, second best within the ANN family underlined.
Class
# test points
M1
M2
M3
PointNet++
PointNet++ + unc.
RandLA-Net
RandLA-Net + unc.
Tree
135,449
0.2748
0.2430
0.3250
0.3460
0.4517
0.4155
0.3753
Impervious surfaces
83,372
0.0000
0.0148
0.0235
0.4467
0.5412
0.4818
0.4220
Roof
69,563
0.1917
0.1803
0.1380
0.4626
0.5559
0.2318
0.3132
Low vegetation
35,512
0.0000
0.0000
0.0001
0.1246
0.1613
0.0855
0.1682
mIoU
0.1166
0.1095
0.1216
0.3450
0.4275
0.3037
0.3197
Table 8: Per-class IoU on Vaihingen (trained on Area 2, tested on Area 3). Best value per class within each model family in boldface, second best within the ANN family underlined.
Class
# test points
M1
M2
M3
PointNet++
PointNet++ + unc.
RandLA-Net
RandLA-Net + unc.
Vegetation
9,377,068
0.6720
0.6884
0.6824
0.9257
0.9073
0.9493
0.9524
Ground
7,018,580
0.0984
0.0002
0.0002
0.5467
0.5795
0.0874
0.1346
Building
791,952
0.0126
0.0206
0.0171
0.3323
0.3619
0.3543
0.3971
Reserved
38
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
Water
23
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
mIoU
0.1566
0.1418
0.1399
0.3609
0.3697
0.2782
0.2968
Table 9: Per-class IoU on AHN3 (trained on tiles 28AN1_16 and 28AZ1_13, tested on 32CN2_11). The never-classified category has no points in the test tile and is omitted. Best value per class within each model family in boldface, second best within the ANN family underlined.
Dataset
Model
AC (%)
AU (%)
IC (%)
IU (%)
PAvPU
Vaihingen
M1
15.61
14.95
34.39
35.05
0.5066
M2
13.71
14.80
36.29
35.20
0.4891
M3
15.10
17.40
34.90
32.60
0.4770
PointNet++
28.62
27.14
21.38
22.86
0.5148
PointNet++ + uncertainty
35.41
30.79
14.59
19.21
0.5462
RandLA-Net
28.20
25.79
21.80
24.21
0.5241
Table 10: PAvPU decomposition on the held out split. AC = accurate & certain, AU = accurate & uncertain, IC = inaccurate & certain, IU = inaccurate & uncertain, as percentages of all patches. AC + AU is the share of accurate patches and is therefore not equal to the point-wise test accuracy. PAvPU = AC + IU. The uncertainty threshold is the median patch uncertainty of the test split and is identical across all models. Best value per dataset within each model family in boldface, second best within the ANN family underlined. Higher is better for AC, IU and PAvPU, and lower is better for AU and IC.
Figure 6: Uncertainty u~ on the test tiles. Column 1 shows the ground truth labels. Columns 2 to 4 show subsets of points in the low (blue), intermediate (green) and high (yellow to red) ranges of u~ , and column 5 shows the histogram of u~ with the same color scale, from which the value range of each subset can be read. The histogram tails were clipped per dataset before mapping the color scale, so the ranges follow each dataset’s distribution, and hence, the scales differ between rows. High uncertainty occurs in regions of vegetation, irregular geometry and class boundaries. Visualized using CloudCompare.
Figure 7: Point density variation within and between AHN3 tiles. Top: 28AN1_16, Bottom: 28AZ1_13. Density is the number of neighboring points within a 5 m search radius, min-max normalised per tile to [0, 1]. Left: point clouds colored by normalised density, from blue (low) through green to yellow (high). Right: histograms of the normalised density, colored with the same scale. Within each tile, the bimodal distribution reflects the alternating bands of single flight-strip coverage (lower peak) and strip overlap (higher peak). Between tiles, the peaks lie at different values, so the same color does not represent the same absolute density in both tiles.