LiDAR perception is fundamental to robotics, enabling machines to understand their environment in 3D. A crucial task for LiDAR-based scene understanding and navigation is ground segmentation. However, existing methods are either handcrafted for specific sensor configurations or rely on costly per-point manual labels, severely limiting their generalization and scalability. To overcome this, we introduce TerraSeg, the first self-supervised, domain-agnostic model for LiDAR ground segmentation. We train TerraSeg on OmniLiDAR, a unified large-scale dataset that aggregates and standardizes data from 12 major public benchmarks. Spanning almost 22 million raw scans across 15 distinct sensor models, OmniLiDAR provides unprecedented diversity for learning a highly generalizable ground model. To supervise training without human annotations, we propose PseudoLabeler, a novel module that generates high-quality ground and non-ground labels through self-supervised per-scan runtime optimization. Extensive evaluations demonstrate that, despite using no manual labels, TerraSeg achieves state-of-the-art results on nuScenes, SemanticKITTI, and Waymo Perception while delivering real-time performance. Our code and model weights are publicly available.
Figures & tables
Figure 2: Overview of TerraSeg. (a) PseudoLabeler generates self-supervised point-wise ground/non-ground labels per raw LiDAR scan. (b) OmniLiDAR unifies 12 major automotive datasets within an aggregated framework, yielding a diverse curated corpus drawn from nearly 22 million raw scans. (c) TerraSeg is a real-time, domain-agnostic model for ground segmentation, trained on OmniLiDAR using these self-supervised pseudo-labels. We introduce two architectures: accurate Base model (TerraSeg-B) and efficient Small model (TerraSeg-S).
Tasks
Attributes
Code release
Method
Year
Method type
Seg.
Elev.
Confidence
Real-time
Python
ROS2
RANSAC [ 11 ]
1981
Handcrafted
✓
✓
✗
✓
† N/A †
N/A
GndNet [ 31 ]
2020
Supervised
✓
✓
✗
✓
✓
✗
PatchWork [ 23 ]
2021
Handcrafted
✓
✗
‡ ✗ ‡
✓
✗
✓
JCP [ 34 ]
2021
Handcrafted
✓
✗
✓
✓
✗
✗
PatchWork++ [ 19 ]
2022
Handcrafted
✓
✗
‡ ✗ ‡
✓
✓
✓
Table 1: Overview of main methods for ground segmentation. In the Method type column, Handcrafted refers to rule-based, non-learnable algorithms, while Supervised and Self-supervised indicate learning-based approaches. In the Tasks columns, Seg. and Elev. mean ground Segmentation and Elevation mapping , respectively. † Implemented in TorchRobotics [ 21 ] . ‡ Confidence is bin-level only, used internally for classification, not output as a per-point score. § We obtained source code from the authors but the project is not open source.
Dataset
Year
# LiDARs
Labels
Duration
# Raw scans
# Train scans
# Eval scans
SemanticKITTI [ 2 ]
2019
1
✓
1.3 h
0.04 M
0.4 k
4.1 k
Lyft Level 5 [ 16 ]
2019
4 †
✗
2.8 h
0.05 M
2.3 k
-
nuScenes [ 4 ]
2020
1
✓
5.4 h
0.39 M
3.0 k
6.0 k
Waymo Perception [ 37 ]
2020
5
✓
6.3 h
1.14 M
15.1 k
6.0 k
PandaSet [ 45 ]
2020
2
✓
0.2 h
0.02 M
0.4 k
-
ONCE [ 25 ]
2021
1
✗
158.2 h
1.02 M
101.1 k
-
Table 2: OmniLiDAR dataset. We unify 12 major public autonomous driving datasets into a single standardized LiDAR format, enabling large-scale cross-dataset training and evaluation. We curate the LiDAR scans to obtain a diverse training set by downsampling the LiDAR sequences to 0.2Hz , i.e . one LiDAR scan every five seconds. During training, we mitigate dataset imbalance by sampling with dataset-specific probabilities (see Section 3.2 ). All reported statistics are computed over the available LiDAR data, as some datasets provide only partial sensor releases. † Two different sensor configurations were used for data collection; we report the total number of distinct LiDARs used across configurations. ‡ Only 3D LiDAR sensors are considered. § Across all datasets, we identify 15 unique LiDAR hardware models.
Figure 3: Post-processing for pseudo-labeling. The point cloud is divided into pillars to recover misclassified non-ground points. Points within a vertical window defined by [h^−Hp1,h^+Hp2] around the estimated height h^=g(xi,yi) are analyzed: if a pillar contains both ground and non-ground points, all points above the pillar’s lowest point plus margin τ are reclassified as non-ground.
Ground
Non-Ground
Method
Labels
Recall
Prec.
IoU
Recall
Prec.
IoU
mIoU
Throughput ( Hz )
Supervised
✓
98.15
97.73
95.96
96.97
97.53
94.65
95.31
0 28.0
RANSAC [ 11 ]
✗
96.02
92.56
89.14
88.19
94.60
83.97
86.55
255.0
PatchWork++ [ 19 ]
✗
91.46
93.73
86.19
90.23
89.29
81.42
83.80
0 30.0
Chodosh et al. [ 6 ]
✗
88.53
92.54
82.63
88.93
85.94
77.64
80.13
00 0.1
TRAVEL [ 28 ]
✗
95.48
93.72
89.76
92.11
94.21
87.16
88.46
365.7
Table 3: LiDAR ground segmentation on nuScenes [ 4 ] validation split. We train TerraSeg on OmniLiDAR using pseudo-labels from our PseudoLabeler. GndNet [ 31 ] is trained on the full nuScenes training split, i.e . no sub-sampling. The column Labels indicates whether manually labeled data is used for training. Prec. stands for precision . Best performance in bold , second-best is underlined .
Ground
Non-Ground
Method
Labels
Recall
Prec.
IoU
Recall
Prec.
IoU
mIoU
Throughput ( Hz )
Supervised
✓
96.70
97.26
94.13
97.69
97.21
95.03
94.58
00 9.7
RANSAC [ 11 ]
✗
97.14
79.61
77.79
78.94
97.12
77.13
77.46
255.0
PatchWork++ [ 19 ]
✗
96.57
91.72
88.82
92.15
97.04
89.64
89.23
0 30.0
Chodosh et al. [ 6 ]
✗
95.52
83.53
80.38
83.88
95.82
80.91
80.64
00 0.1
TRAVEL [ 28 ]
✗
95.38
89.77
86.28
92.72
96.98
90.09
88.19
0 85.1
Table 4: LiDAR ground segmentation on SemanticKITTI [ 2 ] validation split. We train TerraSeg on OmniLiDAR using pseudo-labels from our PseudoLabeler. GndNet [ 31 ] is trained on the full SemanticKITTI training split, i.e . no sub-sampling. The column Labels indicates whether manually labeled data is used for training. Prec. stands for precision . Best performance in bold , second-best is underlined .
Ground
Non-Ground
Method
Labels
Recall
Prec.
IoU
Recall
Prec.
IoU
mIoU
Throughput ( Hz )
Supervised
✓
98.05
98.22
96.34
98.81
98.69
97.53
96.94
0 10.1
RANSAC [ 11 ]
✗
91.86
76.14
71.32
81.64
94.02
77.61
74.47
255.0
PatchWork++ [ 19 ]
✗
69.16
92.00
65.23
96.16
83.02
80.36
72.80
0 30.0
Chodosh et al. [ 6 ]
✗
97.09
84.84
82.74
88.94
97.95
87.31
85.03
00 0.1
TRAVEL [ 28 ]
✗
96.20
91.63
88.55
94.81
96.98
92.24
90.40
0 76.9
Table 5: LiDAR ground segmentation on Waymo Perception [ 37 ] validation split. We train TerraSeg on OmniLiDAR using pseudo-labels from our PseudoLabeler. GndNet [ 31 ] is trained on the full Waymo training split, i.e . no sub-sampling. The column Labels indicates whether manually labeled data is used for training. Prec. stands for precision . Best performance in bold , second-best is underlined .
Method
NS
SK
WP
TP
Chodosh et al. [ 6 ]
80.13
80.64
85.03
0.1
PseudoLabeler Base
79.40
80.01
85.00
0.3
+ Pre-processing
86.38
83.82
85.47
0.3
+ Post-processing
90.63
91.97
92.95
0.3
Table 6: PseudoLabeler ablation study I. We progressively add pre-processing and post-processing to the base model. We report mIoU on nuScenes (NS), SemanticKITTI (SK), and Waymo Perception (WP) validation splits. The column TP indicates Throughput (Hz) . Best performance in bold , second-best is underlined .
vxy (m)
τ (m)
NS
SK
WP
Avg
0.25
0.05
89.96
91.11
92.17
91.08
0.25
0.10
89.79
89.87
91.66
90.44
0.25
0.20
88.98
86.98
89.82
88.59
0.50
0.05
90.63
91.97
92.95
91.85
0.50
0.10
90.50
91.60
92.69
91.59
0.50
0.20
89.75
88.76
90.76
89.76
Table 7: PseudoLabeler ablation study II. We ablate the pillar size vxy and recovery threshold τ for post-processing. We report mIoU on nuScenes (NS), SemanticKITTI (SK), and Waymo Perception (WP) validation splits. The column Avg indicates Average mIoU . Best performance in bold , second-best is underlined .
Dataset
σ≤0.40m
σ>0.40m
nuScenes [ 4 ]
93.12
87.81
SemanticKITTI [ 2 ]
93.37
92.60
Waymo Perception [ 37 ]
95.10
90.82
Table 8: TerraSeg ablation study I. We report mIoU across datasets by partitioning validation scans into flat and non-flat, based on the per-scan standard deviation of ground elevation.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Sensor Model
Configuration
Rate
Dataset(s)
Description
Velodyne HDL-64E
64 channels
10Hz
SemanticKITTI [ 2 ] , KITTI-360 [ 22 ]
Classic high-resolution spinning LiDAR with full 360 ∘ FoV.
Velodyne HDL-64 S3
64 channels
10Hz
View-of-Delft [ 32 ]
Newer HDL-64 generation with improved optics and timing.
Velodyne HDL-32E
32 channels
20Hz
nuScenes [ 4 ]
Compact spinning LiDAR with higher temporal resolution.
Velodyne VLP-32C
32 channels
10Hz
Argoverse 2 Lidar [ 39 ]
Automotive-grade spinning LiDAR; dual-mounted in Argoverse 2.
Short-range peripheral LiDARs for side and rear coverage.
Appendix
Table 9: Hardware diversity in OmniLiDAR. A detailed breakdown of the 15 unique LiDAR sensor models utilized across the datasets. The datasets cover a wide spectrum of sensor configurations, including classic spinning LiDARs, solid-state variants, and FMCW sensors, ensuring robust cross-sensor generalization during training.
Figure 4: Qualitative success cases on nuScenes dataset [ 4 ] . The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs 1787 , 3238 , and 3479 .
Figure 5: Qualitative success cases on SemanticKITTI dataset [ 2 ] . The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs 4006 , 2714 , and 2816 .
Figure 6: Qualitative success cases on Waymo Perception dataset [ 37 ] . The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs 3659 , 3045 , and 3505 .
Figure 7: Qualitative failure cases on nuScenes dataset [ 4 ] . The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs 2124 , 4848 , and 62 .
Figure 8: Qualitative failure cases on SemanticKITTI dataset [ 2 ] . The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs 3300 , 947 , and 3173 .
Figure 9: Qualitative failure cases on Waymo Perception dataset [ 37 ] . The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs 2513 , 4755 , and 2024 .
3D point cloud segmentation assigns semantic, instance, or part labels to points for scene understanding, mapping, robotics, and scientific measurement.
In LiDAR-based environment perception systems, ground segmentation is a key preprocessing step supporting various applications such as mapping and navigation. Although extensively studied, problems such as reflection noise and isolated ground remain challenging. To address these issues, we propose FugSeg, a fast uncertainty-aware ground segmentation method. A polar grid map is adopted as the point cloud representation to ensure generalizability across LiDAR types. Building on that, we develop a within- and cross-segment ground labeling strategy that identifies not only directly visible ground cells but also those that are isolated or occluded. During this process, an adaptive slope is introduced, which incorporates measurement uncertainties to enhance its reliability under complex terrain. Finally, to achieve point-level ground segmentation, a fine-grained ground elevation estimation method is introduced. Throughout the complete workflow, reflection noise is explicitly handled via the proposed noisy ground cells. We conduct comprehensive evaluations on four public datasets covering both structured and unstructured environments. Results show that FugSeg outperforms state-of-the-art non-learning methods, achieving the highest F1, accuracy, and mIoU across all datasets, while maintaining the fastest runtime (135 Hz and 487 Hz for 64- and 32-layer LiDARs) using a single CPU thread, making it suitable for resource-limited systems. The code will be available at https://github.com/Leo-YuLi/FugSeg.
Yu Li, Volker Schwieger
Institute of Engineering Geodesy, University of Stuttgart, Stuttgart, Germany
LiDAR serves as a primary sensing modality for robots operating in outdoor environments. However, the performance of deep learning models in this domain is severely limited by the scarcity of labeled data, a direct result of the high cost of 3D annotation. Self-supervised learning addresses this scarcity by learning general-purpose features from unlabeled data. In this work, we present a multi-modal, multi-teacher distillation framework for self-supervised learning on outdoor LiDAR point clouds. Building upon the Sonata architecture, we introduce Vernata, consisting of three extensions: sparse view augmentation to improve robustness against varying point densities, a memory bank mechanism to stabilize resource-constrained training, and cross-modal distillation utilizing dense, high-resolution 2D image features to enable fine-grained semantic guidance. We evaluate our method on the GrandTour, TartanGround, and Waymo datasets, as well as data collected from our own robotic platforms. Our experiments demonstrate a significant performance improvement over Sonata baselines, yielding mIoU scores of 54.7 on TartanGround (+5.9 points, +12.1%) and 57.1 on Waymo (+7.3 points, +14.7%). Finally, we show that the self-supervised approach maintains strong performance even in reduced-modality settings (lacking color or normals), achieving competitive mIoU scores of 49.4 and 50.2 on the respective datasets.
Accurate 3D semantic perception is critical for safe autonomous navigation. However, supervised LiDAR segmentation remains tied to closed taxonomies and to the cost of point-wise manual annotation. Open-vocabulary methods avoid that cost by projecting the output of 2D vision-language models onto LiDAR and distilling it into a 3D network. These methods rely almost exclusively on voxel-based sparse convolutions, and point transformers have so far been limited to indoor environments, where 3D data is dense and bounded. We present Lang3DSeg, which establishes a point transformer as the backbone for annotation-free open-vocabulary segmentation of outdoor 3D LiDAR, and is trained from scratch without geometric pre-training. This training paradigm necessitates addressing the inherent noise in 2D-to-3D label projections; specifically, naive projection often suffers from depth ambiguity, where points behind an object are erroneously assigned its semantic label. We therefore composite masks using an explicit class-priority rule and truncate each projected instance at the first gap in its depth distribution, correcting the projection error directly rather than averaging it over registered sequences. Lang3DSeg achieves 52.8% mIoU on nuScenes validation and 41.4% on SemanticKITTI, the highest among published annotation-free methods on both benchmarks. Every 3D semantic segmentation is on a single LiDAR sweep, and inference operates in real-time without running vision-language models.
Cigdem Kokenoz, Amir Salarpour, Alkim Domeke +5
Department of Automotive Engineering, Clemson University, Greenville, SC, USA. · School of Computing, Clemson University, Clemson, SC, USA.