TerraSeg: Self-Supervised Ground Segmentation for Any LiDAR

Ted Lentsch; Santiago Montiel-Marín; Holger Caesar; Dariu M. Gavrila

cs.CVMar 28, 2026

TerraSeg: Self-Supervised Ground Segmentation for Any LiDAR

Authors: Ted Lentsch, Santiago Montiel-Marín, Holger Caesar, Dariu M. Gavrila

Organizations: Department of Cognitive Robotics, Delft University of Technology · Department of Electronics, University of Alcalá

Abstract

LiDAR perception is fundamental to robotics, enabling machines to understand their environment in 3D. A crucial task for LiDAR-based scene understanding and navigation is ground segmentation. However, existing methods are either handcrafted for specific sensor configurations or rely on costly per-point manual labels, severely limiting their generalization and scalability. To overcome this, we introduce TerraSeg, the first self-supervised, domain-agnostic model for LiDAR ground segmentation. We train TerraSeg on OmniLiDAR, a unified large-scale dataset that aggregates and standardizes data from 12 major public benchmarks. Spanning almost 22 million raw scans across 15 distinct sensor models, OmniLiDAR provides unprecedented diversity for learning a highly generalizable ground model. To supervise training without human annotations, we propose PseudoLabeler, a novel module that generates high-quality ground and non-ground labels through self-supervised per-scan runtime optimization. Extensive evaluations demonstrate that, despite using no manual labels, TerraSeg achieves state-of-the-art results on nuScenes, SemanticKITTI, and Waymo Perception while delivering real-time performance. Our code and model weights are publicly available.

Figures & tables

Figure 2: Overview of TerraSeg. (a) PseudoLabeler generates self-supervised point-wise ground/non-ground labels per raw LiDAR scan. (b) OmniLiDAR unifies 12 major automotive datasets within an aggregated framework, yielding a diverse curated corpus drawn from nearly 22 million raw scans. (c) TerraSeg is a real-time, domain-agnostic model for ground segmentation, trained on OmniLiDAR using these self-supervised pseudo-labels. We introduce two architectures: accurate Base model (TerraSeg-B) and efficient Small model (TerraSeg-S).

			Tasks		Attributes		Code release
Method	Year	Method type	Seg.	Elev.	Confidence	Real-time	Python	ROS2
RANSAC [ 11 ]	1981	Handcrafted	✓	✓	✗	✓	† N/A †	N/A
GndNet [ 31 ]	2020	Supervised	✓	✓	✗	✓	✓	✗
PatchWork [ 23 ]	2021	Handcrafted	✓	✗	‡ ✗ ‡	✓	✗	✓
JCP [ 34 ]	2021	Handcrafted	✓	✗	✓	✓	✗	✗
PatchWork++ [ 19 ]	2022	Handcrafted	✓	✗	‡ ✗ ‡	✓	✓	✓

Table 1: Overview of main methods for ground segmentation. In the Method type column, Handcrafted refers to rule-based, non-learnable algorithms, while Supervised and Self-supervised indicate learning-based approaches. In the Tasks columns, Seg. and Elev. mean ground Segmentation and Elevation mapping , respectively. † Implemented in TorchRobotics [ 21 ] . ‡ Confidence is bin-level only, used internally for classification, not output as a per-point score. § We obtained source code from the authors but the project is not open source.

Dataset	Year	# LiDARs	Labels	Duration	# Raw scans	# Train scans	# Eval scans
SemanticKITTI [ 2 ]	2019	$1$	✓	$1.3$ h	$0.04$ M	$0.4$ k	4.1 k
Lyft Level 5 [ 16 ]	2019	$4$ †	✗	$2.8$ h	$0.05$ M	$2.3$ k	-
nuScenes [ 4 ]	2020	$1$	✓	$5.4$ h	$0.39$ M	$3.0$ k	6.0 k
Waymo Perception [ 37 ]	2020	$5$	✓	$6.3$ h	$1.14$ M	$15.1$ k	6.0 k
PandaSet [ 45 ]	2020	$2$	✓	$0.2$ h	$0.02$ M	$0.4$ k	-
ONCE [ 25 ]	2021	$1$	✗	$158.2$ h	$1.02$ M	$101.1$ k	-

Table 2: OmniLiDAR dataset. We unify

12

major public autonomous driving datasets into a single standardized LiDAR format, enabling large-scale cross-dataset training and evaluation. We curate the LiDAR scans to obtain a diverse training set by downsampling the LiDAR sequences to

0.2\text{\,}\mathrm{Hz}

, i.e . one LiDAR scan every five seconds. During training, we mitigate dataset imbalance by sampling with dataset-specific probabilities (see Section 3.2 ). All reported statistics are computed over the available LiDAR data, as some datasets provide only partial sensor releases. † Two different sensor configurations were used for data collection; we report the total number of distinct LiDARs used across configurations. ‡ Only 3D LiDAR sensors are considered. § Across all datasets, we identify

15

unique LiDAR hardware models.

Figure 3: Post-processing for pseudo-labeling. The point cloud is divided into pillars to recover misclassified non-ground points. Points within a vertical window defined by $[\hat{h}-H_{p1},\,\hat{h}+H_{p2}]$ around the estimated height $\hat{h}=g(x_{i},y_{i})$ are analyzed: if a pillar contains both ground and non-ground points, all points above the pillar’s lowest point plus margin $\tau$ are reclassified as non-ground.

		Ground			Non-Ground
Method	Labels	Recall	Prec.	IoU	Recall	Prec.	IoU	mIoU	Throughput ( $\text{\,}\mathrm{Hz}$ )
Supervised	✓	98.15	97.73	95.96	96.97	97.53	94.65	95.31	0 28.0
RANSAC [ 11 ]	✗	96.02	92.56	89.14	88.19	94.60	83.97	86.55	255.0
PatchWork++ [ 19 ]	✗	91.46	93.73	86.19	90.23	89.29	81.42	83.80	0 30.0
Chodosh et al. [ 6 ]	✗	88.53	92.54	82.63	88.93	85.94	77.64	80.13	00 0.1
TRAVEL [ 28 ]	✗	95.48	93.72	89.76	92.11	94.21	87.16	88.46	365.7

Table 3: LiDAR ground segmentation on nuScenes [ 4 ] validation split. We train TerraSeg on OmniLiDAR using pseudo-labels from our PseudoLabeler. GndNet [ 31 ] is trained on the full nuScenes training split, i.e . no sub-sampling. The column Labels indicates whether manually labeled data is used for training. Prec. stands for precision . Best performance in bold , second-best is underlined .

		Ground			Non-Ground
Method	Labels	Recall	Prec.	IoU	Recall	Prec.	IoU	mIoU	Throughput ( $\text{\,}\mathrm{Hz}$ )
Supervised	✓	96.70	97.26	94.13	97.69	97.21	95.03	94.58	00 9.7
RANSAC [ 11 ]	✗	97.14	79.61	77.79	78.94	97.12	77.13	77.46	255.0
PatchWork++ [ 19 ]	✗	96.57	91.72	88.82	92.15	97.04	89.64	89.23	0 30.0
Chodosh et al. [ 6 ]	✗	95.52	83.53	80.38	83.88	95.82	80.91	80.64	00 0.1
TRAVEL [ 28 ]	✗	95.38	89.77	86.28	92.72	96.98	90.09	88.19	0 85.1

Table 4: LiDAR ground segmentation on SemanticKITTI [ 2 ] validation split. We train TerraSeg on OmniLiDAR using pseudo-labels from our PseudoLabeler. GndNet [ 31 ] is trained on the full SemanticKITTI training split, i.e . no sub-sampling. The column Labels indicates whether manually labeled data is used for training. Prec. stands for precision . Best performance in bold , second-best is underlined .

		Ground			Non-Ground
Method	Labels	Recall	Prec.	IoU	Recall	Prec.	IoU	mIoU	Throughput ( $\text{\,}\mathrm{Hz}$ )
Supervised	✓	98.05	98.22	96.34	98.81	98.69	97.53	96.94	0 10.1
RANSAC [ 11 ]	✗	91.86	76.14	71.32	81.64	94.02	77.61	74.47	255.0
PatchWork++ [ 19 ]	✗	69.16	92.00	65.23	96.16	83.02	80.36	72.80	0 30.0
Chodosh et al. [ 6 ]	✗	97.09	84.84	82.74	88.94	97.95	87.31	85.03	00 0.1
TRAVEL [ 28 ]	✗	96.20	91.63	88.55	94.81	96.98	92.24	90.40	0 76.9

Table 5: LiDAR ground segmentation on Waymo Perception [ 37 ] validation split. We train TerraSeg on OmniLiDAR using pseudo-labels from our PseudoLabeler. GndNet [ 31 ] is trained on the full Waymo training split, i.e . no sub-sampling. The column Labels indicates whether manually labeled data is used for training. Prec. stands for precision . Best performance in bold , second-best is underlined .

Method	NS	SK	WP	TP
Chodosh et al. [ 6 ]	80.13	80.64	85.03	0.1
PseudoLabeler Base	79.40	80.01	85.00	0.3
+ Pre-processing	86.38	83.82	85.47	0.3
+ Post-processing	90.63	91.97	92.95	0.3

Table 6: PseudoLabeler ablation study I. We progressively add pre-processing and post-processing to the base model. We report mIoU on nuScenes (NS), SemanticKITTI (SK), and Waymo Perception (WP) validation splits. The column TP indicates Throughput (Hz) . Best performance in bold , second-best is underlined .

$v_{xy}$ (m)	$\tau$ (m)	NS	SK	WP	Avg
0.25	0.05	89.96	91.11	92.17	91.08
0.25	0.10	89.79	89.87	91.66	90.44
0.25	0.20	88.98	86.98	89.82	88.59
0.50	0.05	90.63	91.97	92.95	91.85
0.50	0.10	90.50	91.60	92.69	91.59
0.50	0.20	89.75	88.76	90.76	89.76

Table 7: PseudoLabeler ablation study II. We ablate the pillar size

v_{xy}

and recovery threshold

\tau

for post-processing. We report mIoU on nuScenes (NS), SemanticKITTI (SK), and Waymo Perception (WP) validation splits. The column Avg indicates Average mIoU . Best performance in bold , second-best is underlined .

Dataset	$\sigma\leq$ $0.40\text{\,}\mathrm{m}$	$\sigma>$ $0.40\text{\,}\mathrm{m}$
nuScenes [ 4 ]	93.12	87.81
SemanticKITTI [ 2 ]	93.37	92.60
Waymo Perception [ 37 ]	95.10	90.82

Table 8: TerraSeg ablation study I. We report mIoU across datasets by partitioning validation scans into flat and non-flat, based on the per-scan standard deviation of ground elevation.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Sensor Model	Configuration	Rate	Dataset(s)	Description
Velodyne HDL-64E	64 channels	$10\text{\,}\mathrm{Hz}$	SemanticKITTI [ 2 ] , KITTI-360 [ 22 ]	Classic high-resolution spinning LiDAR with full 360 ∘ FoV.
Velodyne HDL-64 S3	64 channels	$10\text{\,}\mathrm{Hz}$	View-of-Delft [ 32 ]	Newer HDL-64 generation with improved optics and timing.
Velodyne HDL-32E	32 channels	$20\text{\,}\mathrm{Hz}$	nuScenes [ 4 ]	Compact spinning LiDAR with higher temporal resolution.
Velodyne VLP-32C	32 channels	$10\text{\,}\mathrm{Hz}$	Argoverse 2 Lidar [ 39 ]	Automotive-grade spinning LiDAR; dual-mounted in Argoverse 2.
Velodyne VLS-128	128 channels	$10\text{\,}\mathrm{Hz}$	Zenseact Open [ 1 ]	High-density long-range automotive spinning LiDAR.
Velodyne VLP-16	16 channels	$10\text{\,}\mathrm{Hz}$	Zenseact Open [ 1 ]	Short-range peripheral LiDARs for side and rear coverage.

Appendix

Table 9: Hardware diversity in OmniLiDAR. A detailed breakdown of the

15

unique LiDAR sensor models utilized across the datasets. The datasets cover a wide spectrum of sensor configurations, including classic spinning LiDARs, solid-state variants, and FMCW sensors, ensuring robust cross-sensor generalization during training.

Panel of Figure 4 — Figure 4: Qualitative success cases on nuScenes dataset [ 4 ] . The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs $1787$ , $3238$ , and $3479$ .

Panel of Figure 5 — Figure 5: Qualitative success cases on SemanticKITTI dataset [ 2 ] . The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs $4006$ , $2714$ , and $2816$ .

Panel of Figure 6 — Figure 6: Qualitative success cases on Waymo Perception dataset [ 37 ] . The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs $3659$ , $3045$ , and $3505$ .

Panel of Figure 7 — Figure 7: Qualitative failure cases on nuScenes dataset [ 4 ] . The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs $2124$ , $4848$ , and $62$ .

Panel of Figure 8 — Figure 8: Qualitative failure cases on SemanticKITTI dataset [ 2 ] . The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs $3300$ , $947$ , and $3173$ .

Panel of Figure 9 — Figure 9: Qualitative failure cases on Waymo Perception dataset [ 37 ] . The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs $2513$ , $4755$ , and $2024$ .

3D Point Cloud Segmentation

3D point cloud segmentation assigns semantic, instance, or part labels to points for scene understanding, mapping, robotics, and scientific measurement.

See all 70 papers

FugSeg: Fast Uncertainty-aware Ground Segmentation for 3D Point Cloud
May 9, 2026Yu Li, Volker Schwieger3D Point Cloud Segmentation Autonomous Driving Perception
Institute of Engineering Geodesy, University of Stuttgart, Stuttgart, Germany
Vernata: Self-Supervised Learning of LiDAR Point Representations
Aug 7, 2026Oliver Lemke, Alexander Liniger, Abel Gawel +1Self-Supervised Point Cloud Learning Self-Supervised Learning
1Robotics and AI Institute · 2ETH Zurich
Lang3DSeg: Annotation-Free Open-Vocabulary 3D Segmentation with Point Transformers
Oct 1, 2026Cigdem Kokenoz, Amir Salarpour, Alkim Domeke +5Open-Vocabulary 3D Segmentation Transformer
Department of Automotive Engineering, Clemson University, Greenville, SC, USA. · School of Computing, Clemson University, Clemson, SC, USA.

May 9, 2026J/K move · Enter open · S save

FugSeg: Fast Uncertainty-aware Ground Segmentation for 3D Point Cloud

Yu Li, Volker Schwieger

Institute of Engineering Geodesy, University of Stuttgart, Stuttgart, Germany

3D Point Cloud Segmentation Autonomous Driving Perception

Open paper AskPDF arXiv

In LiDAR-based environment perception systems, ground segmentation is a key preprocessing step supporting various applications such as mapping and navigation. Although extensively studied, problems such as reflection noise and isolated ground remain challenging. To address these issues, we propose FugSeg, a fast uncertainty-aware ground segmentation method. A polar grid map is adopted as the point cloud representation to ensure generalizability across LiDAR types. Building on that, we develop a within- and cross-segment ground labeling strategy that identifies not only directly visible ground cells but also those that are isolated or occluded. During this process, an adaptive slope is introduced, which incorporates measurement uncertainties to enhance its reliability under complex terrain. Finally, to achieve point-level ground segmentation, a fine-grained ground elevation estimation method is introduced. Throughout the complete workflow, reflection noise is explicitly handled via the proposed noisy ground cells. We conduct comprehensive evaluations on four public datasets covering both structured and unstructured environments. Results show that FugSeg outperforms state-of-the-art non-learning methods, achieving the highest F1, accuracy, and mIoU across all datasets, while maintaining the fastest runtime (135 Hz and 487 Hz for 64- and 32-layer LiDARs) using a single CPU thread, making it suitable for resource-limited systems. The code will be available at https://github.com/Leo-YuLi/FugSeg.

Abstract

3D Point Cloud Segmentation

Explore similar work

FugSeg: Fast Uncertainty-aware Ground Segmentation for 3D Point Cloud

Vernata: Self-Supervised Learning of LiDAR Point Representations

Lang3DSeg: Annotation-Free Open-Vocabulary 3D Segmentation with Point Transformers