Annotation-Free Detection of Drivable Areas and Curbs Leveraging LiDAR Point Cloud Maps

Fulong Ma; Daojie Peng; Jun Ma

cs.CVMar 29, 2026

Annotation-Free Detection of Drivable Areas and Curbs Leveraging LiDAR Point Cloud Maps

Authors: Fulong Ma, Daojie Peng, Jun Ma

Organizations: The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China

Abstract

Drivable areas and curbs are critical traffic elements for autonomous driving, forming essential components of the vehicle visual perception system and ensuring driving safety. Deep neural networks (DNNs) have significantly improved perception performance for drivable area and curb detection, but most DNN-based methods rely on large manually labeled datasets, which are costly, time-consuming, and expert-dependent, limiting their real-world application. Thus, we developed an automated training data generation module. Our previous work generated training labels using single-frame LiDAR and RGB data, suffering from occlusion and distant point cloud sparsity. In this paper, we propose a novel map-based automatic data labeler (MADL) module, combining LiDAR mapping/localization with curb detection to automatically generate training data for both tasks. MADL avoids occlusion and point cloud sparsity issues via LiDAR mapping, creating accurate large-scale datasets for DNN training. In addition, we construct a data review agent to filter the data generated by the MADL module, eliminating low-quality samples. Experiments on the KITTI, KITTI-CARLA and 3D-Curb datasets show that MADL achieves impressive performance compared to manual labeling, and outperforms traditional and state-of-the-art self-supervised methods in robustness and accuracy.

Figures & tables

Fig. 1: We use traditional methods to detect curbs from single-frame point cloud, and then leverage SLAM technology to construct curb maps and drivable area maps. These maps are used in subsequent steps to retrieve curb and drivable area information through localization within the map.

Fig. 2: Given the point cloud input at a certain moment, the positional information at that moment is obtained through point cloud localization. Then, curb points and drivable area points within a specified range are retrieved from the map. The retrieved curb points can be directly used as training data for the curb detection task. The retrieved drivable area point cloud is projected onto the image plane using camera intrinsics, and through post-processing, training data for the drivable area detection task can be obtained.

Fig. 3: The schematic diagram of our Data Review Agent. It takes as input three images from different modalities (a point cloud top-down view, an RGB image, and an altitude difference image), each containing curb detection information. Based on the quality of the curb detection, the agent then outputs whether the detection data from that frame should be retained.

Method	Accuracy $\uparrow$	Precision $\uparrow$	Recall $\uparrow$	F1-score $\uparrow$	IoU $\uparrow$
TDG / SSLG	87.81	59.59	84.03	69.73	53.54
ADL	93.12	90.68	91.78	91.23	85.36
MADL (Ours)	97.34	93.92	94.78	94.35	92.29

TABLE I: The comparison results of our MADL with TDG [ 18 ] / SSLG [ 29 ] and ADL [ 15 ] on the KITTI dataset.

Method	Accuracy $\uparrow$	Precision $\uparrow$	Recall $\uparrow$	F1-score $\uparrow$	IoU $\uparrow$
TDG / SSLG	88.79	79.52	82.41	80.94	68.01
ADL	94.13	91.31	92.38	91.84	86.24
MADL (Ours)	97.25	94.42	95.84	95.13	93.87

TABLE II: The comparison results of our MADL with TDG [ 18 ] / SSLG [ 29 ] and ADL [ 15 ] on the KITTI-CARLA dataset.

Network	Network Architecture	Method	Accuracy $\uparrow$	Precision $\uparrow$	Recall $\uparrow$	F1-score $\uparrow$	mIoU $\uparrow$
U-Net	CNN	TDG / SSLG	90.13	80.15	83.32	82.18	71.80
		ADL	91.43	86.12	81.65	83.83	73.96
		MADL (Ours)	92.76	87.93	82.03	84.88	74.02
		Ground Truth	95.56	89.86	82.95	86.27	75.13
SegFormer	Transformer	TDG / SSLG	90.43	81.43	84.02	82.70	72.11
		ADL	92.44	88.84	90.29	89.56	85.71

TABLE III: Comparison of training results of three semantic segmentation models—U-Net [ 20 ] , SegFormer [ 31 ] , and VM-UNet [ 21 ] —on the training data generated by TDG [ 18 ] / SSLG [ 29 ] , ADL [ 15 ] , and our MADL module on the KITTI dataset, as well as on the ground truth. The results trained on ground truth are marked in teal and are not compared with the baseline methods, they are provided for reference only.

Fig. 4: The qualitative comparison results of our MADL with TDG [ 18 ] / SSLG [ 29 ] and ADL [ 15 ] on the KITTI and KITTI-CARLA datasets. (a): Comparison on the KITTI dataset. (b): Comparison on the KITTI-CARLA dataset.

Network	Network Architecture	Method	Accuracy $\uparrow$	Precision $\uparrow$	Recall $\uparrow$	F1-score $\uparrow$	mIoU $\uparrow$
U-Net	CNN	TDG / SSLG	89.36	80.68	83.97	82.29	72.82
		ADL	91.32	87.75	88.36	88.05	78.53
		MADL (Ours)	92.33	89.12	90.89	90.00	83.44
		Ground Truth	93.03	90.22	92.75	91.47	84.12
SegFormer	Transformer	TDG / SSLG	90.56	81.32	84.37	82.82	73.58
		ADL	93.65	89.17	90.43	89.80	86.12

TABLE IV: Comparison of training results of three semantic segmentation models—U-Net [ 20 ] , SegFormer [ 31 ] , and VM-UNet [ 21 ] —on the training data generated by TDG [ 18 ] / SSLG [ 29 ] , ADL [ 15 ] , and our MADL module on the KITTI-CARLA dataset, as well as on the ground truth. The results trained on ground truth are marked in teal and are not compared with the baseline methods, they are provided for reference only.

Method	Training Data	Precision $\uparrow$	Recall $\uparrow$	F1-Score $\uparrow$
PVKD	3D-Curb	93.98	94.17	94.07
PVKD	MADL (Ours)	93.35	94.53	93.94
CurbNet	3D-Curb	97.38	95.01	96.17
CurbNet	MADL (Ours)	96.87	95.31	96.08
Cylinder3D	3D-Curb	93.60	93.55	92.57
Cylinder3D	MADL (Ours)	91.98	92.39	92.18

TABLE V: Performance comparison of four learning-based models—PVKD [ 10 ] , Cylinder3D [ 41 ] , and CurbNet [ 39 ] —after training on the data generated by our MADL versus the 3D-Curb dataset [ 39 ] , with a tolerance error of 0.2 meters.

Data Review Agent	Accuracy $\uparrow$	Precision $\uparrow$	Recall $\uparrow$	F1-score $\uparrow$	IoU $\uparrow$
w/o	95.63	92.17	93.15	92.66	90.38
w/	97.34	93.92	94.78	94.35	92.29

TABLE VI: Ablation study on the effectiveness of the Data Review Agent conducted on the KITTI dataset.

Autonomous Driving Perception

Autonomous driving perception enables vehicles to detect and interpret road users, drivable space, and hazards using onboard and shared sensors.

See all 379 papers

AutoExpert: Automating 3D LiDAR Annotation from Expert-Crafted Guidelines
Jun 3, 2025Yechi Ma, Wei Hua, Shu KongAutonomous Driving Datasets Lidar-Based 3D Object Detection
Zhejiang University · University of Macau · IAIBS
RS2AD-LiDAR: End-to-End Autonomous Driving LiDAR Data Generation from Roadside Sensor Observations
May 22, 2026Runyi Huang, Ni Ding, Ruidan Xing +3Synthetic Data Augmentation Autonomous Driving Perception
State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing, 100084, China. · School of Vehicle and Mobility, Tsinghua University, Beijing, 100084, China. · College of Artificial Intelligence, Tsinghua University, Beijing, 100083, China. +2
Provisioning Robust and Efficient LiDAR Perception for Autonomous Driving
Oct 20, 2026Tingxu Han, Zijie Shen, An Guo +4Lidar-Based 3D Object Detection Autonomous Driving Perception
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210093, China. · Department of Computing, Hong Kong Polytechnic University, Hong Kong SAR, China · School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China +1

Jun 3, 2025J/K move · Enter open · S save

AutoExpert: Automating 3D LiDAR Annotation from Expert-Crafted Guidelines

Yechi Ma, Wei Hua, Shu Kong

Zhejiang University · University of Macau · IAIBS

Autonomous Driving Datasets Lidar-Based 3D Object Detection Few-Shot Object Detection 3D Object Detection Object Detection

Open paper AskPDF arXiv

The contemporary paradigm of scaling data annotation, crucial for developing machine learning solutions, is to hire ordinary human annotators and instruct them with expert-crafted guidelines to label data. This paradigm is laborious, tedious, and costly, motivating us to study an open problem, auto-annotation with expert-crafted guidelines (dubbed AutoExpert). We develop benchmarks by redesigning the evaluation protocol and re-annotating data with nuScenes and PandaSet, two 3D detection datasets for autonomous driving research that provide expert-crafted annotation guidelines. Their guidelines define 18 and 25 object classes, respectively, using nuanced language descriptions and a few visual examples. Following the guidelines that require using 3D cuboids to label LiDAR data, AutoExpert requires algorithms to learn on few-shot labeled images and texts to perform the task of 3D detection on LiDAR data. Apparently, the challenges of AutoExpert lie in the data-modality and task discrepancy. Nevertheless, public foundation models (FMs) serve as promising tools to tackle these challenges. To address AutoExpert, we adopt a conceptually simple pipeline consisting of three components: (1) 2D object detection and segmentation in RGB images, (2) lifting 2D detections into 3D using known sensor poses, and (3) 3D cuboids generation for the 2D detections. Within this pipeline, we enhance and evaluate a variety of methods such as open-vocabulary detectors, few-shot detectors, and self-supervised learned detectors. We also develop novel techniques, leading to refined components that boost 3D detection mAP from 12.1 to 25.4 on the AutoExpert-nuScenes benchmark.

Abstract

Autonomous Driving Perception

Explore similar work

AutoExpert: Automating 3D LiDAR Annotation from Expert-Crafted Guidelines

RS2AD-LiDAR: End-to-End Autonomous Driving LiDAR Data Generation from Roadside Sensor Observations

Provisioning Robust and Efficient LiDAR Perception for Autonomous Driving