Lidar-Based 3D Object Detection

Overview

Lidar-based 3D object detection turns sparse, irregular point clouds into 3D bounding boxes and class labels, helping mobile systems understand nearby objects and their positions. Common approaches organize points as voxels or pillars, process them directly or in bird’s-eye-view grids, and use convolutional networks, transformers, or newer sequence models; some combine lidar with cameras, radar, or cooperative vehicle data. Current work also addresses long-range and rare-object detection, low-latency inference, adaptation across sensors and environments, and robustness to weather, corruption, and adversarial point changes. A central tension is improving accuracy under sparse or shifted data while keeping computation, sensing, and communication within practical limits.

Latest papers

63 papers11 papers in the last four weeks, up 120% on the four weeks before. 0.1% of all new papers.

Mar 5, 2026

Person Detection and Tracking from an Overhead Crane LiDAR

This paper investigates person detection and tracking in an industrial indoor workspace using a LiDAR mounted on an overhead crane. The overhead viewpoint introduces a strong domain shift from common vehicle-centric LiDAR benchmarks, and limited availability of suitable public training data. Henceforth, we curate a site-specific overhead LiDAR dataset with 3D human bounding-box annotations and adapt selected candidate 3D detectors under a unified training and evaluation protocol. We further integrate lightweight tracking-by-detection using AB3DMOT and SimpleTrack to maintain person identities over time. Detection performance is reported with distance-sliced evaluation to quantify the practical operating envelope of the sensing setup. The best adapted detector configurations achieve average precision (AP) up to 0.84 within a 5.0 m horizontal radius, increasing to 0.97 at 1.0 m, with VoxelNeXt and SECOND emerging as the most reliable backbones across this range. The acquired results contribute in bridging the domain gap between standard driving datasets and overhead sensing for person detection and tracking. We also report latency measurements, highlighting practical real-time feasibility. Finally, we release our dataset and implementations in GitHub to support further research
Jun 13, 2025

Teleoperated Driving: a New Challenge for 3D Object Detection in Compressed Point Clouds

In recent years, the development of interconnected devices has expanded in many fields, from infotainment to education and industrial applications. This trend has been accelerated by the increased number of low-cost sensors and accessibility to powerful hardware and software. One area that significantly benefits from these advancements is Teleoperated Driving. In this scenario, a remote controller (human or software) safely drives a vehicle using sensor data generated onboard the vehicle, and exchanged via Vehicle-to-Everything (V2X) communications. In this work, we tackle the problem of detecting the presence of cars, pedestrians, and cyclists from point cloud data generated by LiDAR sensors to enable safe Teleoperated Driving operations. Specifically, we use the SELMA dataset, a multimodal, open-source, synthetic dataset for autonomous driving, that we expanded by including the ground-truth bounding boxes of 3D objects to support object detection. We analyze the performance of state-of-the-art compression algorithms and object detectors under several metrics, including compression efficiency, (de)compression and inference time, and detection accuracy. Finally, we measure the impact of compression and detection on a simulated V2X network in terms of data rate and latency with respect to the 3GPP requirements of Teleoperated Driving applications. Our results show that G-PCC outperforms its competitors in terms of both compression efficiency and accuracy, and consumes fewer channel resources for data transmissions. On the contrary, the two most conservative quantization modes of Draco, combined with SECOND or PointPillars for object detection, offer superior speed in both compression and decompression with comparable accuracy. Finally, we show that the choice of the detector depends on the class of objects to detect, specifically pedestrians or cyclists.
Jun 3, 2025

AutoExpert: Automating 3D LiDAR Annotation from Expert-Crafted Guidelines

The contemporary paradigm of scaling data annotation, crucial for developing machine learning solutions, is to hire ordinary human annotators and instruct them with expert-crafted guidelines to label data. This paradigm is laborious, tedious, and costly, motivating us to study an open problem, auto-annotation with expert-crafted guidelines (dubbed AutoExpert). We develop benchmarks by redesigning the evaluation protocol and re-annotating data with nuScenes and PandaSet, two 3D detection datasets for autonomous driving research that provide expert-crafted annotation guidelines. Their guidelines define 18 and 25 object classes, respectively, using nuanced language descriptions and a few visual examples. Following the guidelines that require using 3D cuboids to label LiDAR data, AutoExpert requires algorithms to learn on few-shot labeled images and texts to perform the task of 3D detection on LiDAR data. Apparently, the challenges of AutoExpert lie in the data-modality and task discrepancy. Nevertheless, public foundation models (FMs) serve as promising tools to tackle these challenges. To address AutoExpert, we adopt a conceptually simple pipeline consisting of three components: (1) 2D object detection and segmentation in RGB images, (2) lifting 2D detections into 3D using known sensor poses, and (3) 3D cuboids generation for the 2D detections. Within this pipeline, we enhance and evaluate a variety of methods such as open-vocabulary detectors, few-shot detectors, and self-supervised learned detectors. We also develop novel techniques, leading to refined components that boost 3D detection mAP from 12.1 to 25.4 on the AutoExpert-nuScenes benchmark.