Person Detection and Tracking from an Overhead Crane LiDAR

Nilusha Jayawickrama; Henrik Toikka; Risto Ojala

cs.CVMar 5, 2026

Person Detection and Tracking from an Overhead Crane LiDAR

Authors: Nilusha Jayawickrama, Henrik Toikka, Risto Ojala

Organizations: School of Engineering, Aalto University, Espoo, Finland · School of Electrical Engineering, Aalto University, Espoo, Finland

Abstract

This paper investigates person detection and tracking in an industrial indoor workspace using a LiDAR mounted on an overhead crane. The overhead viewpoint introduces a strong domain shift from common vehicle-centric LiDAR benchmarks, and limited availability of suitable public training data. Henceforth, we curate a site-specific overhead LiDAR dataset with 3D human bounding-box annotations and adapt selected candidate 3D detectors under a unified training and evaluation protocol. We further integrate lightweight tracking-by-detection using AB3DMOT and SimpleTrack to maintain person identities over time. Detection performance is reported with distance-sliced evaluation to quantify the practical operating envelope of the sensing setup. The best adapted detector configurations achieve average precision (AP) up to 0.84 within a 5.0 m horizontal radius, increasing to 0.97 at 1.0 m, with VoxelNeXt and SECOND emerging as the most reliable backbones across this range. The acquired results contribute in bridging the domain gap between standard driving datasets and overhead sensing for person detection and tracking. We also report latency measurements, highlighting practical real-time feasibility. Finally, we release our dataset and implementations in GitHub to support further research

Figures & tables

Panel of Figure 1 — Fig. 1: Qualitative predictions from the adapted detection modules for two sample frames, comprising two ( (a) - (e) ) and four ( (f) - (j) ) human targets respectively.

Fig. 2: Workspace depicting the installed LiDAR on the overhead crane.

Fig. 4: Proposed model architecture of the detection and tracking framework.

Fig. 5: Architectures of the candidate detectors. Each model is run independently on the same input point clouds.

Fig. 6: Overview of the two tracking-by-detection pipelines, with the inputs being per-frame 3D detections over time.

Panel of Figure 7 — Fig. 7: Training and validation plot for each detector based on the architecture denoted in the captions (a) - (e) , obtained by the common data splits of the compiled overhead-view LiDAR dataset.

	Precision					Recall					F1-score					AP					mIoU
r (m)	PP	SEC	PV- RCNN	Voxel Next	Voxel RCNN	PP	SEC	PV- RCNN	Voxel Next	Voxel RCNN	PP	SEC	PV- RCNN	Voxel Next	Voxel RCNN	PP	SEC	PV- RCNN	Voxel Next	Voxel RCNN	PP	SEC	PV- RCNN	Voxel Next	Voxel RCNN
1.0	1.000	1.000	1.000	1.000	1.000	0.769	0.828	0.448	0.966	0.069	0.870	0.906	0.619	0.982	0.129	0.713	0.823	0.448	0.966	0.069	0.477	0.586	0.544	0.550	0.530
2.0	0.974	1.000	1.000	0.983	0.500	0.603	0.887	0.403	0.983	0.046	0.745	0.940	0.575	0.983	0.085	0.573	0.846	0.385	0.892	0.023	0.427	0.557	0.550	0.560	0.543
3.0	0.974	1.000	1.000	0.896	0.643	0.574	0.882	0.290	0.868	0.069	0.722	0.937	0.450	0.882	0.125	0.547	0.833	0.285	0.827	0.054	0.452	0.554	0.538	0.545	0.556
4.0	0.967	0.994	1.000	0.907	0.762	0.442	0.848	0.242	0.737	0.081	0.606	0.915	0.390	0.813	0.146	0.452	0.843	0.242	0.728	0.070	0.445	0.535	0.536	0.519	0.480
5.0	0.967	0.935	1.000	0.796	0.762	0.433	0.847	0.241	0.729	0.079	0.599	0.889	0.389	0.761	0.143	0.451	0.837	0.241	0.706	0.068	0.442	0.535	0.536	0.517	0.480

TABLE I: Evaluation metrics for the adapted detection modules on the test split (true positive if IoU

\geq 0.10

\mathbf{r}

denotes the horizontal radial distance from the LiDAR, and each row reports cumulative performance for human targets within that radius.

	Precision		Recall		F1-score		AP		mIoU
Dist. (m)	Voxel NeXt	SEC	Voxel NeXt	SEC	Voxel NeXt	SEC	Voxel NeXt	SEC	Voxel NeXt	SEC
1.0	1.000	0.889	0.483	0.286	0.651	0.432	0.435	0.272	0.497	0.508
2.0	0.950	0.968	0.594	0.462	0.731	0.625	0.546	0.451	0.522	0.525
3.0	0.944	0.974	0.515	0.573	0.667	0.721	0.501	0.557	0.534	0.521
4.0	0.808	0.694	0.426	0.515	0.558	0.591	0.393	0.446	0.513	0.516
5.0	0.635	0.688	0.429	0.522	0.512	0.594	0.367	0.471	0.505	0.519

TABLE II: Detection performance of the pretrained models (no fine-tuning) on the overhead-view LiDAR test split.

		IoU = 0.3			IoU = 0.1
Tracker	Detector	MOTA	IDF1	MOTP	MOTA	IDF1	MOTP
	PP	0.12	0.53	0.46	0.38	0.68	0.40
	SEC	0.33	0.72	0.58	0.57	0.83	0.54
AB3DMOT	VoxelNeXt	0.70	0.87	0.55	0.83	0.93	0.53
	PP	0.26	0.58	0.46	0.52	0.74	0.41
	SEC	0.41	0.74	0.57	0.60	0.83	0.54

TABLE III: Tracking performance for the modules adapted from AB3DMOT and SimpleTrack on the temporal sequences.

	Detection					Tracking
	PP	SEC	PV- RCNN	Voxel NeXt	Voxel RCNN	AB3DMOT	SimpleTrack
p50 (ms)	34	46	229	35	32	1.08	6.30
p90 (ms)	37	47	232	36	38	1.26	6.66

TABLE IV: Comparison of the obtained inference speeds (via percentiles) for the adapted modules, executed on a CPU (Intel Xeon Gold 6148 @ 2.40 GHz)

Lidar-Based 3D Object Detection

Lidar-based 3D object detection locates and classifies objects in point clouds, supporting perception for autonomous vehicles, robots, and other mobile systems.

See all 51 papers

TriSAR-3D and PaintFusion: A Novel Trimodal LiDAR-RGB-Thermal Dataset and Method for 3D Person Detection in Search and Rescue
Oct 15, 2026Markus Kramer, Jan Wachenfeld, Oskar von Stryk3D Object Detection Multimodal Object Detection
The authors are with the SIM Group, Technical University of Darmstadt, Germany.
IndoorCrowd: A Multi-Scene Dataset for Human Detection, Segmentation, and Tracking with an Automated Annotation Pipeline
Apr 2, 2026Sebastian-Ion Nae, Radu Moldoveanu, Alexandra Stefania Ghita +1Instance Segmentation Multi-Object Tracking
National University of Science and Technology Politehnica Bucharest, Romania · Expleo, Romania
Factorized Spatio-Temporal Convolutions for Human Pose Estimation from Planar Lidar
Jul 23, 2026Simone Arreghini, Mirko Nava, Nicholas Carlotti +2Relative Pose Estimation Human Pose Estimation
Dalle Molle Institute for Artificial Intelligence (IDSIA), USI-SUPSI, Lugano, Switzerland

Oct 15, 2026J/K move · Enter open · S save

TriSAR-3D and PaintFusion: A Novel Trimodal LiDAR-RGB-Thermal Dataset and Method for 3D Person Detection in Search and Rescue

Markus Kramer, Jan Wachenfeld, Oskar von Stryk

The authors are with the SIM Group, Technical University of Darmstadt, Germany.

3D Object Detection Multimodal Object Detection Camera-LiDAR Fusion Search-and-Rescue Robotics

Open paper AskPDF arXiv

Reliable 3D person detection is important for rescue robotics to give first responders situational awareness. Yet most multimodal 3D detection research targets automotive scenarios in which pedestrians mostly appear in a narrow range of upright, near-road poses. It is therefore unclear how much of the reported performance transfers to rescue settings, as a person may be lying, partially occluded, or in unstructured indoor and outdoor environments. To investigate this gap, we introduce TriSAR-3D, a multimodal person detection dataset recorded with a high-resolution 128-channel LiDAR, an RGB camera, and a thermal camera. It captures varied poses in indoor as well as outdoor locations at different times of day to study their influence on 3D object detector performance. Furthermore, we propose a novel trimodal 3D detection pipeline, PaintFusion, that fuses PointPainting-based evidence using a simple MLP. We show that the derived method reaches 65.4 AP@50 and thereby outperforms strong LiDAR-only (CenterPoint, 53.4), RGB+LiDAR (BEVFusion, 60.5), and adapted trimodal (Tri-BEVFusion, 60.7) baselines on the dataset. At the same time, it is substantially faster than other fusion-based methods such as BEVFusion, with a per-frame latency of 51.6 ms compared to 101 ms. Additionally, we test how the performance of different 3D object detectors generalizes to unseen pose categories. Removing lying instances from training data collapses the lying-pose AP@50 to near zero across all evaluated detectors. The results indicate that dataset diversity exerts a large effect on 3D object detection performance. The dataset and code are available at https://github.com/MarkusKramer1/3D-Trimodal-People-Detection.

Abstract

Lidar-Based 3D Object Detection

Explore similar work

TriSAR-3D and PaintFusion: A Novel Trimodal LiDAR-RGB-Thermal Dataset and Method for 3D Person Detection in Search and Rescue

IndoorCrowd: A Multi-Scene Dataset for Human Detection, Segmentation, and Tracking with an Automated Annotation Pipeline

Factorized Spatio-Temporal Convolutions for Human Pose Estimation from Planar Lidar