Provisioning Robust and Efficient LiDAR Perception for Autonomous Driving
Authors: Tingxu Han, Zijie Shen, An Guo, Lyuke Wang, Chunrong Fang, Zhenyu Chen, Wei Song
Organizations: State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210093, China. · Department of Computing, Hong Kong Polytechnic University, Hong Kong SAR, China · School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China · Information and Communication Technology School, Griffith University, Gold Coast, QLD, 4215, Australia
LiDAR-based 3D object detection is a core perception service for autonomous driving, but deployment requires robustness to point-cloud corruption while meeting latency and memory requirements. Existing approaches address robustness and resource efficiency separately, resulting in compact detectors that are fragile under corruption or robust detectors that remain costly to deploy. We therefore formulate LiDAR perception provisioning as constructing a detector that remains accurate under point-cloud corruptions while meeting constraints on inference latency and peak GPU memory. To achieve robustness, we first craft corrupted samples that mimic extreme weather conditions. Then we train candidate detectors offline by ordering clean and corrupted samples by perturbation sensitivity, through which detectors learn more from hard corrupted samples. We estimate perturbation sensitivity using an ensemble of seven heterogeneous 3D detectors by tracking changes in confidence, BEV overlap, and unmatched detections. To improve efficiency, we narrow the network and use offline neighborhood-density measurements to determine how many points and neighbors each sampling stage processes. We evaluate our method using AP (Average Precision) under clean and corrupted conditions, inference latency, and peak allocated GPU memory. On KITTI for Car detection with PointRCNN, our method improves Moderate AP under corruption by 3.32 points over the original detector while reducing mean latency by 19.8% and peak allocated GPU memory by 53.9%. Across two datasets, two detector architectures, and three object classes, our method improves AP under corruption by 4.03 points on average over the corresponding original detectors, while reducing mean inference latency and peak allocated GPU memory by 14.4% and 38.0%, respectively. Our code is available at https://github.com/drunklibai/robust-efficient-lidar-perception.
Figures & tables
Fig. 1: Offline profile-construction pipeline. Paired clean and corrupted point clouds are scored by an ensemble reference for clean-versus-corrupted prediction inconsistency. The reordered training data are combined with a density-guided sampling-budget allocation to train a compact detector.
Fig. 2: Case study of perturbation sensitivity. We compare ensemble predictions on an original point cloud and its corrupted version. Differences in false predictions, confidence scores, missing detections, and object locations are combined into a frame-level sensitivity score. A higher score indicates a harder sample that appears later in the prioritized training order.
Class
Method
KITTI Moderate
ONCE Range Avg.
Clean
Corr.
Clean
Corr.
PointRCNN
Car
Original
80.65
61.03
43.78
25.02
Net. Slim.
77.79
56.78
42.95
30.37
Ours
80.04
64.35
43.47
32.20
Pedestrian
Original
54.81
43.86
4.95
4.19
TABLE I: Detection Accuracy across Datasets, Detectors, and Classes
KITTI
ONCE
Detector
Method
Clean
Corrupted
Clean
Corrupted
Easy
Moderate
Hard
Easy
Moderate
Hard
0–30m
30–50m
> 50m
0–30m
30–50m
> 50m
PointRCNN
Original
91.62
80.65
78.07
77.08
61.03
56.90
73.83
39.83
17.68
49.60
17.85
7.62
Retrain
88.71
80.01
77.35
82.13
64.73
60.08
73.21
39.80
17.44
64.55
24.31
7.74
GMP
74.02
62.84
60.37
62.53
49.16
43.84
57.34
27.63
12.65
45.92
16.46
5.52
LTH/IMP
89.11
77.85
75.15
79.56
62.31
57.72
70.96
37.13
14.62
60.02
21.83
5.31
TABLE II: Car Detection Accuracy on Clean and Corrupted Data
KITTI
ONCE
Detector
Method
Mean latency
p95 latency
Peak memory
Mean latency
p95 latency
Peak memory
PointRCNN
Original
47.96/48.88
49.27/50.79
408.1/407.7
669.30/670.48
677.65/695.51
633.3/633.3
Retrain
46.92/46.91
48.71/48.74
408.1/407.7
669.17/670.52
677.04/696.29
633.3/633.3
GMP
47.03/47.65
48.74/49.97
440.4/440.4
669.92/670.40
677.27/687.38
666.0/666.0
LTH/IMP
47.50/47.59
50.03/49.77
440.4/440.4
669.80/670.23
677.86/678.99
666.0/666.0
Net. Slim.
45.55/45.82
47.56/48.34
331.4/331.4
666.93/666.26
675.05/673.82
468.8/468.8
TABLE III: Inference Latency and Peak Allocated GPU Memory on the RTX 4090D
Clean
Corrupted
Training order
E
M
H
E
M
H
Random order
88.05
79.59
77.06
82.07
64.86
61.98
Confidence-based
89.32
80.66
77.96
81.86
66.58
62.20
Perturbation-sensitivity
91.27
82.12
79.50
84.42
67.12
62.52
TABLE IV: Effect of Training Order on KITTI Car with Full-Size PointRCNN (AP R40 , IoU 0.7)
Lidar-based 3D object detection locates and classifies objects in point clouds, supporting perception for autonomous vehicles, robots, and other mobile systems.
Autonomous driving perception is typically evaluated on clean benchmark data, yet real-world deployment requires robustness to rare, structured, and potentially adversarial sensor anomalies. This gap is especially critical for LiDAR, where external actors can physically manipulate the sensing process to induce black-box perception failures without accessing the model. Existing LiDAR benchmarks provide little visibility into this failure mode. Prior adversarial LiDAR studies have largely centered on attack hardware, geometric and algorithmic defenses, and early-generation detectors, leaving the robustness of modern perception systems unexplored. To address this evaluation gap, we introduce ATLAS (Adversarial Temporal LiDAR Attack Suite), the first large-scale, physically grounded evaluation benchmark for LiDAR perception models under black-box sensor attacks, simulating the two primary attack modes -- point injection and point removal -- across real driving sequences. Evaluating a broad cross-section of current state-of-the-art LiDAR perception models, ATLAS reveals a surprising robustness asymmetry: models with stronger performance on standard benchmarks tend to better withstand removal attacks, yet are actually more vulnerable to injection attacks than weaker models. We trace this vulnerability to standard object database sampling augmentations, revealing how current training practices can induce architecture-agnostic robustness failures, and study initial directions for mitigating both attack modes. We release the ATLAS generation code to support extensible, reproducible evaluations as attack capabilities evolve, helping make black-box sensor robustness an explicit consideration in future LiDAR perception development.
Recent advancements in LiDAR-only 3D object detection have demonstrated improved detection accuracy over benchmark datasets. However, the adversarial robustness of these models remains untested. Very few adversarial robustness studies exist for LiDAR-only 3D object detection and unfortunately, even they are limited to legacy models. Moreover, there is a systemic gap in the existing evaluation frameworks that rely simply on mAP ignoring other structural and predictive factors. To fill this gap, we propose a holistic framework that evaluates adversarial robustness using two structural factors (point cloud density and point cloud localization) and three predictive factors (misclassification, localization error, distance from ego). Using this framework, we perform an empirical study and critical analysis on recent and legacy state-of-the-art models using adversarial attacks specifically designed for LiDAR-based models. Our key finding is that high-capacity, voxel-based detectors are more susceptible to structured coordinate perturbations than pillar-based detectors. Additionally, non-anchor-based detectors demonstrate poor adversarial robustness, which necessitates rethinking model training techniques. Overall, our results demonstrate that recent models are as vulnerable to adversarial attacks as their predecessors. Therefore, we argue that there is a need to improve the evaluation benchmarks for 3D object detection that not only reward architectural modifications for improving detection accuracy, but also evaluate whether the design choices improve adversarial robustness.
Adwait Chandorkar, Kai Krink, Yerdana Maulenbay +2
Institute for TMDT, University of Wuppertal, Germany · IKB Faculty of Science, University of British Columbia, Canada
The structural vulnerabilities of point cloud-based 3D object detectors remain poorly understood. Prior work has studied adversarial robustness primarily on isolated 3D object models, while recent LiDAR spoofing attacks target richer and more realistic driving scenes but focus mainly on physical realizability rather than understanding detector behavior or attack efficiency. In this work, we investigate how LiDAR-based detectors rely on spatial evidence in complex scenes and whether these reliance patterns can be exploited to induce failures more efficiently. To this end, we propose an explainability-guided adversarial analysis methodology. We introduce the Saliency-LiDAR (SALL) method, which aggregates Integrated Gradient attributions across scenes to produce universal saliency maps for LiDAR-based 3D object detectors. Guided by these maps, we design the Explainability-aware Frustum Attack (EFA), which selectively perturbs only the most influential frustums rather than uniformly attacking entire object regions. Experiments on KITTI and nuScenes, across detectors such as PointPillars and SECOND, show that EFA reduces detection recall by more than 15 percentage points while requiring 25-50% fewer perturbed frustums than the state-of-the-art non-saliency-aware baseline. These findings reveal that modern 3D detectors concentrate discriminative evidence in a small subset of spatial regions, exposing a structural robustness vulnerability in current LiDAR perception systems. Our code is released at https://github.com/SecMindLab/Saliency_LiDAR.