ALOOD: Exploiting Language Representations for LiDAR-based Out-of-Distribution Object Detection
Authors: Michael Kösel, Marcel Schreiber, Michael Ulrich, Claudius Gläser, Klaus Dietmayer
Organizations: Institute of Measurement, Control, and Microtechnology, Ulm University, Germany · Robert Bosch GmbH, Corporate Research, 71272 Renningen, Germany
LiDAR-based 3D object detection plays a critical role for reliable and safe autonomous driving systems. However, existing detectors often produce overly confident predictions for objects not belonging to known categories, posing significant safety risks. This is caused by so-called out-of-distribution (OOD) objects, which were not part of the training data, resulting in incorrect predictions. To address this challenge, we propose ALOOD (Aligned LiDAR representations for Out-Of-Distribution Detection), a novel approach that incorporates language representations from a vision-language model (VLM). By aligning the object features from the object detector to the feature space of the VLM, we can treat the detection of OOD objects as a zero-shot classification task. We demonstrate competitive performance on the nuScenes OOD benchmark, establishing a novel approach to OOD object detection in LiDAR using language representations. The source code is available at https://github.com/uulm-mrm/mmood3d.
Figures & tables
Fig. 1 : The object features of the LiDAR object detector are aligned to match the embeddings of a frozen text encoder. This allows us to use the zero-shot classification capabilities of VLMs to perform OOD detection.
Fig. 2 : Overview of our proposed ALOOD framework. Given a frozen LiDAR object detector, we extract features. These object-specific features are aligned to text features from a language model by generating text descriptions for each object. During inference, these aligned object features are compared to cached ID text embeddings using cosine-similarity.
CenterPoint
Method
FPR-95 ↓
AUROC ↑
AUPR-S ↑
AUPR-E ↑
Voxel
Default Score
63.00
85.10
99.70
7.91
MSP [ 1 ]
44.60
88.87
99.79
13.74
ODIN [ 2 ]
48.63
87.82
99.77
11.05
MaxLogit [ 3 ]
55.85
85.89
99.71
10.02
Energy [ 4 ]
73.80
82.15
99.64
5.27
Rescaling [ 11 ]
36.96
88.96
99.73
24.68
TABLE I : OOD detection results on the nuScenes OOD benchmark. Best results are in bold . Second best are underlined .
Alignment Model
FPR-95 ↓
AUROC ↑
AUPR-S ↑
AUPR-E ↑
Linear
37.26
90.15
99.81
21.52
MLP (ReLU)
44.86
88.27
99.76
15.67
MLP (no ReLU)
42.50
89.32
99.77
17.11
TABLE II : Comparison of different modules for aligning the object features to the text features.
CNN
Global
Boxes
FPR-95 ↓
AUROC ↑
AUPR-S ↑
AUPR-E ↑
-
-
-
46.20
87.98
99.76
12.89
✓
-
-
47.72
87.83
99.77
11.64
✓
✓
-
46.39
88.71
99.79
13.10
✓
-
✓
37.30
90.05
99.81
20.23
-
✓
✓
40.18
89.09
99.78
18.22
✓
✓
✓
37.26
90.15
99.81
21.52
TABLE III : Ablation study on the importance of the features used.
Spatial Prompt
FPR-95 ↓
AUROC ↑
AUPR-S ↑
AUPR-E ↑
✗
38.56
89.29
99.79
18.40
✓
37.26
90.15
99.81
21.52
TABLE IV : Ablation study on the use of spatial information in the text prompts.
Prompt Format
FPR-95 ↓
AUROC ↑
AUPR-S ↑
AUPR-E ↑
1) A [cls]
37.04
89.83
99.80
19.70
2) This object is a [cls]
37.26
90.15
99.81
21.52
3) A photo of a [cls]
38.01
89.62
99.80
18.90
TABLE V : Analysis of different prompt formats on the final performance
Fig. 3 : Comparison of different OOD score distributions for different scoring methods. Including the object feature norm in the OOD score consistently leads to better results.