DriveOSS: Dataset and Benchmark for 3D Anomaly Segmentation and Open-set Semantic Segmentation in Autonomous Driving

Zhiran Yan; Savankumar Bhanderi; Sourabh Lolge; Rui Song; Girdharbhai Paneliya; Shiva Agrawal; Gordon Elger

cs.CVOct 23, 2026

DriveOSS: Dataset and Benchmark for 3D Anomaly Segmentation and Open-set Semantic Segmentation in Autonomous Driving

Authors: Zhiran Yan, Savankumar Bhanderi, Sourabh Lolge, Rui Song, Girdharbhai Paneliya, Shiva Agrawal, Gordon Elger

Organizations: Technische Hochschule Ingolstadt · UCLA

Abstract

Recent advances in 3D semantic segmentation have significantly improved scene understanding in autonomous driving. However, existing methods are typically developed under a closed-set assumption and evaluated under standard driving conditions, where all categories are known in advance. In real-world driving, autonomous vehicles inevitably encounter previously unseen objects, making closed-set perception insufficient for safe deployment. To address this challenge, we introduce DriveOSS, which, to the best of our knowledge, is the first real-world multimodal dataset and benchmark specifically designed for 3D anomaly segmentation and open-set semantic segmentation. It contains 12,661 synchronized and annotated frames captured using six surround-view RGB cameras, a 128-beam LiDAR, and a GPS/IMU unit. The dataset comprises 52 real-world driving scenes collected under clear, nighttime, cloudy, rainy, and snowy conditions and is organized into ID (in-distribution) and OOD (out-of-distribution) subsets. The ID scenes contain only known classes and are used for training, whereas the OOD scenes additionally contain unknown objects annotated as the outlier class and are used for validation and testing. Extensive experiments show that unseen objects remain a major challenge for current 3D perception systems. The dataset, benchmark protocols, and code are available at https://github.com/chrisyan/DriveOSS.

Figures & tables

Figure 1: Overview of the proposed DriveOSS dataset and benchmark. Top: Data collected under diverse weather and illumination conditions, including clear, nighttime, cloudy, rainy, and snowy scenes. The left shows the in-distribution (ID) subset containing normal driving scenarios, while the right shows the out-of-distribution (OOD) subset containing unknown objects that are not used during training. Bottom: Different perception settings illustrated using the same scene. From left to right: RGB image, closed-set semantic segmentation prediction, anomaly segmentation ground truth, and open-set semantic segmentation ground truth.

Dataset	Year	Weather	Modality	#Beam	Camera	#Scan	ID Evaluation Classes	OOD Scene Labels	Supported 3D Tasks
SemanticKITTI [ 2 ]	2019	Clear	L+C	64	Front-view	43,552	19 a	–	SS, PS
Waymo [ 34 ]	2019	Clear/Night/Rainy	L+C	64	Surround-view	46,000	23	–	SS
Toronto-3D [ 35 ]	2020	Clear	L+C	32	Surround-view	–	8	–	SS
A2D2 [ 12 ]	2020	Cloudy/Rainy/Sunny	L+C	16	Surround-view	41,277	38	–	2D SS, detection
SemanticPOSS [ 27 ]	2020	Clear	L+C	40	Surround-view	2,988	14	–	SS, PS
PandaSet [ 41 ]	2021	Clear/Night	L+C	64/150	Surround-view	16,480	37	–	SS, detection

Table 1: Comparison of representative automotive 3D segmentation datasets. SS, IS, PS, AS, OSS, and SSC denote semantic, instance, panoptic, anomaly, and open-set semantic segmentation, and semantic scene completion, respectively. L and C denote LiDAR and camera, while #Beam indicates the number of LiDAR beams. The ID Evaluation Classes column reports the number of ID semantic classes used for benchmark evaluation. a SemanticKITTI provides 34 raw labels, which are merged or ignored to obtain 19 ID evaluation classes. b DriveOSS provides 32 raw annotation labels, including outlier and unlabeled . After merging semantically equivalent labels and excluding ignored labels, 19 ID semantic classes are retained for training and evaluation. The outlier label is additionally used for anomaly segmentation and open-set semantic segmentation.

Category	Representative Objects	Characteristics
Containers & Packaging	Cardboard boxes, plastic boxes, foam blocks, garbage bags, and small trash bins	Common roadside obstacles with simple geometries and diverse materials
Furniture & Fixtures	Office chairs, folding chairs, wooden chairs, clothes racks, and advertising signs	Bulky objects with complex structures and varying heights
Tools & Industrial Equipment	Ladders, shovels, metal barriers, and maintenance equipment	Rigid structures with distinctive geometries and occasionally high reflectivity
Personal Belongings	Backpacks, bicycle helmets, bags, and other personal items	Predominantly small objects with irregular shapes and mixed materials
Utility & Service Equipment	Shopping carts, waste-collection carts, and mobile bins	Movable service-related objects commonly encountered in urban environments
Roadside Debris & Miscellaneous	Wooden boards, fallen signs, and other uncommon objects	Unstructured obstacles with diverse appearances and physical properties

Table 2: Taxonomy of OOD objects in DriveOSS.

Panel of Figure 2 — Figure 2: DriveOSS data collection platform. Top: the sensor vehicle. Bottom: six surround-view cameras, one 360 ∘ LiDAR, and one GPS/IMU unit.

Figure 3: LiDAR–camera calibration. LiDAR points projected onto the six camera views using the estimated intrinsic and extrinsic parameters.

Figure 4: Representative anomalies in DriveOSS. Physically present objects with diverse scales, geometries, and materials, captured under different environmental conditions.

	ID Train		ID Validation		OOD Validation			OOD Test
Condition	Seq.	Frames	Seq.	Frames	Seq.	Frames	OOD Points	Seq.	Frames	OOD Points
Clear	4	809	6	1,701	9	2,315	252,606	8	1,766	885,729
Nighttime	1	184	1	79	2	287	22,132	2	357	42,979
Cloudy	2	636	1	186	2	742	43,548	2	666	71,440
Rainy	1	399	1	165	2	696	35,104	2	618	66,509
Snowy	1	212	1	253	2	238	12,898	2	352	89,803

Table 3: Distribution across environmental conditions. Numbers of sequences and LiDAR frames for each split, together with annotated OOD points for DriveOSS-OOD.

Figure 5: Label distributions in DriveOSS-ID and DriveOSS-OOD. The outlier label occurs exclusively in DriveOSS-OOD, while labels marked with 1 are excluded from evaluation.

Figure 6: Spatial distribution of OOD points. Normalized OOD-point frequency relative to the ego vehicle over all DriveOSS-OOD validation and test sequences.

Method	M	ID-val	OOD-test	$\Delta$
P3Former	L	43.27	29.58	$-13.69$
LCPS-lidar	L	51.94	33.75	$-18.19$
M4F3D	L	34.98	31.11	$-3.87$
LIDO	L	58.35	40.99	$-17.36$
LCPS	L+C	54.05	34.00	$-20.05$
M4F3D*	L+C	48.44	28.46	$-19.98$

Table 4: Closed-set 3D semantic segmentation. mIoU (%) on DriveOSS-ID validation and the ID-labeled points of DriveOSS-OOD test.

\Delta

denotes the change from ID to OOD.

Method	M	AUROC $\uparrow$	FPR@95 $\downarrow$	AP $\uparrow$
P3Former + MSP	L	26.35	99.50	0.27
P3Former + ML	L	34.60	99.80	0.35
P3Former + Energy	L	37.94	99.79	0.65
P3Former + Entropy	L	26.63	99.57	0.37
P3Former + RbA	L	89.80	45.94	3.53
LCPS-lidar + MSP	L	78.76	47.32	1.11

Table 5: 3D anomaly segmentation on DriveOSS-OOD test. We report AUROC

\uparrow

, FPR@95

\downarrow

, and AP

\uparrow

. ML denotes MaxLogit, and M denotes the input modality. M4F3D* is the modified LiDAR–camera fusion model.

Method	M	car	bicycle	motorcycle	truck	other-veh.	person	bicyclist	motorcyclist	road	parking	sidewalk	other-ground	building	fence	vegetation	trunk	terrain	pole	traffic-sign	outlier	mOpen-IoU
P3Former + MSP	L	24.69	0.0	0.0	6.56	0.0	0.14	1.00	0.0	0.03	0.0	0.0	0.0	0.05	0.13	1.26	0.05	0.0	0.03	0.06	0.28	1.79
P3Former + ML	L	22.96	0.0	0.0	6.33	0.0	0.95	0.28	0.0	0.02	0.0	0.0	0.0	1.37	1.08	0.48	0.28	0.01	0.07	0.27	0.27	1.80
P3Former + Energy	L	25.45	0.0	0.0	6.55	0.0	1.14	0.30	0.0	0.02	0.0	0.0	0.0	1.70	1.16	0.49	0.32	0.0	0.07	0.30	0.27	1.98
P3Former + Entropy	L	24.65	0.0	0.0	6.63	0.0	0.15	0.85	0.0	0.02	0.0	0.0	0.0	0.05	0.16	1.07	0.05	0.0	0.03	0.05	0.27	1.78
P3Former + RbA	L	2.13	0.0	0.0	0.0	0.0	0.0	0.0	0.0	16.62	1.03	0.51	0.72	65.18	53.47	0.08	13.10	4.49	0.38	1.14	0.51	8.36
LCPS-lidar + MSP	L	41.78	9.74	1.46	11.21	0.0	29.56	9.91	0.0	31.21	0.13	8.55	0.0	74.63	35.17	54.87	5.60	0.33	13.55	64.90	0.61	20.66

Table 6: 3D open-set semantic segmentation on DriveOSS-OOD test. We report class-wise open-IoU for the 19 ID classes, outlier IoU, and mOpen-IoU over ID classes. M denotes the input modality. M4F3D* is the modified LiDAR–camera fusion model.

		Segmentation	Post-hoc Scoring Overhead					Total Time
Method	M	Network	+MSP	+ML	+Energy	+Entropy	+RbA	(ms)
P3Former	L	51.29	0.07	0.10	0.13	0.10	0.11	[51.36, 51.42]
LCPS-lidar	L	85.44	0.12	0.17	0.25	0.19	0.18	[85.56, 85.69]
M4F3D	L	104.21	0.11	0.17	0.23	0.18	0.13	[104.32, 104.44]
LIDO	L	–	–	–	–	–	–	68.01
LCPS	L+C	179.80	0.11	0.16	0.21	0.15	0.16	[179.91, 180.01]

Table 7: Inference time on the DriveOSS-OOD test set. Per-frame runtime in ms, measured on a single NVIDIA RTX 4090 GPU and decomposed into the base segmentation network and post-hoc anomaly scoring overhead. Since LIDO jointly performs semantic and anomaly segmentation, only its end-to-end runtime is reported.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Sensor Type	Sensor Model	Number	Details
LiDAR	Ouster OS1-128	$\times$ 1	128 beams, $360^{\circ}$ horizontal FOV, $-22.5^{\circ}$ to $+22.5^{\circ}$ vertical FOV, 20 Hz
Camera	Basler ace2R a2A1920-51gcBAS	$\times$ 6	RGB, $1920\times 1080$ resolution, 10 Hz
GPS/IMU	Certus	$\times$ 1	$>$ 600 Hz update rate, dual-antenna GNSS/INS, 0.01 m horizontal accuracy with RTK, 0.015 m vertical accuracy with RTK, and 0.1 ∘ roll/pitch/heading accuracy

Appendix

Table S1: Sensor specifications of the DriveOSS platform.

Method	M	car	bicycle	motorcycle	truck	other-veh.	person	bicyclist	motorcyclist	road	parking	sidewalk	other-ground	building	fence	vegetation	trunk	terrain	pole	traffic-sign	mIoU
P3Former	L	83.03	1.45	0.20	45.10	44.08	24.12	30.29	4.65	64.77	30.19	19.69	2.34	91.31	57.76	87.92	65.99	46.78	57.73	64.64	43.27
LCPS-lidar	L	85.52	23.06	8.15	48.59	41.16	34.51	54.58	21.42	79.49	34.22	59.63	2.62	94.51	47.55	91.55	70.93	54.05	58.69	76.60	51.94
M4F3D	L	73.96	6.51	22.04	8.69	26.69	15.52	40.65	1.86	63.21	22.79	17.01	2.29	66.02	42.00	87.04	43.74	40.38	34.53	49.69	34.98
LIDO	L	90.93	19.87	29.64	73.91	63.43	51.10	80.35	25.32	81.29	37.98	57.06	8.63	93.52	55.53	88.91	70.43	56.49	50.39	73.92	58.35
LCPS	L+C	91.80	15.22	27.37	50.12	64.59	31.53	55.11	12.66	78.32	32.91	56.54	2.58	96.29	55.64	91.73	71.48	57.63	64.19	71.18	54.05
M4F3D*	L+C	84.25	9.41	27.01	31.98	20.73	46.02	68.33	42.07	72.51	16.85	48.63	1.69	86.78	44.35	88.61	67.21	43.50	49.85	70.55	48.44

Appendix

Table S2 : Closed-set 3D semantic segmentation results on the DriveOSS-ID validation set. Class-wise IoU and mIoU are reported. “M” denotes the input modality, where “L” and “C” represent LiDAR and camera, respectively. M4F3D* denotes the modified LiDAR–camera fusion model.

Method	M	car	bicycle	motorcycle	truck	other-veh.	person	bicyclist	motorcyclist	road	parking	sidewalk	other-ground	building	fence	vegetation	trunk	terrain	pole	traffic-sign	mIoU
P3Former	L	81.50	0.20	0.13	18.31	37.36	33.70	2.81	2.76	52.11	4.80	6.48	0.34	91.88	59.06	85.07	75.38	28.51	36.49	46.12	34.90
M4F3D	L	85.28	30.02	3.47	59.85	54.47	75.57	31.46	23.47	49.90	12.59	14.42	6.98	91.26	40.31	85.04	75.99	33.98	32.90	62.08	45.74
LCPS-lidar	L	86.02	5.02	0.0	64.99	13.03	53.16	21.61	5.74	61.36	9.18	10.41	4.42	91.47	48.72	86.15	76.54	34.45	42.20	57.10	40.61
LIDO	L	88.95	23.22	3.41	73.95	20.92	77.69	40.93	19.48	55.17	29.29	15.71	12.01	92.14	36.93	84.56	65.24	40.66	27.23	58.64	45.59
LCPS	L+C	86.64	1.48	0.48	36.23	21.75	50.84	12.73	4.12	58.26	14.20	8.68	6.39	92.74	51.89	86.42	78.06	33.84	35.93	55.99	38.78
M4F3D*	L+C	73.57	3.28	10.76	10.42	24.50	59.25	0.00	18.53	45.59	0.07	7.99	2.11	77.74	25.48	85.07	67.30	29.49	23.71	34.76	31.56

Appendix

Table S3 : Closed-set 3D semantic segmentation results on the DriveOSS-OOD validation set. Class-wise IoU and mIoU over the 19 known classes are reported. “M” denotes the input modality, where “L” and “C” indicate LiDAR and camera, respectively. M4F3D* represents the modified fusion model.

Method	M	car	bicycle	motorcycle	truck	other-veh.	person	bicyclist	motorcyclist	road	parking	sidewalk	other-ground	building	fence	vegetation	trunk	terrain	pole	traffic-sign	mIoU
P3Former	L	61.64	0.34	0.15	7.42	3.22	22.47	14.27	0.0	53.58	1.88	13.27	3.03	94.60	53.74	83.44	46.87	29.07	18.84	54.13	29.58
LCPS-lidar	L	61.46	21.19	2.83	13.49	0.0	36.86	17.84	0.0	54.59	6.14	19.48	2.71	95.63	52.44	85.76	55.92	22.49	20.31	72.12	33.75
M4F3D	L	59.62	11.83	8.66	10.02	0.60	54.11	43.22	0.00	44.65	16.62	26.85	3.19	82.82	33.25	80.63	38.60	13.87	9.41	53.19	31.11
LIDO	L	68.18	28.94	0.55	15.03	36.02	62.09	58.62	3.47	56.52	25.19	22.48	14.61	96.77	45.39	85.76	42.54	34.06	12.29	70.31	40.99
LCPS	L+C	60.91	9.81	7.50	14.03	0.43	33.93	24.87	0.0	52.71	14.87	22.10	6.32	96.74	58.67	85.36	53.60	28.23	16.54	59.30	34.00
M4F3D*	L+C	60.58	5.40	7.80	7.66	2.52	51.12	35.81	0.50	43.87	0.03	18.70	0.70	83.11	25.50	80.53	38.48	14.26	12.64	51.56	28.46

Appendix

Table S4 : Closed-set 3D semantic segmentation results on the DriveOSS-OOD test set. Class-wise IoU and mIoU are reported. "M" means modality, and "L" and "C" denote LiDAR and Camera, respectively. M4F3D* represents the modified fusion model.

Method	M	AUROC $\uparrow$	FPR@95 $\downarrow$	AP $\uparrow$
P3Former + MSP	L	35.34	99.02	0.13
P3Former + ML	L	41.72	98.55	0.14
P3Former + Energy	L	46.53	98.42	0.26
P3Former + Entropy	L	32.64	99.03	0.14
P3Former + RbA	L	86.43	61.27	1.57
LCPS-lidar + MSP	L	72.51	55.61	0.31

Appendix

Table S5: 3D anomaly segmentation results on the DriveOSS-OOD validation set. We report AUROC

\uparrow

, FPR@95

\downarrow

, and AP

\uparrow

. M4F3D* represents the modified fusion model.

Method	M	car	bicycle	motorcycle	truck	other-veh.	person	bicyclist	motorcyclist	road	parking	sidewalk	other-ground	building	fence	vegetation	trunk	terrain	pole	traffic-sign	outlier	mOpen-IoU
P3former + MSP	L	20.46	0.0	0.08	17.26	0.73	0.03	2.07	0.09	0.05	0.0	0.0	0.0	0.01	0.03	0.15	0.0	0.0	0.0	0.82	0.10	2.20
P3former + ML	L	30.49	0.0	0.16	16.68	12.48	1.40	2.77	0.09	0.0	0.0	0.0	0.0	0.44	3.58	0.10	0.0	0.01	0.01	0.53	0.10	3.62
P3former + Energy	L	33.03	0.0	0.15	17.08	14.80	1.64	2.51	0.10	0.0	0.0	0.0	0.0	0.60	4.35	0.10	0.0	0.01	0.01	0.52	0.10	3.94
P3former + Entropy	L	21.07	0.0	0.10	17.04	0.89	0.05	3.52	0.10	0.02	0.0	0.0	0.0	0.01	0.04	0.13	0.0	0.01	0.0	0.76	0.10	2.30
P3former + RbA	L	3.35	0.0	0.0	0.0	0.0	0.0	0.0	0.0	34.14	3.51	0.01	0.01	69.96	31.04	0.03	39.03	3.92	3.16	0.84	0.14	9.95
LCPS-lidar + MSP	L	69.58	1.87	0.0	44.93	12.12	42.17	25.34	5.18	29.48	0.16	12.79	0.01	72.53	12.50	63.89	21.76	1.94	35.77	46.73	0.17	26.25

Appendix

Table S6 : 3D open-set segmentation results on the DriveOSS-OOD validation set. Class-wise open-IoU is reported for the 19 known classes, along with outlier IoU and mOpen-IoU over known classes. "M" denotes the input modality: L for LiDAR-only and L+C for LiDAR-camera fusion. M4F3D* represents the modified fusion model.

Figure S1 : Qualitative segmentation results. From left to right: RGB image, closed-set semantic segmentation prediction, anomaly segmentation prediction, open-set semantic segmentation prediction, and open-set semantic segmentation ground truth.

Figure S2 : Point cloud-to-image projection under different weather conditions. From left to right, the columns show clear, night, cloudy, rainy, and snowy conditions. From top to bottom, the rows correspond to DriveOSS-ID, DriveOSS-OOD validation, and DriveOSS-OOD test scenes. The projected points show weather-dependent LiDAR distributions, with notably fewer road-surface reflections in rainy scenes.

Figure S3 : Snapshot of the point cloud labeling interface, showing temporally registered 3D point clouds and RGB images.

Figure S4 : Visual comparison between automatic pre-labeling and final human-refined ground truth across DriveOSS subsets. Rows show representative samples from ID , OOD Validation , and OOD Test . Columns show: (1) RGB image, (2) raw Mask4Former pre-labels, and (3) final manual ground truth.

Autonomous Driving Datasets

Autonomous driving datasets collect and organize driving-scene data to train and evaluate perception, prediction, planning, and safety systems.

See all 141 papers

Learning to Identify Out-of-Distribution Objects for 3D LiDAR Anomaly Segmentation
Apr 26, 2026Simone Mosco, Daniel Fusaro, Alberto PrettoPoint Cloud Anomaly Detection 3D Point Cloud Segmentation
University of Padova, Italy
DDStereo: Efficient Dual Decoder Transformers for Stereo 3D Road Anomaly Detection
Jun 23, 2026Shiyi Mu, Zichong Gu, Zhiqi Ai +23D ViTs Autonomous Driving Perception
Shanghai University, Shanghai, China · Xi’an Jiaotong-Liverpool University, Suzhou, China
Road-Aware Anomaly Segmentation with Query-Guided Polygons and CLIP in Autonomous Driving
Jul 5, 2026Zhiran Yan, Gordon ElgerImage Segmentation Autonomous Driving Perception
Institute of Innovative Mobility (IIMo), Technical University Ingolstadt of Applied Sciences, Ingolstadt, 85049, Germany. · Fraunhofer Institute for Transportation and Infrastructure Systems IVI, Ingolstadt, 85051, Germany.

Apr 26, 2026J/K move · Enter open · S save

Learning to Identify Out-of-Distribution Objects for 3D LiDAR Anomaly Segmentation

Simone Mosco, Daniel Fusaro, Alberto Pretto

University of Padova, Italy

Point Cloud Anomaly Detection 3D Point Cloud Segmentation Autonomous Driving Perception Anomaly Localization OOD Detection

Open paper AskPDF arXiv

Understanding the surrounding environment is fundamental in autonomous driving and robotic perception. Distinguishing between known classes and previously unseen objects is crucial in real-world environments, as done in Anomaly Segmentation. However, research in the 3D field remains limited, with most existing approaches applying post-processing techniques from 2D vision. To cover this lack, we propose a new efficient approach that directly operates in the feature space, modeling the feature distribution of inlier classes to constrain anomalous samples. Moreover, the only publicly available 3D LiDAR anomaly segmentation dataset contains simple scenarios, with few anomaly instances, and exhibits a severe domain gap due to its sensor resolution. To bridge this gap, we introduce a set of mixed real-synthetic datasets for 3D LiDAR anomaly segmentation, built upon established semantic segmentation benchmarks, with multiple out-of-distribution objects and diverse, complex environments. Extensive experiments demonstrate that our approach achieves state-of-the-art and competitive results on the existing real-world dataset and the newly introduced mixed datasets, respectively, validating the effectiveness of our method and the utility of the proposed datasets. Code and datasets are available at https://simom0.github.io/lido-page/.

Abstract

Autonomous Driving Datasets

Explore similar work

Learning to Identify Out-of-Distribution Objects for 3D LiDAR Anomaly Segmentation

DDStereo: Efficient Dual Decoder Transformers for Stereo 3D Road Anomaly Detection

Road-Aware Anomaly Segmentation with Query-Guided Polygons and CLIP in Autonomous Driving