Out-of-Distribution Object Detection in Street Scenes via Synthetic Outlier Exposure and Transfer Learning

Sadia Ilyas; Annika Mütze; Klaus Friedrichs; Thomas Kurbiel; Matthias Rottmann

cs.CVMar 17, 2026

Out-of-Distribution Object Detection in Street Scenes via Synthetic Outlier Exposure and Transfer Learning

Authors: Sadia Ilyas, Annika Mütze, Klaus Friedrichs, Thomas Kurbiel, Matthias Rottmann

Organizations: University of Wuppertal · Aptiv Services Deutschland GmbH · University of Osnabrück, Germany

Abstract

Out-of-distribution (OOD) objects are often silently overlooked by object detectors, which produce no detection for them, leaving downstream modules with nothing to act on and no indication that anything was missed. A reliable object detector must detect OOD objects by localizing and correctly classifying them as OOD. Existing approaches primarily focus on OOD detection for image classification or rely on auxiliary branches, and typically do not treat in-distribution (ID) and OOD objects in a unified manner. We present SynOE-OD, a Synthetic Outlier-Exposure-based Object Detection method, which uses text-guided inpainting and an open-vocabulary object detector to generate and verify synthetic outliers offline. These are then used to fine-tune an object detector of choice with an added OOD class, which requires no auxiliary branch at inference time. On the OoDIS benchmark, SynOE-OD outperforms the strongest open-vocabulary baseline, reaching AP_50:95 of 57.6%, 29.4% and 39.2% on RoadObstacle, RoadAnomaly and Fishyscapes, against 48.5%, 22.7% and 22.7%. This state-of-the-art performance holds for both fine-tuned closed-set and open-vocabulary detectors while maintaining competitive ID performance in street scenes.

Figures & tables

Figure 1: Overview of the SynOE-OD method, where an object detector is fine-tuned to jointly detect ID and OOD objects. NuImages is the training set augmented with synthetic outliers. At inference time, the detector is evaluated on an unseen real-world test set, e.g., RoadAnomaly.

Figure 2: Overview of the synthetic outlier data generation process. The input image consists of ID objects, $X_{\textit{ID}}$ , some of which are replaced with synthetic outliers using Stable Diffusion in the content generation step. The synthetic outliers are assigned to the OOD class, $y_{\textit{OOD}}$ , with a refined bounding box, $b_{\textit{OOD}}$ using GDINO.

Figure 3: Left: original scene. Right: the same scene where the ID object car was regenerated instead of an outlier.

	RoadObstacle			RoadAnomaly			Fishyscapes
Method	AP 50:95	AP 50	AP 75	AP 50:95	AP 50	AP 75	AP 50:95	AP 50	AP 75
ZS-OVOD	48.5	77.6	52.2	22.7	33.3	23.3	22.7	33.4	24.5
UGainS	8.1	11.6	8.9	11.7	17.5	12.1	15.6	24.3	18.3
SynOE-OD (DD:ResNet50)	48.8	79.4	49.4	26.9	35.8	29.0	35.3	55.3	37.9
SynOE-OD (DD:Swin-L)	57.6	87.5	61.7	25.7	32.1	27.2	35.9	49.5	40.7
SynOE-OD (GDINO:Swin-T)	55.6	85.1	60.2	29.4	37.3	31.5	39.2	55.8	42.2

Table 1: Results on the OoDIS benchmark. Size specific AP is mentioned in the Appendix appendix B .

Method	Benchmark	AP 50:95	TP	FP	FN	Recall	ID
ID-only	RoadObstacle	11.3	142	144	415	25.5	42.2
	RoadAnomaly	2.4	445	6772	286	60.9
	LostAndFound	5.4	553	3932	1375	28.7
VOS	RoadObstacle	1.4	16	17	541	2.9	42.0
	RoadAnomaly	2.2	291	2359	440	39.8
	LostAndFound	2.1	334	921	1594	17.3

Table 2: Comparison to feature-space outlier synthesis and ID-only baseline. All three methods see the same ID data during training. Recall is computed over all ground-truth OOD instances

Method	RO	RA	LF	ID	ID ${}_{\text{base}}$
GD zero-shot	48.3	19.5	26.4	—	—
GD prompt ensemble	48.5	22.7	—	—	—
SynOE-OD (DD:R50)	48.7	26.0	32.0	38.8	40.4
SynOE-OD (DD:Swin-L)	56.5	24.5	27.0	49.1	50.8
SynOE-OD (GDINO:Swin-T)	54.3	29.3	29.2	45.8	47.2

Table 3: OOD and ID object detection results with three detectors compared against GDINO zero-shot and prompt ensemble [ 21 ] . ID

{}_{\text{base}}

is the same detector without the OOD class and trained on original NuImages.

Table 4: Effect of the OOD class name on SynOE-OD (GDINO).

Method	Training data	RO	RA	LF	NU
SynOE-OD (DD:R50)	+ CS	48.8 $(+0.0)$	26.1 $(+0.1)$	31.2 $(-1.3)$	37.4 $(-1.4)$
SynOE-OD (DD:R50)	+ CS + BDD	49.1 $(+0.3)$	26.0 $(+0.0)$	31.9 $(-0.6)$	37.7 $(-1.1)$
SynOE-OD (GD:Swin-T)	+ CS	55.6 $(+1.3)$	28.2 $(-1.1)$	33.5 $(+4.3)$	45.0 $(-0.8)$
SynOE-OD (GD:Swin-T)	+ CS + BDD	55.0 $(+0.7)$	27.5 $(-1.8)$	34.7 $(+5.5)$	44.8 $(-1.0)$
SynOE-OD (DD:Swin-L)	+ CS	57.6 $(+1.1)$	25.2 $(+0.7)$	30.3 $(+3.3)$	48.5 $(-0.6)$
SynOE-OD (DD:Swin-L)	+ CS + BDD	56.4 $(-0.1)$	24.4 $(-0.1)$	31.3 $(+4.3)$	48.3 $(-0.8)$

Table 5: Effect of domain diversity in the training set. The parentheses show the difference to when trained on NuImages only.

Class	ID baseline	SynOE-OD
car	58.3	57.1 $(-1.2)$
pedestrian	42.3	41.4 $(-0.9)$
truck	42.8	40.6 $(-2.2)$
motorcycle	47.3	44.9 $(-2.4)$
bus	42.4	41.7 $(-0.7)$
bicycle	35.8	32.3 $(-3.5)$

Table 6: Per class ID performance in AP 50..95 . The annotation count is in the Appendix table 19 .

Model	Annotation	RO	RA	LF
SynOE-OD (DD:R50)	baseline — full vocabulary	48.7	26.0	32.0
	20 overlapping classes removed	47.2	25.8	30.5
	20 non-overlapping classes removed	48.3	25.9	31.5
SynOE-OD (GD:Swin-T)	baseline — full vocabulary	54.3	29.3	29.2
	20 overlapping classes removed	53.8	28.4	25.6
	20 non-overlapping classes removed	53.8	27.7	25.1

Table 7: Full vocabulary is the standard dataset with 15,604 outlier annotations. The 20 overlapping classes removal reduces the annotations to 12,863 and the non-overlapping ones to 12,832.

Outliers	RO	RA	LF	NU
$0$ % (ID-only)	12.0	2.4	5.2	40.4
$25$ %	46.4	26.1	29.9	40.0
$50$ %	48.2	26.2	30.4	39.9
$70$ %	48.1	25.5	31.0	39.2
$100$ %	48.7	26.0	32.2	38.7

Table 8: Effect of synthetic outliers proportion. The outlier percentage shows the inpainted images retained. This corresponds to

3.3

6.5

9.2

% and

13.2

% of the annotations being OOD and every other image is the original scene with its ID annotations.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Illustration of the inpainted synthetic outliers on NuImages. The first row shows the ID objects being replaced with OOD objects of different sizes while maintaining a minimum distance. The second row shows objects inpainted directly into the free-space i.e., the road region.

Baseline	Comparison
ZS-OVOD	OoDIS
UGainS	OoDIS
VOS	training-based OOD
ID-only (ours)	effect of inpainted data

Appendix

Table 9: Baselines and what each accounts for.

Configuration	RO	RA	LF
Standard (no warmup, unfrozen)	0.578	0.264	0.306
+ LinearLR warmup	0.563	0.260	0.266
frozen_stages=2	0.575	0.241	0.275
frozen_stages=1	0.574	0.232	0.274

Appendix

Table 10: Backbone freezing and learning-rate warm-up for DINO-DETR with Swin-L on inpainted NuImages. The standard configuration is the mean over three runs; each variant changes one setting.

Config	Epoch	ID mAP	RO	RA	LF
A (frozen=1, flat LR)	1	32	47	26	30
A (frozen=1, flat LR)	3	38	43	22	21
B (frozen=4, flat LR)	1	31	47	26	32
B (frozen=4, flat LR)	3	36	44	23	23
C (frozen=4, cosine+warmup)	1	30	47	26	32
C (frozen=4, cosine+warmup)	3	36	46	24	25

Appendix

Table 11: Ablation grid - ID and OOD metrics averaged over seeds (24, 42). ‘Frozen’ refers to the backbone stages. The dropout is added to the transformer encoder, decoder. The model trained is DINO-DETR with ResNet50 on NuImages.

lr	Test/Val set	ep1	ep2	ep3	ep4
1e-4	RO	0.560	0.538	0.532	0.537
1e-4	RA	0.298	0.284	0.271	0.258
1e-4	LF	0.300	0.276	0.264	0.270
1e-4	ID mAP	0.432	0.460	0.472	0.477
2e-4	RO	0.550	0.532	0.526	0.524
2e-4	RA	0.298	0.269	0.259	0.246

Appendix

Table 12: Doubling the learning rate raises ID mAP and lowers OOD AP.

	with EMA			without EMA
Epoch	RO	RA	LF	RO	RA	LF
1	$49.3$ $\pm 0.7$	$26.8$ $\pm 0.8$	$36.1$ $\pm 0.9$	$46.4$ $\pm 1.9$	$26.3$ $\pm 1.3$	$31.7$ $\pm 2.8$
2	$48.7$ $\pm 0.1$	$26.0$ $\pm 0.1$	$31.7$ $\pm 1.2$	$47.2$ $\pm 2.6$	$24.6$ $\pm 0.9$	$27.5$ $\pm 2.1$
3	$47.6$ $\pm 0.6$	$25.2$ $\pm 0.5$	$28.9$ $\pm 1.4$	$46.8$ $\pm 2.1$	$24.4$ $\pm 1.1$	$27.8$ $\pm 1.9$
4	$47.7$ $\pm 1.0$	$24.5$ $\pm 0.4$	$27.5$ $\pm 0.8$	$46.3$ $\pm 1.1$	$23.7$ $\pm 1.1$	$26.0$ $\pm 1.5$

Appendix

Table 13: Weight averaging against raw weights, DINO-DETR ResNet-50 on inpainted NuImages. Mean

\pm

standard deviation over three seeds (

242

444

2244

) and AP 50..95 in percent.

Epoch	s444 no EMA	s444 EMA	s2244 no EMA	s2244 EMA
1	34.0	34.4	34.3	33.7
2	38.0	38.6	37.5	38.8
3	39.4	40.6	40.4	40.5
4	41.1	41.4	41.1	41.2

Appendix

Table 14: ID validation mAP

Model	Training data	$n$	RO	RA	LF
DD:R50	NuImages	3	$48.7$ $\pm 0.11$	$26.0$ $\pm 0.08$	$31.7$ $\pm 1.15$
GD:Swin-T	+ Cityscapes	2	$55.6$ $\pm 0.57$	$28.2$ $\pm 1.34$	$33.5$ $\pm 0.14$

Appendix

Table 15: Run-to-run variation at the reported epoch. Mean

\pm

standard deviation over independently seeded runs, AP 50..95 in percent.

alligator	fox	Pots pans
antelope	gerbil	Printer
armadillo	giraffe	pylon
badger	Glasses	pylon large
bear	goat	pylong small
bison	gorilla	rabbit
bloated plastic bag	guinea pig	raccoon

Appendix

Table 16: The full outlier vocabulary

	RoadObstacle			RoadAnomaly			Fishyscapes
Method	S	M	L	S	M	L	S	M	L
SynOE-OD (DD:ResNet50)	38.7	69.6	76.3	3.7	13.6	60.9	18.3	59.3	52.7
SynOE-OD (DD:Swin-L)	47.4	76.1	90.2	2.5	10.9	61.2	19.0	58.2	59.4
SynOE-OD (GDINO:Swin-T)	46.1	74.7	87.0	7.4	15.5	64.2	19.7	66.5	61.2

Appendix

Table 17: OoDIS results, averaged over two seeds per model. S, M and L are the COCO area ranges.

Model	RO	RA	LF
R50	0.483 $(-0.4)$	0.200 $(-6.0)$	0.300 $(-2.0)$
GD	0.514 $(-2.9)$	0.201 $(-9.2)$	0.268 $(-2.4)$

Appendix

Table 18: Effect of 2,741 unlabeled outliers. The parentheses show the difference from the baseline.

Class	original	train	ID baseline	SynOE-OD
car	46,745	38,100	58.3	57.1 $(-1.2)$
pedestrian	32,210	32,210	42.3	41.4 $(-0.9)$
truck	8,029	5,386	42.8	40.6 $(-2.2)$
motorcycle	3,406	2,638	47.3	44.9 $(-2.4)$
bus	1,987	1,987	42.4	41.7 $(-0.7)$
bicycle	2,285	1,447	35.8	32.3 $(-3.5)$

Appendix

Table 19: Per class ID performance in AP 50..95 . The train annotations are used for fine-tuning SynOE-OD and the original annotations for the ID baseline. The values in parentheses show the difference from the baseline.

Table 20: Effect of bounding-box refinement (left) and label reassignment (right). Left: Unrefined refers to the original boxes and refined is the accurate boxes returned by GDINO. Right: replaced is the number of ID objects of each class that was inpainted over, reassigned is the number for which GDINO identified the regenerated object as the original class; the last two columns show the class AP without and with the label reassignment.

Object Detection

Object detection locates and classifies objects in images or video, balancing localization accuracy, robustness, and inference speed.

See all 624 papers

SAM3-O2D2: Zero-Shot Object Out-of-Distribution Detection by Object Class Prompting of the SAM3-Image Model
Sep 8, 2026Lucas Görnhardt, Timo Bartels, Tim FingscheidtObject Detection OOD Detection
Technische Universität Braunschweig, Institute for Communications Technology
Geometry over Density: Few-Shot Cross-Domain OOD Detection
May 5, 2026Shawn Li, You Qin, Jiate Li +4Diffusion-Based OOD Detection Distribution Shift Detection
University of Southern California · National University of Singapore · Amazon
Geometrically Constrained Outlier Synthesis
Mar 9, 2026Daniil Karzanov, Marcin DetynieckiOOD Detection Synthetic Anomaly Generation
AXA AI Research · EPFL, Lausanne, Switzerland · TRAIL, Sorbonne Université, Paris, France +1

Sep 8, 2026J/K move · Enter open · S save

SAM3-O2D2: Zero-Shot Object Out-of-Distribution Detection by Object Class Prompting of the SAM3-Image Model

Lucas Görnhardt, Timo Bartels, Tim Fingscheidt

Technische Universität Braunschweig, Institute for Communications Technology

Object Detection OOD Detection

Open paper AskPDF arXiv

Object detectors have shown remarkable performance in various fields, among these medical imaging, surveillance, and autonomous driving. However, they are prone to overconfidence when encountering unseen objects in real-world deployments, causing potential safety issues. To address this, detecting out-of-distribution (OOD) objects is essential for reliable object detection. Modern approaches leverage the broad semantic knowledge of foundation models such as CLIP for post-hoc few- and zero-shot OOD detection. However, these methods typically perform OOD assessment in feature space, which can be sensitive to object detector localization errors and variations in object appearance. Moreover, the current state-of-the-art (SOTA) zero-shot method performs computationally costly diffusion in inference. In this work, for our proposed zero-shot object OOD detection method SAM3-O2D2, we employ the SAM3-image foundation model in an efficient manner. Specifically, we prompt SAM3 only with the object detector's predicted classes and compare the predictions of the object detector and SAM3. An object is in-distribution (ID), if SAM3 also detects an object at the corresponding location. If SAM3 does not detect the prompted object, this indicates a mismatch between the detector's prediction and the image content, suggesting that the object is OOD. Experimental results show that our method significantly surpasses the so-far zero-shot SOTA method. Specifically, we achieve new SOTA AuROC and FPR95 metrics over both ID datasets Pascal-VOC and BDD100K and both OOD datasets MS-COCO and OpenImages.

Abstract

Object Detection

Explore similar work

SAM3-O2D2: Zero-Shot Object Out-of-Distribution Detection by Object Class Prompting of the SAM3-Image Model

Geometry over Density: Few-Shot Cross-Domain OOD Detection

Geometrically Constrained Outlier Synthesis