SPK: Eliciting Structured Prior Knowledge for Interpretable Out-of-Distribution Detection in Real-Time Object Detection

Changshun Wu; Weicheng He; Xiaowei Huang; Saddek Bensalem

cs.CVAug 19, 2026

SPK: Eliciting Structured Prior Knowledge for Interpretable Out-of-Distribution Detection in Real-Time Object Detection

Authors: Changshun Wu, Weicheng He, Xiaowei Huang, Saddek Bensalem

Organizations: University of Liverpool, UK · Université Grenoble Aples, France · CSX-AI, France

Abstract

Object detectors often produce over-confident predictions for objects outside their training categories, leading to so-called out-of-distribution (OoD) hallucinations. Existing approaches for detecting or mitigating such hallucinations typically either construct scoring functions directly over learned object detector representations or modify the object detector itself to suppress hallucination emergence. However, the latent priors implicitly encoded in these representations remain largely unexplored and have not been explicitly decoded for OoD detection. To uncover and exploit these latent priors, we propose Structured Prior Knowledge (SPK), a hallucination-oriented framework that explicitly elicits OoD-relevant priors from pretrained object detectors. Specifically, SPK leverages in-distribution data and hallucination-inducing samples as diagnostic supervision to elicit part-level semantic concepts underlying object detector decision-making, rather than using them merely for rejection or object detector adaptation. The elicited semantic priors are further integrated with geometric and contextual priors to form a compact five-dimensional SPK representation for OoD detection. Extensive experiments across diverse object detector architectures and multiple OoD benchmarks demonstrate that SPK achieves state-of-the-art OoD detection. Our findings reveal that pretrained object detectors already encode substantially richer latent knowledge than is typically exploited for OoD detection. More importantly, this knowledge can be explicitly elicited and organized into a compact, structured, and interpretable knowledge space for prediction reliability analysis. This suggests a promising proactive route for improving object detector reliability by explicitly uncovering and leveraging latent priors. Code and data are available at: https://gricad-gitlab.univ-grenoble-alpes.fr/dnn-safety/spk

Figures & tables

Figure 1: The proposed SPK framework, a proactive OoD hallucination mitigation framework, further reduces OoD-induced hallucinations beyond previous state-of-the-art methods ( Wu et al. 2026 ) , achieving additional improvements in challenging high-performance regimes.

Figure 2: Overview of the proposed SPK framework. SPK elicits semantic, geometric, and contextual priors from a pretrained object detector and organizes them into a compact five-dimensional representation for OoD hallucination detection.

Method	YOLO				Faster R-CNN				RT-DETR
	PASCAL-VOC		BDD-100K		PASCAL-VOC		BDD-100K		PASCAL-VOC		BDD-100K
	Near-OoD	Far-OoD	Near-OoD	Far-OoD	Near-OoD	Far-OoD	Near-OoD	Far-OoD	Near-OoD	Far-OoD	Near-OoD	Far-OoD
MSP	67.48	67.18	72.93	74.12	68.36	78.69	77.46	74.41	70.44	67.81	77.46	74.41
EBO	90.49	90.84	87.22	87.06	60.62	56.21	94.83	94.37	98.03	96.69	94.83	94.37
MLS	89.88	90.08	86.47	87.06	59.62	57.89	92.55	91.08	92.84	89.75	92.55	91.08
SCALE	80.67	80.92	77.44	70.59	92.34	80.70	86.35	84.98	81.12	76.92	86.35	84.98

Table 1: Comparison of OoD detection performance using FPR95 across different detector architectures trained on PASCAL-VOC and BDD-100K. Lower is better.

Model	Method	VOC (N/F)	BDD (N/F)
YOLO	Original	946 / 440	701 / 666
	Proximal-OoD	134 / 60	80 / 47
	SPK	135 / 52	69 / 5
Faster R-CNN	Original	2150 / 1335	2576 / 1634
	Proximal-OoD	710 / 253	207 / 167
	SPK	299 / 140	60 / 25

Table 2: OoD detection counts (Near-OoD/Far-OoD) across different detector architectures. Lower is better.

$\mathcal{L}_{\mathrm{dice}}$	$\mathcal{L}_{\mathrm{suppress}}$	$\mathcal{L}_{\mathrm{group}}$	VOC	BDD	Average
			Near / Far	Near / Far	FPR95 $\downarrow$
✗	✓	✓	25.80 / 21.30	17.85 / 18.26	20.80
✓	✓	✗	22.10 / 16.40	15.29 / 15.97	17.44
✓	✗	✓	20.50 / 15.80	14.18 / 12.93	15.85
✓	✓	✓	14.25 / 11.84	9.86 / 0.70	9.16

Table 3: Ablation study of the SPK loss components on YOLO trained on PASCAL-VOC and BDD-100K. Results are reported as Near-OoD / Far-OoD FPR95. The average is computed over all four results. Lower is better.

Prior components	VOC	BDD	Average
	Near / Far	Near / Far	FPR95 $\downarrow$
Semantic	15.43 / 13.28	30.37 / 25.58	21.17
Semantic + Geometric	13.23 / 11.50	20.88 / 3.84	12.36
Semantic + Geometric + Contextual	14.25 / 11.84	9.86 / 0.70	9.16

Table 4: Ablation study of different prior components on YOLO trained on PASCAL-VOC and BDD-100K. Each dataset column reports Near-OoD / Far-OoD FPR95. The average is computed over all four results. Lower is better.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Automated part-level annotation pipeline. Given an RoI crop and its object category, OWLv2 grounds the corresponding GPT-5-generated concept vocabulary into part bounding boxes. Each box prompts SAM 2 to produce a refined pixel-level part mask. Masks satisfying the object-mask coverage threshold are projected into detector-RoI coordinates and rasterized as binary $7\times 7$ targets. Concepts without a retained mask receive an all-zero target.

Figure 4: Part-Level Concept Annotation Example. We use a bird RoI crop to illustrate the step-by-step annotation process for the concepts wing , head , beak , and torso . Step 1: OWLv2 grounds each concept within the detector RoI and generates a bounding-box proposal (a). Step 2: SAM 2 refines each proposal into a pixel-level part mask, which is retained only if at least 70% of its pixels overlap with the corresponding SAM 2 object mask (b). Step 3: Each retained mask is projected into detector-RoI coordinates and converted into a binary $7\times 7$ supervision target (c). Step 4: The binary target is overlaid on the RoI crop for visualization (d).

Method	Overall			Per-class concept-mIoU
Method	concept-mIoU	[email protected]	# Concepts	bird	bus	car	cat	cow	dog	horse
Ours	40.8	38.4	25	46.5	37.9	43.9	40.4	37.4	41.8	38.2
VLPart ( Sun et al. 2023 )	37.0	36.8	25	35.6	32.8	32.5	38.1	36.8	44.1	36.5
Grounded SAM ( Ren et al. 2024 )	32.3	30.7	25	32.1	31.7	31.3	33.0	29.5	38.3	29.4

Appendix

Table 5: Quality of the automated part-level annotations. We report the overall concept-mIoU, [email protected], number of covered concepts, and per-class concept-mIoU. All values except concept coverage are percentages. Best results are shown in bold.

Detector	Feature source	Channels	RoI feature $\mathbf{F}_{i}$
YOLO	Detect neck	$128+256+512$	$896\times 7\times 7$
RT-DETR	Hybrid-encoder neck	$256+256+256$	$768\times 7\times 7$
Faster R-CNN	ResNet-FPN	$256$	$256\times 7\times 7$

Appendix

Table 6: RoI feature sources and dimensions for each detector. YOLO and RT-DETR concatenate RoI-aligned features from three feature scales, whereas Faster R-CNN uses the Detectron2 box_pooler .

Semantic Elicitation Head	Hyperparameter	Value
Architecture	Hidden channels	256
	Dropout	0.1
	Normalization	GroupNorm
	Activation	GELU
	Residual blocks	2
	Output head	$1\times 1$ conv $\rightarrow$ num_concepts

Appendix

Table 7: Hyperparameters of the Semantic Elicitation Head.

Detector	Feature source	Channels	Embedding dimension
YOLO	Backbone L6 (stride $16$ )	$256$	$\mathrm{mean}+\mathrm{std}:512$
RT-DETR	HGBlock L9 backbone (stride $32$ )	$2048$	$\mathrm{mean}+\mathrm{std}:4096$
Faster R-CNN	ResNet-FPN P2–P5	$4\times 256$	$\mathrm{mean}+\mathrm{std}:2048$

Appendix

Table 8: Image-level embedding sources and dimensions for each detector. We globally pool each selected feature map by its spatial mean and standard deviation, then concatenate the resulting statistics. YOLO and RT-DETR use their deepest selected backbone stage, whereas Faster R-CNN aggregates all four ResNet-FPN levels.

Panel of Figure 5 — Figure 5: Qualitative visualization of part-level semantic responses. The learned concept maps are well aligned with the corresponding regions, demonstrating that the semantic elicitation head successfully decodes spatially grounded semantic evidence from detector RoI features.

Data split	Accuracy (%)
ID training set	96.5
Proximal OoD	81.0
Background	77.0

Appendix

Table 9: Semantic-head group-classification accuracy. Computed by applying

\arg\max

to concept activations for YOLO on PASCAL-VOC.

Figure 6: Distributions of the learned semantic group responses. Samples from different data sources predominantly activate their corresponding semantic groups, validating the effectiveness of the proposed group objective.

Panel of Figure 7 — Figure 7: UMAP visualization of representation spaces for detections from the dog , sheep , and cat categories. The visualizations are obtained using YOLO trained on PASCAL-VOC. We compare detector classification logits (left) with the proposed SPK representations (right), using the same ID-validation, Near-OoD, and Far-OoD samples. SPK yields a more structured representation space with clearer distributional differences between ID and OoD samples.

Method	YOLO				Faster R-CNN				RT-DETR
	PASCAL-VOC		BDD-100K		PASCAL-VOC		BDD-100K		PASCAL-VOC		BDD-100K
	Near-OoD	Far-OoD	Near-OoD	Far-OoD	Near-OoD	Far-OoD	Near-OoD	Far-OoD	Near-OoD	Far-OoD	Near-OoD	Far-OoD
MSP	81.24	79.47	77.63	75.12	78.71	73.84	72.48	76.39	79.16	78.72	74.31	75.06
EBO	60.73	57.92	65.41	62.86	82.58	86.14	52.07	50.83	40.12	43.31	49.68	52.74
MLS	59.41	61.26	63.78	65.02	84.76	83.58	54.82	59.46	56.37	59.74	54.91	57.43
SCALE	69.84	71.73	72.46	79.31	57.49	69.72	64.18	67.82	69.37	75.16	66.42	65.73

Appendix

Table 10: AUROC comparison across detector architectures on PASCAL-VOC and BDD-100K. Higher is better.

Figure 8: Qualitative SPK OoD verdicts for OoD-induced hallucination. Results are shown for YOLO on PASCAL-VOC. Near-OoD examples (left) exhibit strong proximal-concept responses, whereas Far-OoD examples (right) are dominated by background-concept responses. Each example shows the predicted bounding box, detector confidence, and five SPK prior components. Green bands represent class-specific normal ranges estimated from ID training data, red bands indicate values outside these ranges, and circular markers denote the observed component values. All examples are correctly rejected as OoD.

$\mathcal{L}_{\mathrm{dice}}$	$\mathcal{L}_{\mathrm{suppress}}$	$\mathcal{L}_{\mathrm{group}}$	Faster R-CNN				RT-DETR
			PASCAL-VOC		BDD-100K		PASCAL-VOC		BDD-100K
			Near-OoD	Far-OoD	Near-OoD	Far-OoD	Near-OoD	Far-OoD	Near-OoD	Far-OoD
✗	✓	✓	27.87	19.98	18.36	18.74	32.31	27.08	25.35	29.14
✓	✓	✗	24.42	15.42	14.15	16.11	27.86	22.53	20.71	24.36
✓	✗	✓	22.19	14.73	11.34	13.27	24.86	21.64	18.15	20.81
✓	✓	✓	13.92	10.52	2.31	1.52	15.48	17.32	11.42	9.25

Appendix

Table 11: Ablation of SPK loss components. Evaluated across Faster R-CNN and RT-DETR on PASCAL-VOC and BDD-100K. Lower FPR95 is better.

Prior components	Faster R-CNN				RT-DETR
	PASCAL-VOC		BDD-100K		PASCAL-VOC		BDD-100K
	Near-OoD	Far-OoD	Near-OoD	Far-OoD	Near-OoD	Far-OoD	Near-OoD	Far-OoD
Semantic	18.17	18.91	23.92	25.21	18.34	29.17	33.36	34.75
Semantic + Geometric	15.78	16.43	14.52	15.71	15.85	25.35	26.51	26.82
Semantic + Geometric + Contextual	13.92	10.52	2.31	1.52	15.48	17.32	11.42	9.25

Appendix

Table 12: Ablation of SPK prior components. Evaluated across Faster R-CNN and RT-DETR on PASCAL-VOC and BDD-100K. Lower FPR95 is better.

Method	ID: PASCAL-VOC				ID: BDD-100K
	OoD: MS-COCO		OoD: OpenImages		OoD: MS-COCO		OoD: OpenImages
	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$
MDS ( Lee et al. 2018b )	97.39	50.28	97.88	49.08	70.86	76.83	71.43	77.98
Gram matrices ( Sastry and Oore 2020 )	94.16	43.97	95.29	38.81	73.81	60.13	71.56	57.14
KNN ( Sun et al. 2022 )	91.80	62.15	91.36	59.64	64.75	80.90	61.13	79.64
CSI ( Tack et al. 2020 )	84.00	55.07	79.16	51.37	70.27	77.93	71.30	76.42

Appendix

Table 13: Comparison with competitive OoD detection methods for Deformable-DETR. Results are reported on PASCAL-VOC and BDD-100K as ID datasets, with MS-COCO and OpenImages as OoD datasets. Higher AUROC and lower FPR95 indicate better OoD detection performance. Methods marked with

\dagger

employ an external DINO ViT encoder to extract additional visual representations for OoD detection, rather than relying solely on Deformable-DETR features. SPK (DINO ViT) is included to provide a fair comparison with UNO-Adapter, as both methods operate under this setting. The best results are highlighted in bold.

Method	MS-COCO		OpenImages
Method	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$
CSI ( Tack et al. 2020 )	82.95	57.41	81.83	59.91
GAN-Synthesis ( Lee et al. 2018a )	82.67	59.97	83.67	60.93
VOS ( Du et al. 2022b )	85.23	51.33	88.70	47.53
SIREN ( Du et al. 2022a )	85.36	64.68	82.78	68.53
TIB ( Wu and Deng 2023 )	90.36	41.55	88.09	47.19

Appendix

Table 14: Comparison with competitive OoD detection methods for Faster R-CNN. Results are reported on PASCAL-VOC as the ID dataset, with MS COCO and OpenImages as OoD datasets. Higher AUROC and lower FPR95 indicate better OoD detection performance. Methods marked with

\dagger

employ an external DINO ViT encoder to extract additional visual representations for OoD detection, rather than relying solely on Faster R-CNN features. SPK (DINO ViT) is included to provide a fair comparison with UNO-Adapter, as both methods operate under this setting. The best results are highlighted in bold.

Component	Cost (ms)
Original inference	10.15
SPK inference *	12.65
Semantic prior elicitation	0.17
Isolation Forest	0.05

Appendix

Table 15: Per-image runtime of the complete SPK inference pipeline. Reported for a YOLO model pretrained on PASCAL-VOC. The complete SPK pipeline introduces an additional

2.72

ms latency per image, corresponding to a

26.8\%

runtime overhead over the original detector inference. * SPK inference obtains detection outputs, image-level contextual embeddings, and RoI features within the same forward pass.

OOD Detection

Out-of-distribution (OOD) detection identifies inputs unlike a model’s training data so systems can flag uncertainty, reject inputs, or adapt safely.

See all 222 papers

Sparse Autoencoders for Interpretable Out-of-Distribution Detection
Jul 13, 2026Ayush Karmacharya, Luke Luschwitz, Lucia Romero +2OOD Detection
Purdue University · EPFL
Knowledge-Guided Failure Prediction: Detecting When Object Detectors Miss Safety-Critical Objects
Mar 26, 2026Jakob Paul Zimmermann, Gerrit Holzbach, David LerchObject Detection Anomaly Detection
Fraunhofer HHI, Berlin, Germany · Fraunhofer IOSB, Karlsruhe, Germany
PLSP (Pre-hoc Liminal Space Profiling): OOD Prediction over Detection -- An Anticipatory Approach for Machine Learning Model Reliability
Sep 10, 2026Vipul Bansal, Himanshu Buckchash, Balasubramanian Raman +1Distribution Shift Robustness OOD Detection
University of Wisconsin Madison, Madison, WI, USA · IMC University of Applied Sciences, Krems, Austria · Indian Institute of Technology Roorkee, Roorkee, India

Jul 13, 2026J/K move · Enter open · S save

Sparse Autoencoders for Interpretable Out-of-Distribution Detection

Ayush Karmacharya, Luke Luschwitz, Lucia Romero, Yanan Niu+1

Purdue University · EPFL

OOD Detection

Open paper AskPDF arXiv

Reliable detection of out-of-distribution (OOD) samples is crucial for the safe deployment of machine learning models. Neural networks often produce overconfident predictions for inputs that deviate from their training data, leading to significant degradation in performance. While many OOD detection methods focus on the final output layer, they neglect the rich hierarchical information present in intermediate network layers. This paper introduces a novel approach that leverages sparse autoencoders (SAEs) to learn interpretable features from these intermediate activations. We find that in-distribution (ID) and OOD data activate distinct sets of these sparse features. We propose a new OOD score derived from the cosine similarity between the sparse feature activations of a test sample and the mean activations of ID classes. Our post-hoc detection method not only achieves state-of-the-art performance on standard OOD detection benchmarks, but yields interpretable insights into how distribution shift affects learned representations.

Abstract

OOD Detection

Explore similar work

Sparse Autoencoders for Interpretable Out-of-Distribution Detection

Knowledge-Guided Failure Prediction: Detecting When Object Detectors Miss Safety-Critical Objects

PLSP (Pre-hoc Liminal Space Profiling): OOD Prediction over Detection -- An Anticipatory Approach for Machine Learning Model Reliability