PIEDet: Prototype-Driven Intrinsically Explainable Object Detection

Jianlin Xiang; Linhui Dai; Xue Yang; Chaolei Yang; Yanshan Li

cs.CVApr 15, 2026

PIEDet: Prototype-Driven Intrinsically Explainable Object Detection

Authors: Jianlin Xiang, Linhui Dai, Xue Yang, Chaolei Yang, Yanshan Li

Organizations: Institute of Intelligent Information Processing, Shenzhen University · the Guangdong Provincial Key Laboratory of Intelligent Information Processing, Shenzhen University · the Shenzhen Key Laboratory of Modern Communications and Information Processing, Shenzhen University · Shanghai Jiao Tong University, Shanghai, China

Abstract

Existing object detectors typically make predictions in a black-box manner and struggle to simultaneously provide discriminative evidence for their predictions, which limits their deployment in safety-critical scenarios. To explain model predictions, existing post-hoc explanation methods mostly rely on gradient-based or perturbation-based operators. These methods not only introduce additional memory and computational overhead but also make it difficult to ensure that the generated explanations faithfully reflect the model's internal decision-making process. To address these limitations, we propose PIEDet, a prototype-driven intrinsically explainable object detection framework. PIEDet innovatively embeds class prototypes as explicit discriminative units into the classification branch of a one-stage detector, thereby improving detection performance while providing intrinsic interpretability. First, PIEDet constructs hierarchical class prototypes at different detection levels, enabling the model to learn scale-aware class-semantic representations. Second, we propose a prototype-driven feature learning method consisting of prototype regularization and a region-to-prototype matching loss. The former enhances the inter-class discriminability of the prototypes, while the latter encourages prototype responses to focus on object regions. Finally, we introduce a scale-aligned hierarchical prototype supervision mechanism that assigns scale-matched supervision signals to different detection levels, thereby enhancing the scale specificity of the hierarchical prototypes. On the ExDark, RTTS, and VOC2012-FOG datasets, PIEDet improves [email protected] over the baseline by 4.7%, 1.6%, and 4.8%, respectively, while demonstrating superior computational efficiency. Compared with mainstream post-hoc explanation methods, PIEDet achieves a better balance between explanation quality and explanation cost.

Figures & tables

Fig. 1: Comparison of explanation mechanisms for object detection. (a) Conventional detector with post-hoc explanation, where visual explanations are generated after detection by an external explainer. (b) Our proposed PIEDet, which integrates prototype matching into detection and directly produces both detection results and intrinsic prototype visual explanation.

Fig. 2: Overview of the PIEDet architecture. PIEDet performs object localization and category prediction using multi-scale features and decoupled detection heads, where the classification branch generates classification responses and intrinsic explanation maps through region-to-prototype matching. During training, the detection loss, prototype regularization loss, and scale-aligned region-to-prototype matching loss are jointly optimized.

Enhancer	Detector	Bicycle	Boat	Bottle	Bus	Car	Cat	Chair	Cup	Dog	Motorbike	People	Table	[email protected]
–	YOLOv8	71.8	64.2	69.2	82.8	76.8	51.7	54.4	58.9	62.5	61.9	69.6	48.4	64.4
MBLLEN [ 48 ]	YOLOv8	70.8	66.7	71.0	84.1	78.0	50.1	53.9	60.3	62.6	60.7	71.0	52.8	65.2
KinD [ 49 ]	YOLOv8	66.5	62.9	60.0	79.1	72.1	47.5	49.1	48.5	56.0	53.8	62.6	45.6	58.7
Zero-DCE [ 50 ]	YOLOv8	71.5	68.2	70.6	85.5	79.2	51.9	55.1	64.1	63.8	59.5	72.3	50.6	66.0
SCI [ 51 ]	YOLOv8	71.1	67.8	71.2	83.7	79.0	52.6	55.1	62.9	61.1	60.3	72.6	51.6	65.8
Retinexformer [ 52 ]	YOLOv8	78.5	71.3	72.7	83.5	76.3	57.8	53.2	58.2	69.1	63.1	70.6	47.9	66.8

TABLE I: Quantitative comparison of our proposed PIEDet with other representative methods on low-light scenes. All results are reported in percentage (%). The best results are highlighted in bold , and the second-best in underline . † Results of T2 are reported as in the original paper. ‡ YOLA is re-implemented with YOLOv8 trained from scratch.

Enhancer	Detector	[email protected]:0.95	[email protected]	[email protected]	mAP s	mAP m	mAP l
–	YOLOv8	39.7	64.4	43.4	10.9	31.0	45.3
Retinexformer [ 52 ]	YOLOv8	41.3	66.8	45.0	12.6	30.7	47.8
DiffUIR [ 53 ]	YOLOv8	40.4	65.3	44.1	10.0	30.5	46.5
Diff-Retinex++ [ 54 ]	YOLOv8	40.7	65.2	44.8	12.3	30.6	46.9
–	FeatEnHancer [ 42 ]	41.7	66.5	45.8	11.0	31.8	47.8
–	YOLA [ 44 ]	34.0	56.9	36.6	10.4	28.7	37.8

TABLE II: Complementary quantitative comparison on low-light scenes using COCO-style evaluation metrics. All results are reported in percentage (%). The best results are highlighted in bold , and the second-best in underline .

Enhancer	Detector	RTTS	VOC2012-FOG
–	YOLOv8	74.4	58.9
DNMGDT [ 55 ]	YOLOv8	74.6	57.6
DiffDehaze [ 56 ]	YOLOv8	63.0	57.9
SGDN [ 57 ]	YOLOv8	73.9	57.7
–	MASFNet [ 58 ]	67.1	59.2
–	PIEDet (ours)	76.0	63.7

TABLE III: Quantitative comparison ([email protected]) of our proposed PIEDet with other representative methods on hazy scenes.

Fig. 3: Visual comparison of detection results produced by different methods in low-light scenes.

Fig. 4: Visual comparison of detection results produced by different methods in hazy scenes.

Method	[email protected]:0.95	[email protected]	[email protected]
YOLOv8	0.444	0.623	0.476
FeatEnHancer [ 42 ]	0.438	0.615	0.475
YOLA [ 44 ]	0.342	0.518	0.373
MASFNet [ 58 ]	–	0.629	–
PIEDet (Ours)	0.475	0.663	0.516

TABLE IV: Comparison of different detection methods under normal illumination on the VOC2012 dataset.

Method	FPS $\uparrow$	FLOPs (G) $\downarrow$	Params (M) $\downarrow$
YOLOv8	132.67	22.317	11.164
FeatEnHancer [ 42 ]	58.28	22.323	11.303
YOLA [ 44 ]	80.74	28.087	11.172
MASFNet [ 58 ]	125.00	39.616	6.034
PIEDet (Ours)	137.23	21.266	10.815

TABLE V: Comparison of efficiency metrics among the YOLOv8, our method, and other end-to-end detectors.

Dataset	Method	Ins. Class $\uparrow$	Del. Class $\downarrow$	Ins. IoU $\uparrow$	Del. IoU $\downarrow$	Point Game $\uparrow$	Energy PG $\uparrow$	Comp. $\downarrow$	Time $\downarrow$	Memory $\downarrow$
ExDark	GradCAM	0.0755	0.5766	0.1194	0.3346	0.2699	0.1529	2.7177	1845.5072	9353
	GradCAM++	0.1074	0.5211	0.1436	0.3234	0.2718	0.1509	2.7855	1964.5813	9353
	ScoreCAM	0.1070	0.4694	0.1079	0.3031	0.4049	0.3972	0.2134	11062.1576	999
	DRISE	0.0702	0.3937	0.1489	0.2982	0.4620	0.1269	2.9304	38768.7614	999
	ODAM	0.1912	0.3566	0.1910	0.2608	0.4117	0.3555	0.9479	2043.8341	9353
	PIEDet (ours)	0.1621	0.3791	0.1628	0.2851	0.3994	0.3955	0.8856	1313.8374	999

TABLE VI: Quantitative comparison between PIEDet and mainstream post hoc interpretability methods on ExDark, RTTS, and VOC2012-FOG datasets. Time is measured in seconds (s), and Memory is measured in megabytes (MB).

Fig. 5: Visual comparison of explanation maps generated by PIEDet and mainstream post hoc interpretability methods in low-light scenes.

Fig. 6: Visual comparison of explanation maps generated by PIEDet and mainstream post hoc interpretability methods in hazy scenes.

Modules			Datasets
PR-Loss	RPM-Loss	SAHPS	ExDark	RTTS	VOC2012-FOG
			64.4	74.6	58.9
	✓		58.6	74.0	59.5
✓			66.5	73.9	62.8
✓	✓		64.9	74.2	62.2
	✓	✓	65.1	75.4	60.3

TABLE VII: Ablation results (mAP) of different module combinations on ExDark, RTTS, and VOC2012-FOG datasets.

Feature Level	Method	ExDark			RTTS			VOC2012-FOG
		mAP s	mAP m	mAP l	mAP s	mAP m	mAP l	mAP s	mAP m	mAP l
Layer1 ( $8\times 8$ )	YOLOv8	3.9	4.2	0.5	20.1	24.1	1.4	1.1	4.3	0.0
	PIEDet (ours)	4.2 (↑0.3)	2.5	0.2	21.0 (↑0.9)	12.6	0.2	1.8 (↑0.7)	2.2	0.0
Layer2 ( $16\times 16$ )	YOLOv8	0.0	14.9	7.6	5.5	39.2	41.3	0.0	8.6	7.9
	PIEDet (ours)	1.7	23.3 (↑8.4)	9.6	12.5	48.5 (↑9.3)	38.2	0.1	15.7 (↑7.1)	8.9
Layer3 ( $32\times 32$ )	YOLOv8	0.0	4.9	29.0	0.0	1.3	33.8	0.0	0.1	33.5

TABLE VIII: Ablation results using single feature layers on ExDark, RTTS, and VOC2012-FOG datasets to analyze scale sensitivity.

Method	FPS $\uparrow$	FLOPs (G) $\downarrow$	Params (M) $\downarrow$
YOLOv8	132.67	22.317	11.164
Ours ( $\tau_{1}{=}2$ , $\tau_{2}{=}4$ , $\tau_{3}{=}H/s_{3}$ )	138.92 (↑ 4.711 %)	21.262 (↓ 4.727 %)	10.815 (↓ 3.126 %)
Ours ( $\tau_{1}{=}4$ , $\tau_{2}{=}8$ , $\tau_{3}{=}H/s_{3}$ )	137.23 (↑ 3.437 %)	21.266 (↓ 4.709 %)	10.815 (↓ 3.126 %)
Ours ( $\tau_{1}{=}8$ , $\tau_{2}{=}16$ , $\tau_{3}{=}H/s_{3}$ )	135.73 (↑ 2.306 %)	21.274 (↓ 4.673 %)	10.817 (↓ 3.108 %)

TABLE IX: Ablation study on the hyperparameter settings (

\tau_{1},\tau_{2},\tau_{3}

) evaluating efficiency metrics on FPS, FLOPs, and Params.

H

denotes the input image height and

s_{3}

is the stride of the final feature map.

$\tau_{1}$	$\tau_{2}$	$\tau_{3}$	ExDark	RTTS	VOC2012-FOG
2	4	$H/s_{3}$	66.4	74.5	58.8
4	8	$H/s_{3}$	69.1	76.0	63.7
8	16	$H/s_{3}$	67.8	75.2	62.9

TABLE X: Effects of hyperparameter settings across ExDark, RTTS, and VOC2012-FOG datasets.

H

denotes the input image height and

s_{3}

is the stride of the final feature map.

Methods	ExDark	RTTS	VOC2012-FOG
Cosine	62.9	75.4	54.7
POP [ 46 ]	63.7	75.6	59.4
SVD (Ours)	69.1	76.0	63.7

TABLE XI: Comparison ([email protected]) of different prototype regularization methods on ExDark, RTTS, and VOC2012-FOG datasets.

Explainable Object Detection

See all 8 papers

EIVE: End-to-End Instance-Specific Visual Explanations for Detection Transformers
Jun 1, 2026Jianlin Xiang, Yanshan Li, Linhui DaiTransformer Interpretability Feature Attribution
Institute of Intelligent Information Processing, Shenzhen University; the Guangdong Provincial Key Laboratory of Intelligent Information Processing, Shenzhen University; and the Shenzhen Key Laboratory of Modern Communications and Information Processing, Shenzhen University, Shenzhen, China
PiPS: Post-Hoc Prototypical Explanations for Interpretable Semantic Segmentation
Sep 15, 2026Miłosz Adamczyk, Tymoteusz Zapala, Piotr Borycki +1Explainable Artificial Intelligence Efficient Semantic Segmentation
Jagiellonian University · Wrocław University of Science and Technology · IDEAS Research Institute
OPAL: Orthonormal Prototype Alignment Learning for Interpretable Image Classification
Aug 30, 2026Ilán Carretero, Gustavo Jesús Angulo, Rocío del Amor +1Prototype-Based Classification Fine-Grained Image Classification
CVBLab, HumanTech, Universitat Politècnica de València, Valencia, Spain · CMA, Mines Paris, PSL University, Sophia Antipolis, France · Artikode Intelligence S.L., Valencia, Spain

Jun 1, 2026J/K move · Enter open · S save

EIVE: End-to-End Instance-Specific Visual Explanations for Detection Transformers

Jianlin Xiang, Yanshan Li, Linhui Dai

Institute of Intelligent Information Processing, Shenzhen University; the Guangdong Provincial Key Laboratory of Intelligent Information Processing, Shenzhen University; and the Shenzhen Key Laboratory of Modern Communications and Information Processing, Shenzhen University, Shenzhen, China

Transformer Interpretability Feature Attribution Explainable Object Detection Detection Transformer Object Detection

Open paper AskPDF arXiv

Visual explainability for object detection remains challenging due to the multi-instance nature of detection. Existing approaches predominantly adopt post-hoc paradigms, such as gradient-based or perturbation-based explanation methods, to interpret pretrained detectors. However, these methods require additional gradient computation or repeated model inference, resulting in limited efficiency. To address this issue, we propose an End-to-end Instance-specific Visual Explanation framework (EIVE) that directly generates instance-level saliency maps following the forward pass of Detection Transformer (DETR)-like models. Specifically, we reformulate the cross-attention mechanism in the decoder as an instance-level feature attribution pathway, so that the cross-attention of each object query corresponds to the visual attribution of its predicted instance. Based on this formulation, we design a cross-layer hybrid consensus fusion (CLHCF) module to aggregate cross-attention signals across decoder layers, producing stable and compact explanations. The explanation process of EIVE requires neither gradient computation nor input perturbation, yielding high computational efficiency, and applies to single- and multi-scale DETR-like object detectors. Finally, we present an attention-aware joint training strategy (AAJTS) as a training-oriented application, which imposes spatial constraints on cross-attention patterns to encourage stable and concentrated attribution representations, thereby improving both interpretability and detection performance. Experiments on MS COCO 2017, ExDark, and Cityscapes demonstrate that EIVE produces high-quality instance-level saliency maps and achieves performance comparable to, or better than, state-of-the-art post-hoc methods across standard metrics, while substantially improving explanation efficiency. Code is available at https://github.com/xjlDestiny/EIVE.git.

Abstract

Explainable Object Detection

Explore similar work

EIVE: End-to-End Instance-Specific Visual Explanations for Detection Transformers

PiPS: Post-Hoc Prototypical Explanations for Interpretable Semantic Segmentation

OPAL: Orthonormal Prototype Alignment Learning for Interpretable Image Classification