ConEx: Human-Interpretable Saliency Maps via Concept-Aware Attribution

Yehonatan Elisha; Oren Barkan; Ziv Weiss Haddad; Noam Koenigstein

cs.CVOct 3, 2026

ConEx: Human-Interpretable Saliency Maps via Concept-Aware Attribution

Authors: Yehonatan Elisha, Oren Barkan, Ziv Weiss Haddad, Noam Koenigstein

Organizations: Tel Aviv University · The Open University

Abstract

Many visual explanation methods in computer vision highlight pixel importance but struggle to link these low-level cues to semantically meaningful concepts, limiting their interpretability and trustworthiness. We introduce Concept-based Explanations (ConEx), a novel framework that bridges saliency visualization with concept-based reasoning to provide both faithfulness and interpretability. ConEx automatically discovers class-specific concepts and represents them through concept activation vectors (CAVs), learned without manual supervision using an architecture-specific masking mechanism that reduces noise introduced by the segmentation masks to enhance concept purity. ConEx generates faithful saliency maps that reveal where each concept appears in the image and how it contributes to the prediction. To evaluate the reliability of these learned concepts, we propose two complementary metrics, Vector-Concept Match (VCM) and Concept-Class Match (CCM), that quantify concept alignment and enable direct comparison with existing methods. Extensive experiments across diverse settings demonstrate that ConEx achieves state-of-the-art performance on faithfulness, segmentation, and concept-quality benchmarks. Overall, ConEx advances the field toward truly interpretable and concept-grounded explanations in vision models.

Figures & tables

Figure 1: The ConEx Motivation: standard saliency maps (top) indicate where the model attends but not what it perceives, while CBMs (middle) reveal what the model detects but lack spatial grounding. ConEx (bottom) unifies both perspectives by decomposing predictions into spatially grounded, human-interpretable visual concepts (e.g., “yellow beak”) and providing concept-based explanation map.

Figure 2: The ConEx Framework: (top) Automatic discovery of class-specific concepts and CAV construction. (bottom) Generation of concept-based saliency maps.

Dataset	Model	Metric	GC	SC	SHAP	AC	RISE	FG	IG	IIA	ConEx
CUB	RN	INS	55.73	57.64	46.92	56.82	54.67	50.48	48.97	58.13	59.47
	RN	DEL	12.63	11.82	15.51	12.14	13.28	13.66	11.42	9.21	8.19
	DN	INS	56.18	58.09	47.31	57.22	55.04	50.93	49.58	58.81	59.62
	DN	DEL	12.24	11.53	15.35	11.87	12.92	13.45	11.19	8.96	8.03
	CN	INS	54.96	57.51	46.63	56.08	54.24	50.21	48.73	58.48	59.12
	CN	DEL	12.71	11.79	15.64	12.12	13.17	13.71	11.33	9.68	8.25

Table 1: Faithfulness Perturbation Evaluation: Comparison of Insertion (INS

\uparrow

) and Deletion (DEL

\downarrow

) AUC scores across datasets and models.

Metric	RISE	GC	AC	SC	IIA	ConEx
CMP $\uparrow$	0.68	0.70	0.76	0.72	0.74	0.78
CRC $\uparrow$	0.54	0.56	0.61	0.57	0.59	0.64
CNT $\uparrow$	0.63	0.68	0.83	0.85	0.88	0.92

Table 2: FunnyBirds Benchmark: Comparing Completeness (CMP), Correctness (CRC), and Contrastivity (CNT) scores using the RN model.

Metric	ViT-B				ViT-S				RN				DN				CN
Metric	ACE	ICE	MCD	ConEx	ACE	ICE	MCD	ConEx	ACE	ICE	MCD	ConEx	ACE	ICE	MCD	ConEx	ACE	ICE	MCD	ConEx
CINS $\uparrow$	30.97	30.18	33.23	35.33	30.08	29.31	32.27	34.30	44.89	43.74	48.16	51.20	44.63	43.58	47.92	51.37	44.75	43.62	48.03	51.44
CDEL $\downarrow$	13.15	13.70	11.93	10.35	13.31	13.87	12.34	10.51	10.82	11.28	9.56	8.74	10.93	11.41	9.48	8.66	10.87	11.36	9.59	8.72
VCM $\uparrow$	0.38	0.34	0.42	0.53	0.37	0.33	0.41	0.51	0.53	0.49	0.56	0.68	0.52	0.48	0.57	0.69	0.52	0.48	0.56	0.67
CCM $\uparrow$	12.31	10.18	11.49	18.43	11.76	10.05	11.63	17.92	23.04	21.38	20.22	28.01	22.48	20.91	20.17	27.63	22.27	21.01	21.35	27.41

Table 3: Concept Quality and Faithfulness Evaluation: results for Concept Faithfulness (CINS, CDEL) and our proposed CAV validation metrics (VCM, CCM) on IN dataset across all models.

T-test p-values
Method	SA $\uparrow$	PRC $\uparrow$	INNS $\uparrow$	INTS $\downarrow$
ACE	58.47	49.51	0.41	0.34
MCD	34.32	62.28	0.47	0.39
ConEx	72.13	70.48	0.52	0.28
ANOVA test p-values	$<0.001$	$<0.001$	$<0.001$	$<0.001$
ConEx vs ACE	$<0.001$	$<0.001$	$<0.001$	$0.0149$

Table 4: Human Evaluation: understandability tests on IN dataset using RN model over SA (%), PRC (%), INNS, and INTS.

Figure 3: Qualitative Results: Explanation maps produced using RN w.r.t. the classes (top to bottom): ’wing’, ’hornbill’, ’motor scooter, scooter’, ’convertible’.

Method	Con-ACE	Con-ICE	Con-Vis	Con-MCD	ConEx
INS $\uparrow$	57.45	57.92	56.98	58.16	59.21
DEL $\downarrow$	9.58	9.83	10.61	10.22	8.34

Table 5: Ablation on CAV creation mechanism: faithfulness tests comparing different methods using the IN dataset and RN model.

Method	INS $\uparrow$	DEL $\downarrow$
GC	54.92	12.68
SHAP	47.54	15.49
IIA	58.39	9.48
Con-GC	58.68	8.92
Con-SHAP	56.99	9.84
Con-IIA	58.96	8.65

Table 6: Ablation on explanation method fusion: faithfulness tests comparing different explanation methods using the IN dataset and RN model.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Class	Discovered Concepts
Green Heron	dark glossy cap , long sharp beak , green-grey back , yellow legs , white throat stripe , s-shaped neck, large wingspan
Red-winged Blackbird	red shoulder patches , yellow wing bar , black plumage , black beak , black pointed beak
Golden Retriever	floppy ears , thick tail , golden or cream coat , black nose , friendly expression , wavy fur , athletic build

Appendix

Table 7: Example concept sets discovered for selected classes. Bold concepts passed validation.

Dataset	Model	Metric	GC	SC	SHAP	AC	RISE	FG	IG	IIA	ConEx
SD	RN	INS	53.94	56.28	46.41	55.71	52.83	49.88	48.22	57.69	58.17
	RN	DEL	13.12	12.37	15.78	12.79	13.63	13.82	11.96	9.54	8.91
	DN	INS	54.38	56.83	46.87	56.12	53.47	50.27	48.74	57.96	58.53
	DN	DEL	12.98	12.29	15.52	12.58	13.39	13.74	11.83	9.17	8.76
	CN	INS	53.22	55.94	46.26	55.23	52.56	49.66	48.09	57.48	58.04
	CN	DEL	13.43	12.61	15.67	12.92	13.77	13.93	12.04	9.78	8.88

Appendix

Table 8: Faithfulness Perturbation Evaluation on the SD dataset: Comparison of Insertion (INS

\uparrow

) and Deletion (DEL

\downarrow

) AUC scores across datasets and models.

	ViT-B			RN
Method	$L=1$	$L=6$	$L=12$	$L-2$	$L-1$	$L$
VCM $\uparrow$	0.49	0.54	0.40	0.63	0.66	0.68
CCM $\uparrow$	17.89	18.52	16.31	27.56	27.94	28.19
CINS $\uparrow$	34.86	35.38	32.40	50.39	50.84	51.10
CDEL $\downarrow$	11.03	10.39	11.89	9.15	9.02	8.83

Appendix

Table 9: Ablation study evaluating the impact of different layers used for segment representation on the IN dataset.

Method	$N=10$	$N=50$	$N=100$	$N=200$
VCM $\uparrow$	0.55	0.68	0.69	0.71
CCM $\uparrow$	21.41	27.96	28.10	28.98

Appendix

Table 10: Ablation - Sensitivity to the number of samples (

N

) used in CAV construction (IN dataset, RN model).

Method	RN			DN			CN
Method	mIoU	mAP	PA	mIoU	mAP	PA	mIoU	mAP	PA
RISE	0.52	0.73	0.69	0.53	0.74	0.70	0.54	0.73	0.71
GC	0.58	0.77	0.73	0.59	0.78	0.74	0.60	0.77	0.74
SC	0.63	0.81	0.76	0.64	0.82	0.77	0.65	0.80	0.78
AC	0.69	0.84	0.77	0.68	0.83	0.80	0.69	0.87	0.80
IIA	0.75	0.86	0.79	0.69	0.81	0.84	0.72	0.86	0.81

Appendix

Table 11: Segmentation Evaluation on IN-S dataset: across all CNN models (RN, DN, CN), on the mIoU, mAP, and PA metrics. For all metrics higher is better.

Explore similar work

OPTIMUS-Prime: Minimal and Sufficient Concept Explanations for Deep Vision Models
Jun 5, 2026Arthur Hoarau, Chenrui Zhu, Vu Linh NguyenExplainable Artificial Intelligence Concept-Based Explanations
Université de Lorraine, CentraleSupélec Loria, CNRS, Metz, France · Université de technologie de Compiègne UMR CNRS 7253 Heudiasyc, France
Hide and Seek: HIDE Concepts to SEEK explanations for black box models
Oct 9, 2026Bhanu Tokas, Rahul Nair, Rockwell Jackson +1Interpretable ML Concept-Based Explanations
Arizona State University
Naming the Concepts Classifiers Rely On: Language-Anchored Decomposition for Faithful Explanation
Jul 8, 2026Ahsan Habib Akash, Dipkamal Bhusal, Stacey Jones +3Concept-Based Explanations Neural Network Interpretability
West Virginia University, USA · Rochester Institute of Technology, USA · O Analytics +3

Jun 5, 2026, cs.CVJ/K move · Enter open · S save

OPTIMUS-Prime: Minimal and Sufficient Concept Explanations for Deep Vision Models

Arthur Hoarau, Chenrui Zhu, Vu Linh Nguyen

Université de Lorraine, CentraleSupélec Loria, CNRS, Metz, France · Université de technologie de Compiègne UMR CNRS 7253 Heudiasyc, France

Explainable Artificial Intelligence Concept-Based Explanations Neural Network Interpretability

Open paper AskPDF arXiv

The growing demand for transparency in automated decision-making has propelled eXplainable Artificial Intelligence (XAI) to the forefront of machine learning research. In computer vision, however, existing explanation methods often prioritize end-user accessibility at the expense of formal guarantees, leaving a critical gap between practical utility and theoretical rigor. In this paper, we address this gap by introducing OPTIMUS, a novel framework for generating concept-based visual explanations for deep classification models. OPTIMUS explanations take the form of visual heatmaps that not only remain interpretable to end users, but are grounded in the well-established theory of prime implicants, providing formal guarantees that have been largely absent from existing saliency-based methods. Specifically, OPTIMUS explanations satisfy two desirable properties: sufficiency, ensuring that the highlighted concepts provably guarantee the classifier's prediction, and minimality, ensuring that no strict subset of those concepts retains this guarantee. Together, these properties yield explanations that are both logically tight and visually coherent. We validate our approach on a visual classification benchmark, demonstrating that OPTIMUS heatmaps naturally and faithfully surface the decision-relevant concepts underlying model predictions.

Abstract

Explore similar work

OPTIMUS-Prime: Minimal and Sufficient Concept Explanations for Deep Vision Models

Hide and Seek: HIDE Concepts to SEEK explanations for black box models

Naming the Concepts Classifiers Rely On: Language-Anchored Decomposition for Faithful Explanation