Hide and Seek: HIDE Concepts to SEEK explanations for black box models

Bhanu Tokas; Rahul Nair; Rockwell Jackson; Hannah Kerner

cs.CVOct 9, 2026

Hide and Seek: HIDE Concepts to SEEK explanations for black box models

Authors: Bhanu Tokas, Rahul Nair, Rockwell Jackson, Hannah Kerner

Abstract

Interpreting machine learning model predictions is a central challenge, particularly for black-box vision models. Concept-based attribution methods interpret models using human-understandable concepts. Existing concept-based methods can only interpret classification and regression models. They require access to model weights or large concept-annotated datasets, and most methods cannot provide image-level attribution. We propose Hide and Seek (HnS), a new method for concept-level attribution that does not require access to model weights or labeled concept datasets. HnS can interpret a wide range of vision models. Given a concept and image set, HnS retrieves concept-aligned images using a vision-language model, localizes the concept within each image, masks the localized region, and measures attribution from the change in the black-box model's output between the original and masked images. This enables faithful attribution by directly measuring the model's behavior and supporting local attributions. We validated HnS on CelebA and COCO-Stuff. Compared to existing methods, HnS showed better agreement with ground-truth attributions. We demonstrated the applicability of HnS on diverse tasks, including image captioning and satellite image segmentation, where no existing concept attribution method can be used. We provided our anonymous code in the supplementary material.

Figures & tables

Figure 1: Overview of Hide and Seek ( HnS ). For a concept $c$ (smiling), we encode its text prompt (“photo of a person smiling”) into a text vector $v_{c}$ using SigLIP (a vision-language model). We pre-process $v_{c}$ (using modality correction and orthogonalization) to obtain $v_{c}^{*}$ . From an image database, we select a set of images $R_{c}$ that show the highest cosine similarity with $v_{c}^{*}$ . For each image $x_{i}$ in $R_{c}$ , HnS identifies the region $c$ in $x_{i}$ and hides this region to create a masked image $\tilde{x}_{i}$ . We pass $x_{i}$ and $\tilde{x}_{i}$ to our model $M$ to obtain predicted probabilities for a class $y$ (male): $P(x_{i})$ and $P(\tilde{x}_{i})$ , respectively. We measure attribution for the concept $c$ (on class $y$ ) as $P(x_{i})$ - $P(\tilde{x}_{i})$ .

Method	Black Box	Concept Label-Free	Surrogate Free	Local Attribution	Wide Task Support
TCAV ( Kim et al., 2018 )	✗	✗	✓	✓	✗
ConceptSHAP ( Yeh et al., 2020 )	✗	✗	✓	✓	✗
PACE ( Wang et al., 2024 )	✗	✗	✓	✓	✗
PCBM ( Yuksekgonul et al., 2023 )	✓	✓	✗	✗	✗
Hide and Seek (ours)	✓	✓	✓	✓	✓

Table 1: Comparison of Concept Attribution Techniques

	CelebA			COCO-Stuff
Method	ResNet18	ViT	ConvNeXt	ResNet18	ViT	ConvNeXt
TCAV	0.319	0.213	0.192	0.030	0.085	0.117
PCBM (ResNet-18)	0.180	0.164	0.258	0.058	0.018	0.059
PCBM (SigLIP)	0.208	0.197	0.291	0.163	0.211	0.214
PCBM (CLIP-RN50)	0.149	0.142	0.201	0.168	0.199	0.214
HnS (ours)	0.615	0.507	0.421	0.315	0.342	0.257

Table 2: For three models, we reported the correlation (

\rho

) between ground-truth global attribution scores and the global attribution scores from HnS , TCAV, and PCBM. A higher

\rho

value for a method indicates that it shows better agreement with ground-truth attributions.

Panel of Figure 2 — Figure 2: For each class-concept pair, we ranked images by magnitude of local attribution scores of Hide and Seek ( HnS ) and TCAV. For each method, we visualized the top $3$ and bottom $3$ images. (a)-(b) are for CelebA, (c)-(d) are for COCO-Stuff. We show more examples in Appendix I .

Male-centric concepts	Number of models	Female-centric concepts	Number of models
tie	9/10	cake	7/10
toaster	8/10	hair dryer	6/10
backpack	7/10	teddy bear	6/10
sandwich	7/10	carrot	6/10
bottle	6/10	umbrella	6/10

Table 3: Across all models, we identified concepts that frequently occurred in the list of top

20

concepts with the highest positive attribution (male-centric concepts), and in the list of top

20

concepts with the highest negative attribution (female-centric concepts). For instance, tie was a male-centric concept in

9

out of

10

models.

Figure 3: HnS attributions for PRUE, a model used to segment crop fields. For each concept $c$ (e.g., inland waters), we reported three attribution scores: $(1)$ mask window A, with a blue bar ( $c$ is only hidden in window A), $(2)$ mask window B, with a orange bar ( $c$ is only hidden in window B), and $(3)$ mask both, with an green bar ( $c$ is hidden in both windows).

Appendix figures & tables39 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Reusable (encoder-side)	Black-Box-Dependent	Total
Hide and Seek (ours)	232.0s	6.0s	238.0s
TCAV	-	349.5s	349.5s
PCBM (SigLIP)	844.7s	391.7s	1236.4s
PCBM (CLIP)	516.6s	348.1s	864.7s
PCBM (ResNet-18)	615.2s	347.3s	962.5s

Appendix

Table 4: Wall-clock cost with an Nvidia A-100 for different techniques, split into encoder-side cost that is reusable across black-box models and cost that depends on the specific black-box model being explained.

Method	Reusable (encoder-side)	Black-Box-Dependent	Total
Hide and Seek (ours)	338.1s	10.6s	348.7s
TCAV	-	405.8s	405.8s
PCBM (SigLIP)	1187.0s	461.5s	1648.5s
PCBM (CLIP)	678.5s	429.6s	1108.1s
PCBM (ResNet-18)	664.1s	481.6s	1145.7s

Appendix

Table 5: Wall-clock cost with an Nvidia RTX-4090 for different techniques, split into encoder-side cost that is reusable across black-box models and cost that depends on the specific black-box model being explained.

Table 6: Joint ablation of encoder, orthogonalization, and Modality Shift for CelebA dataset with ResNet18 black box classifier.

Table 7: Joint ablation of encoder, orthogonalization, and Modality Shift for COCO-Stuff dataset with ResNet18 black box classifier.

Dataset	SigLIP	CLIP	OpenCLIP-H	Perception
CelebA	0.633	0.409	0.354	0.573
COCO-Stuff	0.318	0.331	0.197	0.245

Appendix

Table 8: Each encoder’s own best setting (over the orthogonalize x modality-correction grid)

Dataset	K=15	K=30	K=50	K=75	K=100
CelebA	0.603	0.607	0.615	0.604	0.594
COCO-Stuff	0.269	0.297	0.315	0.311	0.305

Appendix

Table 9: K (retrieval pool size) ablation.

Figure 4: Spearman $\rho$ reported on CelebA and COCO-stuff dataset with ResNet-18 black box model and SigLIP encoder. We observe that for both datasets, the curve remains relatively flat until $\alpha=1.75$ , after which it declines sharply. As such, we suggest keeping the default $\alpha=1.5$ for real-world usage.

Prompt	CelebA	COCO-Stuff
a photo of concept	0.621	0.308
a photo showing concept	0.634	0.311
an image of concept	0.605	0.306
Ensemble	0.615	0.315

Appendix

Table 10: Prompt ablation: 3 phrasing templates and ensemble, for HnS on CelebA classifier and COCO-Stuff dataset. Class-pooled Spearman

\rho

is reported against ground truth annotations.

Figure 5: Modality Shift: image embeddings (blue) and text embeddings (green) occupy two separate regions, each offset from the origin by its own center, $\mu_{img}$ and $\mu_{text}$ respectively, rather than being centered at zero. For a given concept $C$ , the full image embedding $V_{C,img}$ and text embedding $V_{C,text}$ (solid arrows from the origin) each decompose into their ellipsoid’s center plus a concept-specific offset $V_{C}$ (dotted) that is shared between the two modalities. Because $\mu_{img}\neq\mu_{text}$ , directly comparing $V_{C,img}$ and $V_{C,text}$ conflates this modality offset with the concept-specific signal we actually want, motivating the modality shift correction.

Panel of Figure 6 — Figure 6: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 7 — Figure 7: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 8 — Figure 8: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 9 — Figure 9: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 10 — Figure 10: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 11 — Figure 11: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 12 — Figure 12: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 13 — Figure 13: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 14 — Figure 14: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 15 — Figure 15: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 16 — Figure 16: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 17 — Figure 17: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 18 — Figure 18: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 19 — Figure 19: A random sample of images labeled as each concept, shown to illustrate that the labels are unreliable.

Panel of Figure 20 — Figure 20: The accuracy gap $\Delta(s)$ is shown in the black dotted line (left axis, real scale). Average absolute concept attributions for each method are shown on the right-hand axis, min-max-normalized for visual comparability.

Metric	COCO-Stuff	CelebA
TCAV	-0.42	0.27
PCBM (Resnet-18)	-0.28	-0.84
PCBM (CLIP-RN50)	-0.62	0.71
PCBM (SigLIP)	-0.63	0.66
Hide and Seek (ours)	0.82	0.85

Appendix

Table 11: Pearson correlation between the accuracy gap

\Delta(s)

and mean absolute concept attributions across the 11 shortcut rates (

s\in\{0\%,10\%,\ldots,100\%\}

Method	Attractive	Male	Average
TCAV	0.478	0.160	0.319
PCBM (ResNet-18)	0.131	0.228	0.180
PCBM (CLIP-RN50)	0.267	0.032	0.149
PCBM (SigLIP)	0.320	0.096	0.208
Hide and Seek (UTKFace)	0.501	0.426	0.463
Hide and Seek (LFW)	0.641	0.502	0.571

Appendix

Table 12: Concept attribution agreement with the official-train classifier’s masking ground truth, averaged across the Attractive and Male tasks (previously_used ground truth).

Panel of Figure 21 — Figure 21: Example images from UTKFace (left) and CelebA (right). UTKFace’s crops are tight to the face, leaving little to no visible neck or hairline, so concepts like Bald , Necklace , and Receding Hairline are frequently absent from the frame entirely. CelebA’s native crops are looser, consistently including hair, neck, and shoulders, so these same concepts remain visible.

Panel of Figure 22 — Figure 22: Top-3 (highest $|\mathrm{score}|$ ) and bottom-3 (lowest $|\mathrm{score}|$ ) Male-task images for HnS and TCAV, for each concept shown.

Panel of Figure 23 — Figure 23: Top-3 (highest $|\mathrm{score}|$ ) and bottom-3 (lowest $|\mathrm{score}|$ ) Male-task images for HnS and TCAV, for each concept shown.

Panel of Figure 24 — Figure 24: Top-3 (highest $|\mathrm{score}|$ ) and bottom-3 (lowest $|\mathrm{score}|$ ) Male-task images for HnS and TCAV, for each concept shown.

Panel of Figure 25 — Figure 25: Top-3 (highest $|\mathrm{score}|$ ) and bottom-3 (lowest $|\mathrm{score}|$ ) Male-task images for HnS and TCAV, for each concept shown.

Panel of Figure 26 — Figure 26: Top-3 (highest $|\mathrm{score}|$ ) and bottom-3 (lowest $|\mathrm{score}|$ ) Male-task images for HnS and TCAV, for each concept shown.

Panel of Figure 27 — Figure 27: Top-3 (highest $|\mathrm{score}|$ ) and bottom-3 (lowest $|\mathrm{score}|$ ) images for HnS and TCAV, for each (task class, concept) pair shown.

Panel of Figure 28 — Figure 28: Top-3 (highest $|\mathrm{score}|$ ) and bottom-3 (lowest $|\mathrm{score}|$ ) images for HnS and TCAV, for each (task class, concept) pair shown.

Panel of Figure 29 — Figure 29: Top-3 (highest $|\mathrm{score}|$ ) and bottom-3 (lowest $|\mathrm{score}|$ ) images for HnS and TCAV, for each (task class, concept) pair shown.

Panel of Figure 30 — Figure 30: Top-3 (highest $|\mathrm{score}|$ ) and bottom-3 (lowest $|\mathrm{score}|$ ) images for HnS and TCAV, for each (task class, concept) pair shown.

Panel of Figure 31 — Figure 31: Top-3 (highest $|\mathrm{score}|$ ) and bottom-3 (lowest $|\mathrm{score}|$ ) images for HnS and TCAV, for each (task class, concept) pair shown.

Panel of Figure 32 — Figure 32: Per-model attribution for the top consensus masculine-leaning concept ( tie , left) and top consensus feminine-leaning concept ( cake , right). Bar color shows sign (blue = masculine-leaning, pink = feminine-leaning). 9/10 models agree on tie ’s masculine direction (BakLLaVA is the lone, near-zero exception); 7/10 models agree on cake ’s feminine direction, with the 3 disagreeing models (LLaVA, BakLLaVA, GPT-4o) all showing a positive value instead.

Figure 33: Hide and Seek’s land-cover concept attributions for the FTW field-segmentation model, across the 14 countries with sufficient retrieved locations. Each cell is the mean drop in the model’s predicted field score when that concept’s most-associated region is masked (normalized by 255), colored on a diverging scale: red indicates the concept supports a “field” prediction (e.g., Arable land), blue indicates it suppresses one (e.g., Urban fabric). Concepts are ordered left to right by their mean effect across countries; countries are ordered top to bottom by mean $|\text{effect}|$ . The agriculture-supports/urban-suppresses gradient holds consistently across most countries.

Explore similar work

Mar 13, 2025cs.CV

VisionLogic: Discovering and Grounding Decision-Relevant Visual Concepts

Concept-based explanations help users understand vision models through recognizable visual patterns. However, existing methods often rely on correlational signals without directly validating which image cues support prediction-relevant internal features. To this end, we introduce VisionLogic, a post-hoc framework that grounds these features in visual concepts through intervention-based validation. VisionLogic first identifies compact sets of features whose contributions reproduce the model's original prediction. It then represents their activation states as predicates using class-specific thresholds. An iterative refinement procedure grounds these predicates in visual regions through ablation tests. A region is accepted when its removal deactivates the corresponding predicate, linking the feature's numerical role to visual evidence in the input. The same predicates allow us to examine how features are activated, selected, and reused across images and classes. Across CNNs and vision transformers on ImageNet-1k, we find that only a few features are selected to explain each prediction, and frequently active features are not always selected. In a large-scale human evaluation with 465 participants, VisionLogic significantly improves participants' understanding of model behavior over established methods ACE and CRAFT. Code is available at https://github.com/allengeng123/VisionLogic.

Chuqin Geng, Yuhe Jiang, Li Zhang +5

School of Computer Science, McGill University, Montreal, Canada · Department of Computer Science, University of Toronto, Toronto, Canada

Explainable Artificial Intelligence Concept-Based Explanations Explainable Image Classification Visual Grounding Explainability Evaluation

Ask PDF

May 7, 2026cs.LG

Concept-Based Abductive and Contrastive Explanations for Behaviors of Vision Models

Concept-based explanations offer a promising approach for explaining the predictions of deep neural networks in terms of high-level, human-understandable concepts. However, existing methods either do not establish a causal connection between the concepts and model predictions or are limited in expressivity and only able to infer causal explanations involving single concepts. At the same time, the parallel line of work on formal abductive and contrastive explanations computes the minimal set of input features causally relevant for model outcomes but only considers low-level features such as pixels. Merging these two threads, in this work, we propose the notion of concept-based abductive and contrastive explanations that capture the minimal sets of high-level concepts causally relevant for model outcomes. We then present a family of algorithms that enumerate all minimal explanations while using concept erasure procedures to establish causal relationships. By appropriately aggregating such explanations, we are not only able to understand model predictions on individual images but also on collections of images where the model exhibits a user-specified, common behavior. We evaluate our approach on multiple models, datasets, and behaviors, and demonstrate its effectiveness in computing helpful, user-friendly explanations.

Ronaldo Canizales, Divya Gopinath, Corina Păsăreanu +1

Colorado State University, Fort Collins CO, USA · KBR Inc., NASA Ames, Moffett Field CA, USA · Carnegie Mellon University, Pittsburgh PA, USA

Abductive Reasoning Concept-Based Explanations Neural Network Interpretability Explainable Image Classification Causal Interpretability

Ask PDF

Oct 3, 2026cs.CV

ConEx: Human-Interpretable Saliency Maps via Concept-Aware Attribution

Many visual explanation methods in computer vision highlight pixel importance but struggle to link these low-level cues to semantically meaningful concepts, limiting their interpretability and trustworthiness. We introduce Concept-based Explanations (ConEx), a novel framework that bridges saliency visualization with concept-based reasoning to provide both faithfulness and interpretability. ConEx automatically discovers class-specific concepts and represents them through concept activation vectors (CAVs), learned without manual supervision using an architecture-specific masking mechanism that reduces noise introduced by the segmentation masks to enhance concept purity. ConEx generates faithful saliency maps that reveal where each concept appears in the image and how it contributes to the prediction. To evaluate the reliability of these learned concepts, we propose two complementary metrics, Vector-Concept Match (VCM) and Concept-Class Match (CCM), that quantify concept alignment and enable direct comparison with existing methods. Extensive experiments across diverse settings demonstrate that ConEx achieves state-of-the-art performance on faithfulness, segmentation, and concept-quality benchmarks. Overall, ConEx advances the field toward truly interpretable and concept-grounded explanations in vision models.

Yehonatan Elisha, Oren Barkan, Ziv Weiss Haddad +1

Tel Aviv University · The Open University

Concept-Based Explanations Explainable Image Classification Saliency Map Evaluation

Ask PDF