Explaining CLIP Zero-shot Predictions Through Concepts

Onat Ozdemir; Anders Christensen; Stephan Alaniz; Zeynep Akata; Emre Akbas

cs.CVMar 30, 2026

Explaining CLIP Zero-shot Predictions Through Concepts

Authors: Onat Ozdemir, Anders Christensen, Stephan Alaniz, Zeynep Akata, Emre Akbas

Organizations: School of Informatics, University of Edinburgh · Dept. of Computer Eng., Middle East Technical University (METU) · Orbital · DTU Compute, Technical University of Denmark ·

Abstract

Large-scale vision-language models such as CLIP have achieved remarkable success in zero-shot image recognition, yet their predictions remain largely opaque to human understanding. In contrast, Concept Bottleneck Models provide interpretable intermediate representations by reasoning through human-defined concepts, but they rely on concept supervision and lack the ability to generalize to unseen classes. We introduce EZPC that bridges these two paradigms by explaining CLIP's zero-shot predictions through human-understandable concepts. Our method projects CLIP's joint image-text embeddings into a concept space learned from language descriptions, enabling faithful and transparent explanations without additional supervision. The model learns this projection via a combination of alignment and reconstruction objectives, ensuring that concept activations preserve CLIP's semantic structure while remaining interpretable. Extensive experiments on five benchmark datasets, CIFAR-100, CUB-200-2011, Places365, ImageNet-100, and ImageNet-1k, demonstrate that our approach maintains CLIP's strong zero-shot classification accuracy while providing meaningful concept-level explanations. By grounding open-vocabulary predictions in explicit semantic concepts, our method offers a principled step toward interpretable and trustworthy vision-language models. Code is available at https://github.com/oonat/ezpc.

Figures & tables

Figure 1: Overview of EZPC. CLIP image and text embeddings are projected into a shared concept space using a learnable matrix

A

. The projected representations

c_{x}

and

c_{k}

provide (i) concept-based explanations via a Hadamard product and (ii) class logits via a dot-product in concept space. Training jointly optimizes a matching loss and a reconstruction loss to preserve CLIP’s predictive behavior.

	CIFAR-100			ImageNet-100			CUB			ImageNet-1k			Places365
Model	Seen	Unseen	H	Seen	Unseen	H	Seen	Unseen	H	Seen	Unseen	H	Seen	Unseen	H
CLIP [ 24 ]	$0.370$	$0.454$	$0.408$	$0.680$	$0.707$	$0.693$	$0.468$	$0.481$	$0.474$	$0.513$	$0.548$	$0.530$	$0.350$	$0.375$	$0.362$
Z-CBM [ 32 ]	$0.319$	$0.425$	$0.365$	$0.592$	$0.579$	$0.585$	$0.183$	$0.195$	$0.189$	$0.439$	$0.486$	$0.462$	0.349	$0.365$	0.357
SpLiCE [ 3 ]	$0.248$	$0.298$	$0.270$	$0.371$	$0.409$	$0.389$	$0.100$	$0.053$	$0.070$	$0.275$	$0.331$	$0.300$	$0.276$	$0.288$	$0.282$
EZPC	0.365	0.449	0.403	0.675	0.690	0.682	0.457	0.473	0.465	0.468	0.494	0.481	$0.339$	0.366	$0.352$

Table 1: Generalized zero-shot performance across five datasets. Each dataset is partitioned into seen (80%) and unseen (20%) classes. All models use the CLIP RN50 backbone. EZPC retains strong performance compared to CLIP while introducing interpretability.

Backbone	Variant	Zero-shot		Generalized
		Seen	Unseen	Seen	Unseen	H
CLIP RN50	Base	0.706	0.855	0.680	0.707	0.693
CLIP RN50	EZPC	0.699	0.851	0.675	0.690	0.682
CLIP ViT-B/32	Base	0.729	0.887	0.703	0.715	0.709
CLIP ViT-B/32	EZPC	0.724	0.879	0.694	0.716	0.705
CLIP ViT-L/14	Base	0.839	0.925	0.821	0.836	0.828

Table 2: Effect of backbone architecture on zero-shot and generalized zero-shot performance. Larger backbones consistently improve both zero-shot and generalized zero-shot performance.

	Zero-shot		Generalized Zero-shot
$\boldsymbol{\lambda}$	Seen	Unseen	Seen	Unseen	H
0.01	$0.377$	$0.508$	$0.347$	$0.371$	$0.358$
0.1	$0.654$	$0.820$	$0.626$	$0.633$	$0.630$
1	$0.699$	$0.851$	$0.675$	$0.690$	$0.682$
10	$0.707$	$0.859$	$0.681$	$0.709$	$0.695$
100	$0.706$	$0.857$	$0.680$	$0.704$	$0.692$

Table 3: Effect of the reconstruction weighting parameter

\lambda

. Larger

\lambda

values improve both zero-shot and generalized performance, with moderate to high settings giving the best results.

Panel of Figure 2 — Figure 2: Qualitative comparison of image-level explanations for different $\lambda$ values. For $\lambda=1$ , EZPC produces semantically consistent concept activations. For larger values (e.g., $\lambda=100$ ), unrelated concepts appear among the top activations.

Target Dataset	Model	Zero-shot		Generalized Zero-shot
		Seen	Unseen	Seen	Unseen	H
CIFAR-100	CLIP	$0.686$	$0.387$	$0.663$	$0.266$	$0.380$
CIFAR-100	EZPC	0.684	0.363	0.659	0.296	0.409
CUB	CLIP	$0.686$	$0.471$	$0.617$	$0.458$	$0.526$
CUB	EZPC	0.674	0.461	0.607	0.448	0.515

Table 4: Cross-dataset transfer results for EZPC trained on ImageNet-100 compared to CLIP. We evaluate on two target datasets: CIFAR-100 and CUB. Seen classes correspond to ImageNet-100 categories, while unseen classes correspond to the target dataset.

Panel of Figure 3 — Figure 3: Image-level Explanations. For each image, EZPC displays the top-10 activated concepts that contribute most to the zero-shot prediction. The highlighted concepts closely correspond to salient visual characteristics of the input images.

Panel of Figure 4 — Figure 4: Class-level Concept Explanations. For each class, we average concept activations over nine sampled images. EZPC produces coherent class signatures, highlighting concepts that characterize each category.

Panel of Figure 5 — Figure 5: Concept Clustering Results. For each concept, we retrieve the nine images with the highest activation. Clusters from ImageNet-100 and Places365 show coherent semantic structure, indicating that EZPC learns interpretable concept directions.

Figure 6: Region-level concept alignment for Indigo Bunting. Top row: positive concept ( a blue-gray body ). Bottom row: negative concept ( a red face ).

Metric	Positive Concept	Negative Concept
Pointing Accuracy $\uparrow$	$0.967\pm 0.180$	$0.017\pm 0.128$
Inside Activation Ratio $\uparrow$	$0.507\pm 0.191$	$0.031\pm 0.054$
IoU@10% $\uparrow$	$0.423\pm 0.159$	$0.019\pm 0.067$
IoU@20% $\uparrow$	$0.408\pm 0.148$	$0.044\pm 0.087$

Table 5: Quantitative evaluation of concept-region alignment using CUB ground-truth segmentation masks. Positive concept consistently localizes on the object, while unrelated (negative) concept shows near-zero alignment.

Method	Embedding (ms/img)	Full Pipeline (ms/img)	Overhead
CLIP	$0.0001\pm 0.0000$	$5.77\pm 0.55$	$1.0\times$
Z-CBM	$97.55\pm 1.33$	$542.34\pm 6.02$	$94.0\times$
SpLiCE	$4.50\pm 0.54$	$338.51\pm 4.39$	$58.7\times$
EZPC	$0.0006\pm 0.0000$	$5.90\pm 0.73$	${\sim}1.0\times$

Table 6: Inference time comparison on ImageNet-100 (NVIDIA H100). We report median per-image latency (ms) with 95% confidence intervals. EZPC adds negligible overhead (

p=0.31

, Wilcoxon signed-rank test), while SpLiCE and Z-CBM are

59\times

and

94\times

slower due to per-image optimization and retrieval.

Top- $n$	Logit Drop	Flip Rate
1	0.0306	0.059
3	0.0816	0.099
5	0.1263	0.132
10	0.2256	0.169

Table 7: Faithfulness results on ImageNet-100 (CLIP RN50). Mean logit drop and prediction flip rate after ablating the top-

n

most influential concepts.

Figure 7: Faithfulness distributions for

n{=}1,3,5,10

. Removing more highly ranked concepts yields consistently larger drops in model confidence.

Removal type	Flip Count	Flip Rate
Top-10 concepts	845	0.169
Random-10 concepts	70	0.014

Table 8: Top-10 vs Random-10 concept removal. Prediction flip counts and flip rates on ImageNet-100 (5000 samples).

Figure 8: Causal intervention analysis. Removing the top-10 influential concepts produces a substantially larger drop in the predicted class logit than removing 10 random concepts.

Metric	CLIP ( $\Phi$ )	Learned ( $A$ )
Alignment Mean / Median	-	0.651 / 0.648
Alignment Std	-	0.036
Alignment Min / Max	-	0.559 / 0.822
Total PCA Variance	0.1396	0.1047
Top-10 Variance Fraction	0.4586	0.4364
Off-diagonal Mean ( $\Phi^{\top}\Phi$ vs. $A^{\top}A$ )	0.6920	0.7461

Table 9: Summary of concept space statistics. All measurements are computed on ImageNet-100 using CLIP RN50. The learned space

A

preserves semantic identity while becoming more compact and uniformly correlated.

Figure 9: Alignment distribution between learned concept directions

A_{j}

and original CLIP directions

\Phi_{j}

. High alignment values indicate semantic consistency between the learned space and CLIP.

Figure 10: PCA comparison between CLIP concept space ( $\Phi$ ) and learned space ( $A$ ). The learned space is more compact yet preserves the structural layout of concept groups.

Figure 11: Activation density : number of active concepts per image.

Figure 12: Activation coverage : number of images activating each concept.

Dataset	Top-1 Agree. (%)	Spearman	Kendall	KL
CIFAR-100	80.26	0.959	0.733	$6.79\times 10^{-6}$
IN-100	92.92	0.994	0.904	$6.10\times 10^{-6}$
CUB	84.03	0.994	0.876	$6.83\times 10^{-6}$
ImageNet-1k	72.37	0.924	0.536	$7.98\times 10^{-5}$
Places365	79.92	0.972	0.715	$1.46\times 10^{-5}$

Table 10: Prediction-level fidelity between CLIP and EZPC (RN50 backbone) with label consistency, ranking preservation, and KL. Top-1 agreement measures label consistency. Spearman and Kendall correlations quantify ranking preservation. Kendall correlation is computed over the top-50 CLIP-ranked classes.

$m$	Top-1 Agree. (%)	Seen Acc.	Unseen Acc.	H
250	$69.05\pm 0.34$	$0.5527\pm 0.0027$	$0.5637\pm 0.0105$	$0.5581\pm 0.0054$
500	$80.95\pm 1.35$	$0.6252\pm 0.0064$	$0.6423\pm 0.0090$	$0.6336\pm 0.0072$
1000	$89.09\pm 0.33$	$0.6622\pm 0.0028$	$0.6827\pm 0.0032$	$0.6723\pm 0.0030$
2000	$91.73\pm 0.17$	$0.6702\pm 0.0017$	$0.6890\pm 0.0026$	$0.6795\pm 0.0004$
3000	$92.58\pm 0.26$	$0.6742\pm 0.0010$	$0.6900\pm 0.0026$	$0.6820\pm 0.0010$
4751 (full)	$92.92$	$0.6745$	$0.6900$	$0.6821$

Table 11: Effect of concept vocabulary size

m

on performance, tested on IN-100 under the generalized ZSL setting using the RN50 backbone.

Training Setting	Seen Acc.	Unseen Acc.	H
$A=\Phi$ (0-step, no training)	0.013	0.0	0.0
Matching loss only ( $\mathcal{L}_{\text{match}}$ )	0.013	0.0	0.0
Reconstruction loss only ( $\mathcal{L}_{\text{recon}}$ )	0.680	0.708	0.693
Full objective ( $\mathcal{L}_{\text{match}}+\lambda\mathcal{L}_{\text{recon}}$ )	0.674	0.690	0.682

Table 12: Ablation study on the training objective of projection matrix

A

using RN50 on IN-100 (generalized zero-shot setting).

Figure 13: CIFAR-100 image-level explanations.

Figure 14: ImageNet-100 image-level explanations.

Figure 15: CUB image-level explanations.

Figure 16: Places365 image-level explanations.

Figure 17: CIFAR-100 class-level visualizations.

Figure 18: ImageNet-100 class-level visualizations.

Figure 19: CUB class-level visualizations.

Figure 20: Places365 class-level visualizations.

Panel of Figure 21 — Figure 21: Concept clustering examples across datasets.

Panel of Figure 22 — Figure 22: Region-level concept alignment examples across datasets.

VLM Interpretability

Vision-language model interpretability explains how image and language representations shape predictions, evidence use, and multimodal reasoning.

See all 268 papers

Multimodal Concept Bottleneck Models
Jun 18, 2026Tongqing Shi, Ge Yan, Tuomas Oikarinen +1Concept Bottleneck Models VLM Interpretability
UC San Diego
Beyond Heuristic Prompting: A Concept-Guided Bayesian Framework for Zero-Shot Image Recognition
Mar 9, 2026Hui Liu, Kecheng Chen, Jialiang Wang +3Zero-Shot Image Classification Zero-Shot Learning
City University of Hong Kong · Harbin Institute of Technology · Nanyang Technological University
Concept-Constrained Prompt Learning for Few-Shot CLIP Adaptation
Jun 21, 2026Na Sang, Ding Ma, Rui Sang +1Prompt Learning VLM Adaptation
University of California, San Diego, La Jolla, CA, USA · Georgia Institute of Technology, Atlanta, GA, USA · Independent Researcher

Jun 18, 2026J/K move · Enter open · S save

Multimodal Concept Bottleneck Models

Tongqing Shi, Ge Yan, Tuomas Oikarinen, Tsui-Wei Weng

UC San Diego

Concept Bottleneck Models VLM Interpretability Zero-Shot Image Classification Multimodal Model Interpretability Image Retrieval

Open paper AskPDF arXiv

Concept Bottleneck Models (CBMs) enhance the interpretability of deep learning networks by aligning the features extracted from images with natural concepts. However, existing CBMs are constrained in their ability to generalize beyond a fixed set of predefined classes and the risk of non-concept information leakage, where predictive signals outside the intended concepts are inadvertently exploited. In this paper, we propose Multimodal Concept Bottleneck Model (MM-CBM) to address these issues and extend CBMs into CLIP. MM-CBM utilizes dual Concept Bottleneck Layers (CBLs) to align both the image and text embeddings into interpretable features. This allows us to perform new vision tasks like zero-shot classification or image retrieval in an interpretable way. Compared to existing methods, MM-CBM achieves up to 51.26% accuracy improvement on average across four standard benchmarks. Our method maintains high accuracy, staying within ~5% of black-box performance while offering greater interpretability.

Abstract

VLM Interpretability

Explore similar work

Multimodal Concept Bottleneck Models

Beyond Heuristic Prompting: A Concept-Guided Bayesian Framework for Zero-Shot Image Recognition

Concept-Constrained Prompt Learning for Few-Shot CLIP Adaptation