When Are Concept Bottleneck Model Explanations Faithful and Compact?

Stefano Teso; Emanuele Marconato; Steve Azzolin; Antonio Vergari

cs.LGOct 5, 2026

When Are Concept Bottleneck Model Explanations Faithful and Compact?

Authors: Stefano Teso, Emanuele Marconato, Steve Azzolin, Antonio Vergari

Organizations: CIMeC & DISI University of Trento · Department of Mathematical Sciences University of Copenhagen · DISI University of Trento · School of Informatics University of Edinburgh

Abstract

Concept bottleneck models (CBMs) are neural classifiers that allow to explain their decisions via high-level concepts, potentially enabling understanding, steering and debugging. However, their explanations are often derived heuristically. Building on formal explainability, we argue they should also be faithful, i.e., not misreport which concepts actually matter. We show that, for widespread CBM architectures, including recent VLM-based variants, faithful explanations must include all concepts in the bottleneck, compromising interpretability when this is large. This result applies to both heuristic and faithful-by-construction formal explanations. To encourage the existence of compact faithful explanations, we suggest i) modeling concepts probabilistically as binary or categorical random variables (rather than logits), and ii) employing per-concept training-time sparsification via group lasso (rather than regular elastic net). We also extend algorithms from formal explainability to CBMs, and show they outperform natural heuristics in terms of guarantees and explanation size. Overall, our work warns against naive interpretability claims and provides formal conditions and practical strategies for ensuring CBMs are as interpretable as advertised.

Figures & tables

Figure 1: Current heuristic explanation for CBMs can be unfaithful as shown for a CBM combining a neural backbone generating concept activations ${\bm{\mathrm{s}}}\in\mathcal{S}$ and a linear inference layer with per-class weights ${\bm{\mathrm{w}}}_{y}$ , with $y\in[m]$ . Left: Standard heuristics compute explanations $\mathcal{E}$ by looking at the magnitude of activations, weights, or both, cf. Eq. 2 . Right: A heuristic explanation $\mathcal{E}$ that covers the $k=2$ top weights: it justifies two decisions on the ground that the same two activations attain the same values. Yet, the decisions differ: the CBM must be looking at activations outside of $\mathcal{E}$ to produce them. Hence, $\mathcal{E}$ is insufficient to fully justify either decision.

Act. $\mathcal{S}$	Wei. $W$	Representative Models (Not Exhaustive)
$(-\infty,\infty)^{b}$	Dense	CBM ( Koh et al., 2020 ) , ProbCBM ( Kim et al., 2023 ) , MCBM ( Almudévar et al., 2026 )
$(-\infty,\infty)^{b}$	Elastic Net	LF-CBM ( Oikarinen et al., 2023 ) , VLG-CBM ( Srivastava et al., 2024 )
$[\alpha,\infty)^{b},\alpha\geq 0$	Elastic Net	Selective CBM ( Schrodi et al., 2025 )
$[-1,1]^{b}$	Dense	LaBo ( Yang et al., 2023 )
$\subseteq[0,1]^{b}$	Dense	CB2M ( Steinmann et al., 2024 ) , GlanceNets ( Marconato et al., 2022 )

Table 1: Representative CBMs grouped by domain of concept activations

\mathcal{S}

( P1 ) and structure of the inference layer’s weights

W

( P2 ). Constraining

\mathcal{S}

and

W

can make it more difficult to construct counterexamples that flip the prediction, allowing (weak)

\mathsf{AXp}

to be smaller. Few CBMs employ probabilistic activations and none group lasso.

Dataset	$m$	$b$	Cat.	Train	Val	Test
CelebA	2	39	0	162,770	19,867	19,962
Derm7pt	5	28	7	413	203	395
WBCAtt	5	31	11	6,169	1,030	3,099
CUB	200	312	28	5,394	600	5,794

Table 2: Datasets. Cat. is the # of categorical concepts.

	$F_{1}^{C}$		$F_{1}^{Y}$
Dataset	prob	logit	prob	logit
CelebA	$0.760$	$0.760$	$0.989$	$0.991$
Derm7pt	$0.676$	$0.670$	$0.653$	$0.666$
WBCAtt	$0.877$	$0.872$	$0.943$	$0.948$
CUB	$0.796$	$0.781$	$0.817$	$0.832$

Table 3: Probabilities are competitive with logits in terms of

F_{1}

score.

Panel of Figure 2 — Figure 2: Modeling concepts as probabilities shrinks $\mathsf{AXp}$ s on Derm7pt (left), WBCAtt (middle), and CUB (right). Top row: logits. Bottom row: probabilities. CelebA is left to Section D.2 .

	$F_{1}^{C}$	$F_{1}^{Y}$
Dataset		dense	enet	glasso
CUB	$0.796$	$0.817$	$0.759$	$0.756$
Derm7pt	$0.676$	$0.653$	$0.648$	$0.638$
WBCAtt	$0.877$	$0.943$	$0.916$	$0.927$
LF-CBM	–	$0.545$	$0.589$	$0.564$
VLG-CBM	–	$0.626$	$0.632$	$0.576$

Table 4: glasso shrinks faithful explanations at less than

10\%

F_{1}^{Y}

cost . VLM-CBMs have no

F_{1}^{C}

as they have no ground-truth annotations.

	Derm7pt ( $b=28$ )				WBCAtt ( $b=31$ )				CUB ( $b=312$ )
	Act	Wei	Prod	EA+	Act	Wei	Prod	EA+	Act	Wei	Prod	EA+
dense	$34.1$	$47.3$	$35.1$	$29.6$	$34.0$	$57.6$	$35.6$	$26.3$	$16.6$	$28.6$	$18.5$	$15.4$
enet	$27.7$	$36.9$	${\color[rgb]{1,0,0}36.9}$	$9.5$	$29.9$	$31.0$	$30.9$	7.3	13.1	${\color[rgb]{1,0,0}87.0}$	${\color[rgb]{1,0,0}87.0}$	$10.2$
glasso	27.4	8.1	7.8	7.3	28.1	11.7	13.6	$8.1$	$14.3$	9.8	9.4	7.7
	LF-CBM ( $b=202$ )				VLG-CBM ( $b=419$ )				LaBo ( $b=10{,}000$ )
	Act	Wei	Prod	EA+	Act	Wei	Prod	EA+	Act	Wei	Prod	EA+

Table 5: glasso consistently yields smaller faithful explanations than enet . Explanation size is reported as a percentage ( i.e. , normalized by bottleneck size

b

) for concision. Bold indicates best in column and red no shrinkage w.r.t. the dense baseline.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8

		CBM				CBM + glasso
		EA		EA+		EA		EA+
Dataset	$b$	Size	Calls	Size	Calls	Size	Calls	Size	Calls
CelebA	$39$	20.8 $\pm$ 4.6	$39$	12.6 $\pm$ 3.7	$102$	–	–	–	–
Derm7pt	$28$	17.1 $\pm$ 1.8	$28$	8.3 $\pm$ 2.0	$53$	3.1 $\pm$ 2.9	$28$	2.1 $\pm$ 0.8	$33$
WBCAtt	$31$	9.9 $\pm$ 3.6	$31$	8.2 $\pm$ 3.0	$60$	2.6 $\pm$ 0.9	$31$	2.4 $\pm$ 0.8	$37$
CUB	$312$	169.5 $\pm$ 28.5	$312$	46.3 $\pm$ 18.6	$221$	47.0 $\pm$ 9.9	$312$	23.5 $\pm$ 3.6	$134$

Appendix

Table 6: Comparison between EA and our EA+ on regular CBMs with prob. activations. Bold indicates a size improvement of at least one activation.

		VLM-CBM				VLM-CBM + glasso
		EA		EA+		EA		EA+
Dataset	$b$	Size	Calls	Size	Calls	Size	Calls	Size	Calls
LF-CBM	$202$	201.0 $\pm$ 0.8	$202$	200.9 $\pm$ 0.9	$633$	36.9 $\pm$ 5.8	$202$	35.3 $\pm$ 5.9	$308$
VLG-CBM	$419$	416.5 $\pm$ 1.5	$419$	416.3 $\pm$ 1.5	$1286$	78.0 $\pm$ 8.4	$419$	66.5 $\pm$ 11.3	$593$

Appendix

Table 7: Comparison between EA and our EA+ on VLM-CBMs. Bold indicates a size improvement of at least one activation.

		CBM				CBM + glasso
		$\textsc{TopK}^{*}$		EA+		$\textsc{TopK}^{*}$		EA+
Dataset	$b$	Calls	Time	Calls	Time	Calls	Time	Calls	Time
CelebA	$39$	$6$	$0.001$	$102$	$0.007$	–	–	–	–
Derm7pt	$28$	$6$	$0.004$	$53$	$0.025$	$6$	$0.004$	$33$	$0.016$
WBCAtt	$31$	$6$	$0.005$	$60$	$0.040$	$6$	$0.006$	$37$	$0.026$
CUB	$312$	$9$	$0.704$	$221$	$16.607$	$9$	$0.704$	$134$	$10.667$

Appendix

Table 8: Runtime cost of all algorithms for regular CBMs . Time is measured in seconds.

		VLM-CBM				VLM-CBM + glasso
		$\textsc{TopK}^{*}$		EA+		$\textsc{TopK}^{*}$		EA+
Dataset	$b$	Calls	Time	Calls	Time	Calls	Time	Calls	Time
LF-CBM	$202$	$8$	$0.005$	$633$	$0.188$	$8$	$0.006$	$308$	$0.147$
VLG-CBM	$419$	$9$	$0.012$	$1286$	$1.095$	$9$	$0.020$	$593$	$1.215$

Appendix

Table 9: Runtime cost of all algorithms for VLM-CBMs . Time is measured in seconds.

	CBM				CBM + glasso
Dataset	Act	Wei	Prod	EA+	Act	Wei	Prod	EA+
CelebA	35.0 $\pm$ 3.8	15.7 $\pm$ 4.9	29.6 $\pm$ 5.1	12.6 $\pm$ 3.7	–	–	–	–
Derm7pt	9.5 $\pm$ 2.8	13.4 $\pm$ 5.0	9.7 $\pm$ 2.5	8.3 $\pm$ 2.0	7.7 $\pm$ 1.8	2.3 $\pm$ 0.7	2.2 $\pm$ 0.9	2.1 $\pm$ 0.8
WBCAtt	10.7 $\pm$ 4.3	17.5 $\pm$ 4.8	11.0 $\pm$ 4.9	8.2 $\pm$ 3.0	8.7 $\pm$ 3.2	3.5 $\pm$ 0.9	4.1 $\pm$ 1.3	2.4 $\pm$ 0.8
CUB	50.3 $\pm$ 19.0	84.7 $\pm$ 36.8	55.4 $\pm$ 21.8	46.3 $\pm$ 18.6	44.4 $\pm$ 14.6	30.0 $\pm$ 5.2	28.2 $\pm$ 4.3	23.5 $\pm$ 3.6

Appendix

Table 10: Comparison between explanation sizes across datasets for regular CBMs.

	CBM				CBM + glasso
Dataset	Act	Wei	Prod	EA+	Act	Wei	Prod	EA+
LF-CBM	202.0 $\pm$ 0.2	201.8 $\pm$ 0.5	201.8 $\pm$ 0.5	200.9 $\pm$ 0.9	201.3 $\pm$ 1.3	39.0 $\pm$ 6.8	39.1 $\pm$ 6.8	35.3 $\pm$ 5.9
VLG-CBM	418.6 $\pm$ 0.6	418.1 $\pm$ 1.1	418.2 $\pm$ 1.1	416.3 $\pm$ 1.5	418.0 $\pm$ 2.1	71.6 $\pm$ 12.2	71.7 $\pm$ 12.2	66.5 $\pm$ 11.3

Appendix

Table 11: Comparison between explanation sizes for CUB using VLM-CBM vocabularies.

Dataset	Act	Wei	Prod	EA+
	TopK
CelebA	$0\%$	$10\%$	$1\%$	${\bf 100}\%$
Derm7pt	$37\%$	$2\%$	$37\%$	${\bf 100}\%$
WBCAtt	$9\%$	$0\%$	$12\%$	${\bf 100}\%$
CUB	$5\%$	$1\%$	$5\%$	${\bf 100}\%$
Average	$12.8\%$	$3.5\%$	$13.8\%$	${\bf 100}\%$

Appendix

Table 12: Whereas EA+ always computes

\mathsf{AXp}

s, heuristics do not . Left : dense CBMs. Right : glasso -sparsified CBMs.

Panel of Figure 3 — Figure 3: Modeling concepts as probabilities shrinks $\mathsf{AXp}$ s the most on all datasets. From the top: CelebA , Derm7pt , WBCAtt , CUB ). Left to right: logits, ReLU, probabilities.

Panel of Figure 4 — Figure 4: enet (middle) works partially, glasso (below) works consistently.

Panel of Figure 5 — Figure 5: Q3: For VLM-CBMs, glasso outperforms other sparsification strategies , as shown here with CUB . Each VLM-CBM employs its own LLM-generated vocabulary, with LaBo ’s sporting $10,000$ activations.

Panel of Figure 6 — Figure 6: Selective CBMs behave like ReLU CBMs and worse than sparsified CBMs in terms of $\mathsf{AXp}$ size, shown here for CelebA (top) and WBCAtt (bottom).

Explore similar work

Hoeffding Concept Bottleneck Models with Applications to Overhead Images
May 22, 2026Clément Bénard, Manon Arfib, Christophe Labreuche +1Dermoscopic Concept Bottleneck Models Concept Bottleneck Models
Thales cortAIx-Labs, Palaiseau, France · Université Paris-Saclay, CentraleSupélec, Gif-sur-Yvette, France
Are Concept Bottleneck Models Effective as Decision-Support Systems?
Aug 26, 2026Alessandro Bogani, Nicola Debole, Emanuele Marconato +3Dermoscopic Concept Bottleneck Models Concept Bottleneck Models
DISI, University of Trento, Italy · CIMeC, University of Trento, Italy
Hyperbolic Concept Bottleneck Models
May 7, 2026Daniel Uyterlinde, Swasti Shreya Mishra, Pascal MettesDermoscopic Concept Bottleneck Models Concept Bottleneck Models
Informatics Institute, University of Amsterdam, The Netherlands

May 22, 2026, cs.LGJ/K move · Enter open · S save

Hoeffding Concept Bottleneck Models with Applications to Overhead Images

Clément Bénard, Manon Arfib, Christophe Labreuche, Victor Quétu

Thales cortAIx-Labs, Palaiseau, France · Université Paris-Saclay, CentraleSupélec, Gif-sur-Yvette, France

Dermoscopic Concept Bottleneck Models Concept Bottleneck Models Explainability Deep Learning Bottlenecks

Open paper AskPDF arXiv

Explainability of deep learning algorithms is critical for computer-vision applications with high-stake decisions. Concept bottleneck models (CBM) have recently shown promising performance to provide explainable and accurate predictions for classification problems, based on a bottleneck of high-level concepts. Existing CBM methods rely on a linear aggregation of the concept scores to compute predictions. However, a large number of concepts is often used in this linear approach, which undermines explainability and favors information leakage. In general, the underlying relation between concepts and output logits is not linear. Therefore, we introduce Hoeffding Concept Bottleneck Models (HCBM), which build on the Hoeffding functional decomposition of gradient-boosted trees to provide non-linear and sparse aggregations of concept scores, and generate compact predictions using prime implicants. HCBM are proved to be robust to interconcept leakage, and outperform standard linear CBM in practice, as shown in extensive experiments. Beyond classification, HCBM can be adapted to object detection, and we focus on a challenging case with overhead images to show the high performance of HCBM in these settings.

Abstract

Explore similar work

Hoeffding Concept Bottleneck Models with Applications to Overhead Images

Are Concept Bottleneck Models Effective as Decision-Support Systems?

Hyperbolic Concept Bottleneck Models