cs.LGFeb 27, 2026

Hierarchical Concept-based Interpretable Models

Authors: Oscar Hill, Mateo Espinosa Zarlenga, Mateja Jamnik

Organizations: University of Cambridge · University of Oxford

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Modern deep neural networks remain challenging to interpret due to the opacity of their latent representations, impeding model understanding, debugging, and debiasing. Concept Embedding Models (CEMs) address this by mapping inputs to human-interpretable concept representations from which tasks can be predicted. Yet, CEMs fail to represent inter-concept relationships and require concept annotations at different granularities during training, limiting their applicability. In this paper, we introduce Hierarchical Concept Embedding Models (HiCEMs), a new family of CEMs that explicitly model concept relationships through hierarchical structures. To enable HiCEMs in real-world settings, we propose Concept Splitting, a method for automatically discovering finer-grained sub-concepts from a pretrained CEM's embedding space without requiring additional annotations. This allows HiCEMs to generate fine-grained explanations from limited concept labels, reducing annotation burdens. Our evaluation across multiple datasets, including a user study and experiments on PseudoKitchens, a newly proposed concept-based dataset of 3D kitchen renders, demonstrates that (1) Concept Splitting discovers human-interpretable sub-concepts absent during training that can be used to train highly accurate HiCEMs, and (2) HiCEMs enable powerful test-time concept interventions at different granularities, leading to improved task accuracy.

Figures & tables

Figure 1: Concept Splitting . (a) Train a CEM and calculate concept embeddings. (b) Train SAEs on the embeddings (the image depicts a single embedding set). (c) Create concept labels. The green points are marked as having the new concept, and the black points (where the parent concept is not active or where the SAE feature is not active) are marked as not having the new concept.
Figure 2: Hierarchical CEM: as in a CEM, from a latent code h\mathbf{h} , we learn two embeddings per concept ( c^i+′\mathbf{\hat{c}_{i}^{+\prime}} and c^i−′\mathbf{\hat{c}_{i}^{-\prime}} ). These embeddings are then passed through sub-concepts modules (Figure 3 ), which produce new embeddings ( c^i+\mathbf{\hat{c}_{i}^{+}} and c^i−\mathbf{\hat{c}_{i}^{-}} ) that include information about sub-concepts. The sub-concepts modules also output the most likely sub-concept probabilities, which are used to calculate top-level concept probabilities. These probabilities are used to output an embedding for each concept via a weighted mixture of positive and negative embeddings.
Figure 3: Positive sub-concepts module: positive sub-concept embedding generators ϕkj+\phi_{kj}^{+} (illustrated for “contains apples” and “contains pears”) produce sub-concept embeddings from the preliminary parent concept embedding c^k+′\mathbf{\hat{c}}_{k}^{+\prime} . A shared scoring function s(⋅)s(\cdot) predicts probabilities for each sub-concept. The positive parent concept embedding c^k+\mathbf{\hat{c}}_{k}^{+} is computed as a weighted mixture of sub-concept embeddings, whilst an estimate for the parent concept probability p^k+\hat{p}_{k}^{+} is obtained via a differentiable soft maximum operation over the sub-concept probabilities.
 MNIST-ADDSHAPESCUBAwA2PseudoKitchens
LF-CBM–0.75±0.000.75_{\pm 0.00}0.77±0.000.77_{\pm 0.00}0.78±0.000.78_{\pm 0.00}–
HiCEM w/o concept splitting (control)0.86±0.010.86_{\pm 0.01}0.88±0.020.88_{\pm 0.02}0.81±0.000.81_{\pm 0.00}0.73±0.000.73_{\pm 0.00}0.75±0.060.75_{\pm 0.06}
CEM + Concept Splitting (ours)0.93±0.01\mathbf{0.93_{\pm 0.01}}0.93±0.01\mathbf{0.93_{\pm 0.01}}0.85±0.01\mathbf{0.85_{\pm 0.01}}0.88±0.01\mathbf{0.88_{\pm 0.01}}0.88±0.01\mathbf{0.88_{\pm 0.01}}
HiCEM + Concept Splitting (ours)0.93±0.01\mathbf{0.93_{\pm 0.01}}0.93±0.01\mathbf{0.93_{\pm 0.01}}0.85±0.01\mathbf{0.85_{\pm 0.01}}0.88±0.01\mathbf{0.88_{\pm 0.01}}0.88±0.00\mathbf{0.88_{\pm 0.00}}
Table 1: Mean ROC-AUC for discovered concepts. LF-CBMs were unable to discover concepts on the PseudoKitchens dataset. Sub-concepts discovered with Concept Splitting are predicted accurately.
 YesNoNot sureThey are the same
Sub-concept relationship (control)16 (3.8%)387 (91.3%)20 (4.7%)1 (0.2%)
Sub-concept relationship (experimental)241 (60.6%)109 (27.4%)19 (4.8%)29 (7.3%)
Image labels (control)4 (0.9%)424 (97.9%)5 (1.2%)–
Image labels (experimental)244 (54.8%)172 (38.7%)29 (6.5%)–
Table 2: User study results. Users are far more likely to say sub-concepts generated by our method are examples of their parent concept than randomly chosen words are (first two rows). Users also agree that, in a lot of cases, the images labelled as having a discovered sub-concept are consistent with the automatically generated name of that sub-concept (second two rows).
 MNIST-ADDSHAPESCUBAwA2PseudoKitchensImageNet
Black box (not interpretable)0.94±0.000.94_{\pm 0.00}0.89±0.000.89_{\pm 0.00}0.80±0.000.80_{\pm 0.00}0.98±0.000.98_{\pm 0.00}0.67±0.010.67_{\pm 0.01}0.770.77
LF-CBM–0.59±0.010.59_{\pm 0.01}0.80±0.00\mathbf{0.80_{\pm 0.00}}0.94±0.000.94_{\pm 0.00}–0.720.72 *
PCBM0.16±0.030.16_{\pm 0.03}0.54±0.010.54_{\pm 0.01}0.65±0.010.65_{\pm 0.01}0.95±0.000.95_{\pm 0.00}0.16±0.000.16_{\pm 0.00}–
PCBM-h0.53±0.010.53_{\pm 0.01}0.73±0.000.73_{\pm 0.00}0.73±0.000.73_{\pm 0.00}0.96±0.000.96_{\pm 0.00}0.52±0.010.52_{\pm 0.01}–
CBM0.23±0.010.23_{\pm 0.01}0.78±0.010.78_{\pm 0.01}0.65±0.000.65_{\pm 0.00}0.97±0.000.97_{\pm 0.00}0.66±0.01\mathbf{0.66_{\pm 0.01}}0.770.77
CEM0.92±0.010.92_{\pm 0.01}0.89±0.00\mathbf{0.89_{\pm 0.00}}0.76±0.010.76_{\pm 0.01}0.98±0.00\mathbf{0.98_{\pm 0.00}}0.66±0.01\mathbf{0.66_{\pm 0.01}}0.79\mathbf{0.79}
Table 3: Task accuracies. The task accuracy of HiCEMs is competitive with all our baselines. * indicates reported accuracy, with a ResNet-50 backbone.
 MNIST-ADDSHAPESCUBAwA2PseudoKitchensImageNet
CBM0.99±0.00\mathbf{0.99_{\pm 0.00}}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.89±0.000.89_{\pm 0.00}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.91±0.000.91_{\pm 0.00}0.990.99
CEM0.99±0.00\mathbf{0.99_{\pm 0.00}}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.95±0.00\mathbf{0.95_{\pm 0.00}}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.92±0.00\mathbf{0.92_{\pm 0.00}}1.00\mathbf{1.00}
HiCEM w/o concept splitting (control)0.99±0.00\mathbf{0.99_{\pm 0.00}}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.93±0.000.93_{\pm 0.00}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.91±0.000.91_{\pm 0.00}–
CEM + Concept Splitting (ours)0.99±0.00\mathbf{0.99_{\pm 0.00}}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.95±0.00\mathbf{0.95_{\pm 0.00}}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.92±0.00\mathbf{0.92_{\pm 0.00}}–
HiCEM + Concept Splitting (ours)0.99±0.00\mathbf{0.99_{\pm 0.00}}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.93±0.010.93_{\pm 0.01}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.91±0.000.91_{\pm 0.00}0.990.99
Table 4: Mean ROC-AUCs for provided concepts. HiCEMs are able to predict provided concepts just as well as CEMs.
Figure 4: Task accuracy as discovered concepts are intervened. Intervening on discovered sub-concepts improves task accuracy. In some cases, such as in CUB and PseudoKitchens, interventions in HiCEMs lead to a greater increase in task accuracy than in CEMs trained with Concept Splitting’s discovered concepts. LF-CBMs ( Oikarinen et al., 2023 ) do not easily support interventions.
Figure 5: Change in task accuracy as provided concepts are intervened. Provided concept interventions on ImageNet are shown in Appendix I . Provided concept interventions work just as well in HiCEMs as they do in CEMs.
Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

 MNIST-ADDSHAPESCUBAwA2PseudoKitchens
HiCEM + Concept Splitting (clustering)0.94±0.02\mathbf{0.94_{\pm 0.02}}0.94±0.03\mathbf{0.94_{\pm 0.03}}0.90±0.01\mathbf{0.90_{\pm 0.01}}0.81±0.030.81_{\pm 0.03}0.86±0.030.86_{\pm 0.03}
HiCEM + Concept Splitting (SAE)0.93±0.010.93_{\pm 0.01}0.93±0.010.93_{\pm 0.01}0.85±0.010.85_{\pm 0.01}0.88±0.01\mathbf{0.88_{\pm 0.01}}0.88±0.00\mathbf{0.88_{\pm 0.00}}
Appendix
Table 5: Mean ROC-AUC for discovered concepts. The clustering method performs better in some cases, and the SAE method performs better in others.
 MNIST-ADDSHAPESCUBAwA2PseudoKitchens
HiCEM + Concept Splitting (clustering)0.93±0.00\mathbf{0.93_{\pm 0.00}}0.88±0.02\mathbf{0.88_{\pm 0.02}}0.76±0.00\mathbf{0.76_{\pm 0.00}}0.98±0.00\mathbf{0.98_{\pm 0.00}}0.63±0.000.63_{\pm 0.00}
HiCEM + Concept Splitting (SAE)0.92±0.000.92_{\pm 0.00}0.87±0.020.87_{\pm 0.02}0.74±0.010.74_{\pm 0.01}0.98±0.00\mathbf{0.98_{\pm 0.00}}0.65±0.01\mathbf{0.65_{\pm 0.01}}
Appendix
Table 6: Task accuracies. The clustering method and the SAE method perform similarly.
 MNIST-ADDSHAPESCUBAwA2PseudoKitchens
HiCEM + Concept Splitting (clustering)0.99±0.00\mathbf{0.99_{\pm 0.00}}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.93±0.00\mathbf{0.93_{\pm 0.00}}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.91±0.00\mathbf{0.91_{\pm 0.00}}
HiCEM + Concept Splitting (SAE)0.99±0.00\mathbf{0.99_{\pm 0.00}}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.93±0.01\mathbf{0.93_{\pm 0.01}}1.00±0.00\mathbf{1.00_{\pm 0.00}}0.91±0.00\mathbf{0.91_{\pm 0.00}}
Appendix
Table 7: Mean ROC-AUCs for provided concepts. The clustering and SAE methods perform identically.
Figure 6: Task accuracy as discovered concepts are intervened. Interventions are mostly effective on concepts discovered with clustering and with SAEs.
Figure 7: Change in task accuracy as provided concepts are intervened. The two methods perform similarly.
Figure 8: Images from PseudoKitchens demonstrating the dataset’s photorealistic quality and diversity in kitchen layouts, ingredient combinations, lighting conditions, and camera perspectives. Each scene contains ingredients for recipe classification.
Figure 9: An example from the PseudoKitchens dataset. Left: a photorealistic kitchen scene. Right: ground-truth concept annotations, with colours indicating the spatial locations of individual concepts.
GroupIngredients
FruitBanana, Orange, Apple, Pear, Pineapple
VegetablesOnion, Carrot, Potato, Pepper, Courgette
PastaMacaroni, Spaghetti
Appendix
Table 8: Ingredient groups in PseudoKitchens.
RecipeIngredients
Fruit SaladFruit
Vegetable PastaPasta, Onion, Garlic, Oil, Vegetables, Spice, Tin Tomatoes
RisottoCheese, Onion, Garlic, Vegetables, Oil, Spice, Rice
ChipsPotato, Oil, Flour, Garlic, Spice
ChilliMince, Oil, Onion, Garlic, Chilli, Tin Tomatoes, Spice, Rice
SmoothieMilk, Yoghurt, Fruit
Appendix
Table 9: Recipes in PseudoKitchens.
High-level conceptMerged concepts
patternedpatches, spots, stripes
distal_limbflippers, hands, hooves, pads, paws
teethchewteeth, meatteeth, buckteeth, strainteeth
weaponshorns, claws, tusks
Appendix
Table 10: High-level concepts and their corresponding merged fine-grained concepts in the AwA2 task.
Discovered conceptGround truth concept
A red eyehas_eye_color::red
Glossy black wingshas_wing_color::black
a black backhas_back_color::black
a black beakhas_bill_color::black
a black breasthas_breast_color::black
a black throathas_throat_color::black
Appendix
Table 11: Matches between concepts discovered by the LF-CBM baseline and ground truth concepts in the CUB dataset.
Figure 10: A sub-concept verification question from our user study.
Figure 11: An image label verification question from our user study.
Figure 12: A random sample of images from the MNIST-ADD training dataset that were labelled as having one of the discovered sub-concepts. The interpretation assigned to this concept was “the top digit is 6”.
 MNIST-ADD
HiCEM + Concept Splitting (one-hot embeddings)1.00±0.00\mathbf{1.00_{\pm 0.00}}
HiCEM + Concept Splitting (CEM embeddings)0.93±0.010.93_{\pm 0.01}
Appendix
Table 12: Mean ROC-AUC for discovered concepts. When concept embeddings contain perfect information about sub-concepts, our method effectively recovers them.
 MNIST-ADD
HiCEM + Concept Splitting (one-hot embeddings)0.93±0.00\mathbf{0.93_{\pm 0.00}}
HiCEM + Concept Splitting (CEM embeddings)0.92±0.000.92_{\pm 0.00}
Appendix
Table 13: Task accuracies. HiCEM achieves similar task accuracy whether trained on concepts discovered from one-hot encoded or CEM embeddings.
 MNIST-ADD
HiCEM + Concept Splitting (one-hot embeddings)1.00±0.00\mathbf{1.00_{\pm 0.00}}
HiCEM + Concept Splitting (CEM embeddings)0.99±0.000.99_{\pm 0.00}
Appendix
Table 14: Mean ROC-AUCs for provided concepts. HiCEM achieves similar provided concept accuracy whether trained on concepts discovered from one-hot encoded or CEM embeddings.
Figure 13: Interventions on discovered concepts are slightly more effective with idealised one-hot encoded embeddings, whilst provided concept interventions perform similarly in both scenarios. Mean and standard deviation over three runs are shown; the standard deviation is negligibly small.
Figure 14: Change in task accuracy as provided concepts are intervened on ImageNet. Provided concept interventions work just as well in HiCEMs as they do in CEMs.
More in this topic

Interpretable ML

Interpretable machine learning develops models and explanations that make predictions, evidence, and decision processes understandable to people.

See all 301 papers

Explore similar work

CardsList
Mar 10, 2026

Digging Deeper: Learning Multi-Level Concept Hierarchies

Although concept-based models promise interpretability by explaining predictions with human-understandable concepts, they typically rely on exhaustive annotations and treat concepts as flat and independent. To circumvent this, recent work has introduced Hierarchical Concept Embedding Models (HiCEMs) to explicitly model concept relationships, and Concept Splitting to discover sub-concepts using only coarse annotations. However, both HiCEMs and Concept Splitting are restricted to shallow hierarchies. We overcome this limitation with Multi-Level Concept Splitting (MLCS), which discovers multi-level concept hierarchies from only top-level supervision, and Deep-HiCEMs, an architecture that represents these discovered hierarchies and enables interventions at multiple levels of abstraction. Experiments across multiple datasets show that MLCS discovers human-interpretable concepts absent during training and that Deep-HiCEMs maintain high accuracy while supporting test-time concept interventions that can improve task performance.

Oscar Hill, Mateo Espinosa Zarlenga, Mateja Jamnik

Department of Computer Science and Technology, University of Cambridge, UK · Department of Computer Science, University of Oxford, UK

Interpretable MLConcept LearningHierarchical Representation Learning
AskPDF
Apr 2, 2026

Hierarchical, Interpretable, Label-Free Concept Bottleneck Model

Concept Bottleneck Models (CBMs) introduce interpretability to black-box deep learning models by predicting labels through human-understandable concepts. However, unlike humans, who identify objects at different levels of abstraction using both general and specific features, existing CBMs operate at a single semantic level in both concept and label space. We propose HIL-CBM, a Hierarchical Interpretable Label-Free Concept Bottleneck Model that extends CBMs into a hierarchical framework to enhance interpretability by more closely mirroring the human cognitive process. HIL-CBM enables classification and explanation across multiple semantic levels without requiring relational concept annotations. HIL-CBM aligns the abstraction level of concept-based explanations with that of model predictions, progressing from abstract to concrete. This is achieved by (i) introducing a gradient-based visual consistency loss that encourages abstraction layers to focus on similar spatial regions, and (ii) training dual classification heads, each operating on feature concepts at different abstraction levels. Experiments on benchmark datasets demonstrate that HIL-CBM outperforms state-of-the-art sparse CBMs in classification accuracy. Human evaluations further show that HIL-CBM provides more interpretable and accurate explanations, while maintaining a hierarchical and label-free approach to feature concepts.

Haodong Xie, Yujun Cai, Rahul Singh Maharjan +3

Department of Computer Science, University of Manchester, United Kingdom. · School of Electrical Engineering and Computer Science, the University of Queensland, Australia. · Department of Computer Science, University of California at Merced, United State.

Concept Bottleneck ModelsExplainable Image ClassificationHierarchical Representation Learning
AskPDF
Jun 17, 2026

Concept Flow Models: Anchoring Concept-Based Reasoning with Hierarchical Bottlenecks

Concept Bottleneck Models (CBMs) enhance interpretability by projecting learned features into a human-understandable concept space. Recent approaches leverage vision-language models to generate concept embeddings, reducing the need for manual concept annotations. However, these models suffer from a critical limitation: as the number of concepts approaches the embedding dimension, information leakage increases, enabling the model to exploit spurious or semantically irrelevant correlations and undermining interpretability. In this work, we propose Concept Flow Models (CFMs), which replace the flat bottleneck with a hierarchical, concept-driven decision tree. Each internal node in the hierarchy focuses on a localized subset of discriminative concepts, progressively narrowing the prediction scope. Our framework constructs decision hierarchies from visual embeddings, distributes semantic concepts at each hierarchy level, and trains differentiable concept weights through probabilistic tree traversal. Extensive experiments on diverse benchmarks demonstrate that CFMs match the predictive performance of flat CBMs, while substantially mitigating information leakage by reducing effective concept usage. Furthermore, CFMs yield stepwise decision flows that enable transparent and auditable model reasoning with hierarchical class structures.

Ya Wang, Adrian Paschke

Fraunhofer Institute for Open Communication Systems · Freie Universität Berlin

Concept Bottleneck Models
AskPDF