Digging Deeper: Learning Multi-Level Concept Hierarchies

Oscar Hill; Mateo Espinosa Zarlenga; Mateja Jamnik

cs.LGMar 10, 2026

Digging Deeper: Learning Multi-Level Concept Hierarchies

Authors: Oscar Hill, Mateo Espinosa Zarlenga, Mateja Jamnik

Organizations: Department of Computer Science and Technology, University of Cambridge, UK · Department of Computer Science, University of Oxford, UK

Abstract

Although concept-based models promise interpretability by explaining predictions with human-understandable concepts, they typically rely on exhaustive annotations and treat concepts as flat and independent. To circumvent this, recent work has introduced Hierarchical Concept Embedding Models (HiCEMs) to explicitly model concept relationships, and Concept Splitting to discover sub-concepts using only coarse annotations. However, both HiCEMs and Concept Splitting are restricted to shallow hierarchies. We overcome this limitation with Multi-Level Concept Splitting (MLCS), which discovers multi-level concept hierarchies from only top-level supervision, and Deep-HiCEMs, an architecture that represents these discovered hierarchies and enables interventions at multiple levels of abstraction. Experiments across multiple datasets show that MLCS discovers human-interpretable concepts absent during training and that Deep-HiCEMs maintain high accuracy while supporting test-time concept interventions that can improve task performance.

Figures & tables

Figure 1: Deep-HiCEM: as in a HiCEM, from a latent code $\mathbf{h}$ , we learn two embeddings per concept ( $\mathbf{\hat{c}_{i}^{+\prime}}$ and $\mathbf{\hat{c}_{i}^{-\prime}}$ ), which are then passed through sub-concepts modules, which produce new embeddings ( $\mathbf{\hat{c}_{i}^{+}}$ and $\mathbf{\hat{c}_{i}^{-}}$ ) that include information about sub-concepts and their descendants.

	MNIST-ADD	SHAPES	CUB	AwA2	PseudoKitchens-2
LF-CBM	–	$0.75_{\pm 0.00}$	$0.77_{\pm 0.00}$	$0.78_{\pm 0.00}$	–
HiCEM + Concept Splitting	$0.93_{\pm 0.01}$	$\mathbf{0.93_{\pm 0.01}}$	$\mathbf{0.85_{\pm 0.01}}$	$\mathbf{0.88_{\pm 0.01}}$	$\mathbf{0.80_{\pm 0.00}}$
Deep-HiCEM + MLCS (ours)	$\mathbf{0.94_{\pm 0.01}}$	$\mathbf{0.93_{\pm 0.01}}$	$0.84_{\pm 0.01}$	$0.85_{\pm 0.01}$	$0.79_{\pm 0.01}$

Table 1: Mean ROC-AUC for discovered concepts. LF-CBMs were unable to discover concepts on the MNIST-ADD and PseudoKitchens-2 datasets. Concepts discovered with MLCS are predicted accurately.

	MNIST-ADD	SHAPES	CUB	AwA2	PseudoKitchens-2
Black box (not interpretable)	$0.94_{\pm 0.00}$	$0.89_{\pm 0.00}$	$0.80_{\pm 0.00}$	$0.98_{\pm 0.00}$	$0.63_{\pm 0.02}$
LF-CBM	–	$0.59_{\pm 0.01}$	$\mathbf{0.80_{\pm 0.00}}$	$0.94_{\pm 0.00}$	–
PCBM	$0.16_{\pm 0.03}$	$0.54_{\pm 0.01}$	$0.65_{\pm 0.01}$	$0.95_{\pm 0.00}$	$0.12_{\pm 0.00}$
PCBM-h	$0.53_{\pm 0.01}$	$0.73_{\pm 0.00}$	$0.73_{\pm 0.00}$	$0.96_{\pm 0.00}$	$0.48_{\pm 0.00}$
CBM	$0.23_{\pm 0.01}$	$0.78_{\pm 0.01}$	$0.65_{\pm 0.00}$	$0.97_{\pm 0.00}$	$\mathbf{0.60_{\pm 0.01}}$
CEM	$\mathbf{0.92_{\pm 0.01}}$	$\mathbf{0.89_{\pm 0.00}}$	$0.76_{\pm 0.01}$	$\mathbf{0.98_{\pm 0.00}}$	$0.59_{\pm 0.01}$

Table 2: Task accuracies. The task accuracy of Deep-HiCEMs is competitive with all our baselines.

	MNIST-ADD	SHAPES	CUB	AwA2	PseudoKitchens-2
CBM	$\mathbf{0.99_{\pm 0.00}}$	$\mathbf{1.00_{\pm 0.00}}$	$0.89_{\pm 0.00}$	$\mathbf{1.00_{\pm 0.00}}$	$0.90_{\pm 0.00}$
CEM	$\mathbf{0.99_{\pm 0.00}}$	$\mathbf{1.00_{\pm 0.00}}$	$\mathbf{0.95_{\pm 0.00}}$	$\mathbf{1.00_{\pm 0.00}}$	$\mathbf{0.91_{\pm 0.00}}$
HiCEM + Concept Splitting	$\mathbf{0.99_{\pm 0.00}}$	$\mathbf{1.00_{\pm 0.00}}$	$0.93_{\pm 0.01}$	$\mathbf{1.00_{\pm 0.00}}$	$\mathbf{0.91_{\pm 0.00}}$
Deep-HiCEM + MLCS (ours)	$\mathbf{0.99_{\pm 0.00}}$	$\mathbf{1.00_{\pm 0.00}}$	$0.94_{\pm 0.00}$	$\mathbf{1.00_{\pm 0.00}}$	$\mathbf{0.91_{\pm 0.00}}$

Table 3: Mean ROC-AUCs for provided concepts. Deep-HiCEMs are able to predict provided concepts just as well as HiCEMs.

Panel of Figure 2 — Figure 2: Task accuracy as discovered concepts are intervened on. Intervening on discovered concepts improves task accuracy, with a few exceptions that would benefit from further investigation.

Panel of Figure 3 — Figure 3: Change in task accuracy as provided concepts are intervened on. Provided concept interventions work just as well in Deep-HiCEMs as they do in HiCEMs.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Group	Ingredients
Fruit	Banana, Orange, Apple, Pear, Pineapple
Vegetables	Onion, Carrot, Potato, Pepper, Courgette
Pasta	Macaroni, Spaghetti

Appendix

Table 4: Ingredient groups in PseudoKitchens and PseudoKitchens-2.

Recipe	Ingredients
Fruit Salad	Fruit
Vegetable Pasta	Pasta, Onion, Garlic, Oil, Vegetables, Spice, Tin Tomatoes
Risotto	Cheese, Onion, Garlic, Vegetables, Oil, Spice, Rice
Chips	Potato (2, 3, 4), Oil, Flour, Garlic, Spice
Chilli	Mince, Oil, Onion, Garlic, Chilli, Tin Tomatoes, Spice, Rice
Smoothie	Milk, Yoghurt, Fruit

Appendix

Table 5: Recipes in PseudoKitchens-2.

Interpretable ML

Interpretable machine learning develops models and explanations that make predictions, evidence, and decision processes understandable to people.

See all 301 papers

Hierarchical Concept-based Interpretable Models
Feb 27, 2026Oscar Hill, Mateo Espinosa Zarlenga, Mateja JamnikInterpretable ML Concept-Based Explanations
University of Cambridge · University of Oxford
Concept Flow Models: Anchoring Concept-Based Reasoning with Hierarchical Bottlenecks
Jun 17, 2026Ya Wang, Adrian PaschkeConcept Bottleneck Models
Fraunhofer Institute for Open Communication Systems · Freie Universität Berlin
Hierarchical, Interpretable, Label-Free Concept Bottleneck Model
Apr 2, 2026Haodong Xie, Yujun Cai, Rahul Singh Maharjan +3Concept Bottleneck Models Explainable Image Classification
Department of Computer Science, University of Manchester, United Kingdom. · School of Electrical Engineering and Computer Science, the University of Queensland, Australia. · Department of Computer Science, University of California at Merced, United State.

Feb 27, 2026J/K move · Enter open · S save

Hierarchical Concept-based Interpretable Models

Oscar Hill, Mateo Espinosa Zarlenga, Mateja Jamnik

University of Cambridge · University of Oxford

Interpretable ML Concept-Based Explanations Hierarchical Representation Learning

Open paper AskPDF arXiv

Modern deep neural networks remain challenging to interpret due to the opacity of their latent representations, impeding model understanding, debugging, and debiasing. Concept Embedding Models (CEMs) address this by mapping inputs to human-interpretable concept representations from which tasks can be predicted. Yet, CEMs fail to represent inter-concept relationships and require concept annotations at different granularities during training, limiting their applicability. In this paper, we introduce Hierarchical Concept Embedding Models (HiCEMs), a new family of CEMs that explicitly model concept relationships through hierarchical structures. To enable HiCEMs in real-world settings, we propose Concept Splitting, a method for automatically discovering finer-grained sub-concepts from a pretrained CEM's embedding space without requiring additional annotations. This allows HiCEMs to generate fine-grained explanations from limited concept labels, reducing annotation burdens. Our evaluation across multiple datasets, including a user study and experiments on PseudoKitchens, a newly proposed concept-based dataset of 3D kitchen renders, demonstrates that (1) Concept Splitting discovers human-interpretable sub-concepts absent during training that can be used to train highly accurate HiCEMs, and (2) HiCEMs enable powerful test-time concept interventions at different granularities, leading to improved task accuracy.

Abstract

Interpretable ML

Explore similar work

Hierarchical Concept-based Interpretable Models

Concept Flow Models: Anchoring Concept-Based Reasoning with Hierarchical Bottlenecks

Hierarchical, Interpretable, Label-Free Concept Bottleneck Model