HyDI: A hybrid Deep Learning-Inductive Logic Programming ensemble for multi-label classification

Simon Flügel; Till Mossakowski

cs.LGSep 29, 2026

HyDI: A hybrid Deep Learning-Inductive Logic Programming ensemble for multi-label classification

Authors: Simon Flügel, Till Mossakowski

Organizations: Institute for Computer Science, University of Osnabrück, Neuer Graben 29, 49074 Osnabrück, Germany

Abstract

While attaining remarkable results for many applications, Deep Learning models are notoriously difficult to explain. This work introduces HyDI, a hybrid ensemble architecture for hierarchical multi-label classification. It combines a Deep Learning (DL) model with rule-based classifiers generated by Inductive Logic Programming (ILP). For leaf classes of the label hierarchy, the rule-based classifiers replace the DL model, leading to more transparent classification results. HyDI is applied to the Chemical Entities of Biological Interest (ChEBI) ontology, providing ILP-generated rules for 314 classes. For these classes, HyDI can generate global explanations as well as local explanations that combine visual and text-based descriptions.

Figures & tables

Figure 1: Abstract visualisation of how HyDI is applied to a HMLC problem. Non-leaf classes (highlighted in yellow) are predicted by a DL model. Leaf classes (highlighted in blue) are predicted by either a DL model or an ILP rule (depending on performance).

Figure 2: Representation of a molecule in the augmented graph used by the augmented GAT model. Figure taken from [ 13 ] , adapted.

Predicate	Description
chebi_{class_id}/1	molecule belongs to ChEBI class
has_atom/2	atom belongs to molecule
c/1 , n/1 , …	element symbols
charge_p/1 , charge_n/1	positive / negative atom charge
charge0/1 , charge_m1/1 , charge1/1 , …	specific atom charge
has_0_hs/1 , has_1_hs/1 , …	atom has this number of attached hydrogen atoms

Table 1: Predicates used to represent molecules as logic programs.

Figure 3: Representation of methanol (CH 3 OH) as a logic program.

Figure 4: Local explanation for N-methylpyrazole . It shows the molecular graph with numbered atoms and a textual description of the classification. Atoms relevant for the classification as pyrazoles are highlighted.

		All classes with rules			Shared classes
ILP configuration	#rules	Precision	Recall	F1 score	Precision	Recall	F1 score
ILP-FN	666	0.677	0.809	0.680	0.712	0.877	0.730
ILP-FP	520	0.732	0.666	0.640	0.734	0.667	0.642
ILP-REG	636	0.699	0.774	0.681	0.731	0.814	0.716

Table 2: Performance of all ILP-generated rules (independent of the ensemble). Shared classes are the 508 classes for which each ILP configuration has produced a rule. All metrics are macro-aggregated.

DL	0.929	0.910	0.920	0.784	0.734	0.737
	Micro-aggregated			Macro-aggregated
Model	Precision	Recall	F1 score	Precision	Recall	F1 score
HyDI	0.919	0.910	0.914	0.782	0.729	0.731

Table 3: Performance of the DL model vs. the HyDI ensemble on the ChEBI25-3STAR dataset.

		Precision		Recall		F1 score
Class subset	# classes	DL	HyDI	DL	HyDI	DL	HyDI
DL	1,494	0.794	0.794	0.741	0.741	0.746	0.746
ILP-FN	260	0.730	0.735	0.712	0.700	0.698	0.680
ILP-REG	24	0.766	0.558	0.640	0.524	0.674	0.481
ILP-FP	30	0.738	0.732	0.664	0.602	0.668	0.613

Table 4: Performance for the ChEBI subsets predicted by each ILP type / DL, macro-aggregated.

Multi-Label Classification

Multi-label classification assigns several potentially related tags to each example, from medical findings and topics to objects and user interests.

See all 132 papers

Digging Deeper: Learning Multi-Level Concept Hierarchies
Mar 10, 2026Oscar Hill, Mateo Espinosa Zarlenga, Mateja JamnikInterpretable ML Concept Learning
Department of Computer Science and Technology, University of Cambridge, UK · Department of Computer Science, University of Oxford, UK
Deep Arguing
May 11, 2026Adam Gould, Francesca ToniCase-Based Reasoning Neural Network Interpretability
Department of Computing Imperial College London
MOXIE: Discovering Alternative Explanations for Biomedical Image Classifiers
Oct 3, 2026Abiha Tahsin Chowdhury, Rahul DubeyExplainable Image Classification Explainable Medical Image Analysis
Department of Computer Science, Missouri State University, Springfield, MO, USA

Mar 10, 2026J/K move · Enter open · S save

Digging Deeper: Learning Multi-Level Concept Hierarchies

Oscar Hill, Mateo Espinosa Zarlenga, Mateja Jamnik

Department of Computer Science and Technology, University of Cambridge, UK · Department of Computer Science, University of Oxford, UK

Interpretable ML Concept Learning Hierarchical Representation Learning

Open paper AskPDF arXiv

Although concept-based models promise interpretability by explaining predictions with human-understandable concepts, they typically rely on exhaustive annotations and treat concepts as flat and independent. To circumvent this, recent work has introduced Hierarchical Concept Embedding Models (HiCEMs) to explicitly model concept relationships, and Concept Splitting to discover sub-concepts using only coarse annotations. However, both HiCEMs and Concept Splitting are restricted to shallow hierarchies. We overcome this limitation with Multi-Level Concept Splitting (MLCS), which discovers multi-level concept hierarchies from only top-level supervision, and Deep-HiCEMs, an architecture that represents these discovered hierarchies and enables interventions at multiple levels of abstraction. Experiments across multiple datasets show that MLCS discovers human-interpretable concepts absent during training and that Deep-HiCEMs maintain high accuracy while supporting test-time concept interventions that can improve task performance.

Abstract

Multi-Label Classification

Explore similar work

Digging Deeper: Learning Multi-Level Concept Hierarchies

Deep Arguing

MOXIE: Discovering Alternative Explanations for Biomedical Image Classifiers