Explainable cluster analysis: a bagging approach

Federico Maria Quetti; Elena Ballante; Silvia Figini; Paolo Giudici

stat.MLMar 20, 2026

Explainable cluster analysis: a bagging approach

Authors: Federico Maria Quetti, Elena Ballante, Silvia Figini, Paolo Giudici

Organizations: Department of Mathematics, University of Pavia, Pavia, 27100, Italy. · Department of Social and Political Sciences, University of Pavia, Pavia, 27100, Italy. · Department of Economics and Management, University of Pavia, Pavia, 27100, Italy.

Abstract

A major limitation of clustering approaches is their lack of explainability: methods rarely provide insight into which features drive the grouping of similar observations. To address this limitation, we propose an ensemble-based clustering framework that integrates bagging and feature dropout to generate feature importance scores, in analogy with feature importance mechanisms in supervised random forests. By leveraging multiple bootstrap resampling schemes and aggregating the resulting partitions, the method improves stability and robustness of the cluster definition, particularly in small-sample or noisy settings. Feature importance is assessed through an information-theoretic approach: at each step, the mutual information between each feature and the estimated cluster labels is computed and weighted by a measure of clustering validity to emphasize well-formed partitions, before being aggregated into a final score. The method outputs both a consensus partition and a corresponding measure of feature importance, enabling a unified interpretation of clustering structure and variable relevance. Its effectiveness is demonstrated on multiple simulated and real-world datasets.

Figures & tables

Figure 1: Kernel density estimation of each feature for a simulation of the synthetic dataset. A rugplot of the occurrences colored by true cluster label is shown.

Variable	Type / Distribution	Generation
$X_{1}$	Gaussian (cluster-dependent)	First coordinate of a bivariate Gaussian $\mathcal{N}(\mu_{i},\Sigma)$ , where $\mu_{i}\in\{(0,0),(3,3),(3,6)\}$ is conditional on the cluster and $\Sigma=\begin{pmatrix}0.5&0\\ 0&0.1\end{pmatrix}$ , constant for all clusters.
$X_{2}$	Gaussian (cluster-dependent)	Second coordinate of the same bivariate Gaussian distribution as $X_{1}$ . Note that, while two cluster means are the same in the first coordinate, they are not in the second. For this reason, $X_{1}$ is bimodal, and $X_{2}$ trimodal.
$X_{3}$	Gaussian (cluster-dependent)	Sampled from three independent and well separated univariate Gaussian distributions, conditionally on the cluster: $\mathcal{N}(1,0.09)$ , $\mathcal{N}(3,0.09)$ , $\mathcal{N}(5,0.09)$
$X_{4}$	Uniform in the range $[a,b]$	Each cluster is sampled independently from a uniform distribution in $[a,b]$ where $a=\min(X_{1},X_{2})$ , $b=\max(X_{1},X_{2})$ .
$X_{5}$	Uniform in the range $[\min X_{1}$ , $\max X_{1}]$	Each cluster is sampled independently from a uniform distribution over the $X_{1}$ observed range.
$X_{6}$	Uniform in the range $[\min X_{2}$ , $\max X_{2}]$	Each cluster is sampled independently from a uniform distribution over the $X_{2}$ observed range.

Table 1: Description of the feature generation process of the illustrative dataset.

Figure 2: Comparison of the mean importance of each feature evaluated by our three proposals for different choices of subspace size, over 20 generations of dataset.

Table 2: Clustering performance in terms of Adjusted Rand and Fowlkes-Mallows index of the illustrative dataset. Results are reported as mean

\pm

standar deviation over 20 replications for different values of the subspace size

m

Table 3: Comparison of feature importance measures across the proposals and the literature methods considered. Results are reported as mean

\pm

standard deviation over 50 runs.

Dataset	$K$	$N\cdot p$	$d_{f}$	$d_{n}$	$\mu_{k}$	$\Sigma_{k}$
Overlap	4	(30,30,30,30)	5+5+5	2+2+2	$k\cdot\mathbf{1}_{d_{f}}$	$\begin{pmatrix}(0.2)^{2}\cdot\mathbf{1}_{5\times 5}&\mathbf{0}&\mathbf{0}\\ \mathbf{0}&(0.35)^{2}\cdot\mathbf{1}_{5\times 5}&\mathbf{0}\\ \mathbf{0}&\mathbf{0}&(0.5)^{2}\cdot\mathbf{1}_{5\times 5}\\ \end{pmatrix}$
Proportion	5	(15,20,40,30,15)	4+4	4+4	$k\cdot\mathbf{1}_{d_{f}}$	$\begin{pmatrix}(0.25)^{2}\cdot\mathbf{1}_{4\times 4}&\mathbf{0}&\\ \mathbf{0}&(0.5)^{2}\cdot\mathbf{1}_{4\times 4}\\ \end{pmatrix}$
Correlation	4	(30,30,30,30)	4+4+4	2+2+2	$k\cdot\mathbf{1}_{d_{f}}$	$\begin{pmatrix}(0.25)^{2}\mathbf{R}&\mathbf{0}&\mathbf{0}\\ \mathbf{0}&(0.35)^{2}\mathbf{R}&\mathbf{0}\\ \mathbf{0}&\mathbf{0}&(0.45)^{2}\mathbf{R}\end{pmatrix}$ $\mathbf{R}_{i,j}=\lambda_{i}\lambda_{j}$ , $\mathbf{R}_{i,i}=1$ , $\lambda=(1,0.75,0.25,0.1)$

Table 4: Dataset classes and specifications of the choice of parameters.

Table 5: Clustering performance in terms of Adjusted Rand and Fowlkes-Mallows index of the overlap dataset. Results are shown as mean

\pm

standard deviation over 20 replications for different values of the subspace size

m

Table 6: Feature importance scores for the overlap dataset. Results are expressed as

[\min,\max]

of the average value per feature in each feature group. Features in the same group have same marginal distribution (see Table 4 ).

Table 7: Clustering performance in terms of Adjusted Rand and Fowlkes-Mallows index of the proportion dataset. Results are shown as mean

\pm

standard deviation over 20 replications for different values of the subspace size

m

Table 8: Feature importance scores for the proportion dataset. Results are expressed as

[\min,\max]

of the average value per feature in each feature group. Features in the same group have same marginal distribution (see Table 4 ).

Table 9: Clustering performance in terms of Adjusted Rand and Fowlkes-Mallows index of the correlation dataset. Results are shown as mean

\pm

standard deviation over 20 replications for different values of the subspace size

m

Table 10: Feature importance scores for the correlation dataset. Results are expressed as

[\min,\max]

of the average value of each feature in each feature group. Features in the same group have same marginal distribution (see Table 4 ).

Panel of Figure 3 — Figure 3: Clustering performance of the proposed method across the three resampling schemes on the Wine dataset. Results are computed in terms of Adjusted Rand and Fowlkes-Mallows index, and averaged over 20 runs on the dataset, for different subspace size choices. The horizontal axis represents the subspace size chosen ( $m$ ).

Figure 4: Comparison of the mean importance of each feature evaluated by our three proposals for different choices of subspace size, over 20 runs on the Wine dataset.

Metric	K-means	Hierarchical	Basic	Efron	BBC
Adjusted Rand	$0.90\pm 0$	$0.58\pm 0$	$0.91\pm 0$	$0.91\pm 0.02$	$0.91\pm 0.02$
Fowlkes–Mallows	$0.93\pm 0$	$0.72\pm 0$	$0.94\pm 0$	$0.94\pm 0.01$	$0.94\pm 0.01$

Table 11: Clustering performances are reported as mean

\pm

standard deviation over 50 runs of each method on the Wine dataset. The subspace size is set as

m=12

Table 12: Feature importance scores for the Wine dataset. Results are reported as mean

\pm

standard deviation over 50 runs. Rankings of features for each method are reported.

Panel of Figure 5 — Figure 5: Clustering performance of the proposed method across the three resampling schemes on the Breast dataset. Results are computed in terms of Adjusted Rand and Fowlkes-Mallows index, and averaged over 50 runs on the dataset, for different subspace size choices. The horizontal axis represents the subspace size chosen ( $m$ ).

Figure 6: Comparison of the mean importance of each feature evaluated by the three versions of the proposal for different choices of subspace size $m$ , over 20 runs on the Breast dataset.

Metric	K-means	Hierarchical	Basic	Efron	BBC
Adjusted Rand	$0.84\pm 0$	$0.47\pm 0$	$0.87\pm 0.01$	$0.87\pm 0.01$	$0.86\pm 0.02$
Fowlkes–Mallows	$0.93\pm 0$	$0.79\pm 0$	$0.94\pm 0$	$0.94\pm 0$	$0.94\pm 0.01$

Table 13: Clustering performances are reported as mean

\pm

standard deviation over 50 runs of each method on the Breast dataset. The subspace size of the proposed method is set as

m=8

Table 14: Feature importance scores for the Breast dataset. Results are reported as mean

\pm

standard deviation over 50 runs. Rankings of features for each method are reported.

Clustering

Clustering groups data points by shared structure, developing methods to discover, validate, and interpret groups across diverse data and constraints.

See all 342 papers

Cluster LOCO: Feature Importance For Interpreting Clusters
Jun 12, 2026Claire M. He, Genevera I. AllenClustering
Department of Statistics · Columbia University · New York, NY 10027, USA
Beyond Feature Importance: A Comparative Analysis of Pattern Detection Methods in Cluster Interpretation
Aug 6, 2026Benjamin Connor, Anna Jurek-Loughrey, Lu Bai +1Clustering Unsupervised Clustering
School of EEECS Queen’s University Belfast 18 Malone Road, Belfast, BT9 5AF
Weight-Informed Self-Explaining Clustering for Mixed-Type Tabular Data
Apr 7, 2026Lehao Li, Qiang Huang, Yihao Ang +3Clustering
School of Computing, National University of Singapore, Singapore · School of Intelligence Science and Engineering, Harbin Institute of Technology (Shenzhen), China

Jun 12, 2026J/K move · Enter open · S save

Cluster LOCO: Feature Importance For Interpreting Clusters

Claire M. He, Genevera I. Allen

Department of Statistics · Columbia University · New York, NY 10027, USA

Clustering

Open paper AskPDF arXiv

Clustering is widely used for exploratory analysis and scientific discovery, driving insights from market segmentation to biological data analysis, but its outputs can be difficult to interpret, audit, and reproduce as modern datasets become increasingly large and complex. Reliable use of clustering requires understanding which features drive the discovered structure, yet feature-level explanations for clustering remain scarce compared with methods in supervised learning. Furthermore, existing clustering feature importance scores are often tied to specific algorithms and data assumptions. To address these challenges, we propose Cluster LOCO (Leave-One-Covariate-Out), a family of model-agnostic feature importance scores for clustering. Cluster LOCO is built on feature occlusion and clustering generalizability, defined as whether cluster labels learned on one subset of the data can be accurately predicted on held-out samples. For any chosen clustering algorithm, Cluster LOCO quantifies a feature's importance by measuring how much its removal degrades generalizability. We first introduce Cluster LOCO-Split, which relies on data splitting, and then extend it to Cluster LOCO-MP, a minipatch ensemble-based version designed for large-scale data. Across synthetic simulations and an application to cell-type discovery in single-cell transcriptomics, we show that Cluster LOCO more reliably recovers informative features than existing clustering feature importance methods.

Abstract

Clustering

Explore similar work

Cluster LOCO: Feature Importance For Interpreting Clusters

Beyond Feature Importance: A Comparative Analysis of Pattern Detection Methods in Cluster Interpretation

Weight-Informed Self-Explaining Clustering for Mixed-Type Tabular Data