cs.LGAug 11, 2025

Shapley-Inspired Feature Weighting in k-means with No Additional Hyperparameters

Authors: Richard J. FawleyRenato Cordeiro de Amorim

Abstract

Clustering algorithms often assume all features contribute equally to the data structure, an assumption that usually fails in high-dimensional or noisy settings. Feature weighting methods can address this, but most require additional parameter tuning. We propose SHARK (Shapley Reweighted kk-means), a feature-weighted clustering algorithm motivated by the use of Shapley values from cooperative game theory to quantify feature relevance, which requires no additional parameters beyond those in kk-means. We prove that the kk-means objective can be decomposed into a sum of per-feature Shapley values, providing an axiomatic foundation for unsupervised feature relevance and reducing Shapley computation from exponential to polynomial time. SHARK iteratively re-weights features by the inverse of their Shapley contribution, emphasising informative dimensions and down-weighting irrelevant ones, and is equivalent to replacing the arithmetic mean of feature dispersions with their harmonic mean. Experiments on synthetic and real-world data sets show that SHARK consistently matches or outperforms existing methods, achieving superior robustness and accuracy, particularly in scenarios where noise may be present. Software: https://github.com/rickfawley/SHARK.

Explore similar work

Jun 6, 2025cs.LG

Scalable unsupervised feature selection via weight stability

Unsupervised feature selection is critical for improving clustering performance in high-dimensional data, where irrelevant features can obscure meaningful structure. In this work, we propose the Minkowski weighted kk-means++, a novel initialisation strategy for the Minkowski Weighted kk-means. Our initialisation selects centroids probabilistically using feature relevance estimates derived from the data itself. Building on this, we propose two new feature selection algorithms, FS-MWK++, which aggregates feature weights across a range of Minkowski exponents identifying stable and informative features, and SFS-MWK++, a scalable variant based on subsampling. We support our approach with a theoretical analysis, demonstrating that, under explicit assumptions on noise features and cluster structure, relevant features are assigned consistently higher weights than noise features across a range of Minkowski exponents. Our software can be found at https://github.com/xzhang4-ops1/FSMWK.
Xudong Zhang, Renato Cordeiro de Amorim
Jul 16, 2026cs.LG

Counterfactuals for Feature-Weighted Clustering

Counterfactual explanations provide local, interpretable insight by identifying changes to an input that would alter its assigned outcome. Although well established in supervised learning, their extension to clustering is less direct, since cluster assignments are unlabeled and governed by the geometry of the partition. This paper introduces VoICE, a Voronoi-Induced Counterfactual Explainability framework for feature-weighted kk-means clustering. Rather than treating cluster change as a crossing of a single pairwise centroid boundary, VoICE formulates counterfactual generation as projection onto the full weighted Voronoi region of a target cluster, incorporating feature weights directly into both the clustering geometry and the counterfactual objective to yield least-cost and parsimonious explanations under actionability constraints. Target regions are further intersected with data-derived bounds and homothetically contracted towards their centroids, limiting extrapolation and boundary sensitivity. VoICE consistently produces valid target-cluster membership, across several benchmark datasets, where the leading pairwise baseline does not.
Richard J. Fawley, Renato Cordeiro de Amorim
Sep 7, 2026cs.CG

A Sub-4 Approximation for Fair k-Means

Fairness in clustering has attracted sustained research interest, motivated by the need to ensure equitable representation of protected groups in machine learning applications. We study fair kk-means clustering in Euclidean space, where the proportion of each protected group in every cluster must lie within specified lower and upper bounds. These constraints make it challenging to determine both cluster centers and point assignments. We propose an approximation algorithm that combines a linear programming relaxation with geometric transformations of the input to construct candidate center sets. Given a ρρ-approximate algorithm for weighted kk-means and any ε>0ε>0, our algorithm returns a fractional solution whose cost is at most 1+(31/Γ)ρ+O(ε)1+(3-1/Γ)ρ+O(ε) times the optimal integral fair cost, where Γ6.357Γ\approx6.357 is an upper bound on the integrality gap of the standard Euclidean kk-means LP. With a PTAS as the subroutine, the approximation ratio becomes 3.8427+O(ε)3.8427+O(ε), improving the previous factor of 5+O(ε)5+O(ε) to below 44. The solution satisfies all fairness constraints exactly and can be rounded to an integral assignment with a bounded additive violation of fairness and no increase in cost. The same approximation guarantee extends to the kk-sparse Wasserstein barycenter problem.
Kangke Cheng, Guanlin Mo, Shihong Song +1