How to evaluate clustering with ground truth?

Pasi Fränti

cs.AIJun 25, 2026

How to evaluate clustering with ground truth?

Authors: Pasi Fränti

Organizations: School of Computing University of Eastern Finland Joensuu, Finland

Abstract

External indexes can be used for cluster evaluation when ground truth is available. We review the most common external validity indexes focusing on set-matching-based measures. We recommend centroid index (CI), because it is an intuitive cluster-level measure with an explainable result. If we need a more fine-tuned, point-level measure, there are more choices. Pair-set index (PSI) provides a normalized score which is not biased by cluster sizes. If all points should matter equally, then clustering accuracy (ACC) or any other set-matching measure is suitable.

Explore similar work

External Clustering Validation by the Homogeneity-Parsimony Trade-off
Jul 22, 2026Andreas Tiffeau-MayerImage Clustering Cross Validation
Associativity-Peakiness Metric for Contingency Tables
Apr 24, 2026Naomi E. Zirkind, William J. DiehlImage Clustering Performance Metric
Central Description Length (CDL) Clustering Validation Index
Jun 2, 2026Mahdi Shamsi, Soosan BeheshtiImage Clustering Cluster Structure

Jul 22, 2026 · cs.LGJ/K move · Enter open · S save

External Clustering Validation by the Homogeneity-Parsimony Trade-off

Andreas Tiffeau-Mayer

Division of Infection and Immunity & Institute for the Physics of Living Systems, University College London

Scalar metrics are often used to evaluate clusterings against known classes, but they can obscure a fundamental trade-off: clusterings should be informative about class labels while avoiding unnecessary fragmentation. Here we describe normalized scores of cluster homogeneity and parsimony that quantify this trade-off. These scores build on the information bottleneck principle, modified to not reward lossy compression. We show by example and mathematical proof that our definitions of these scores have the intuitive property of varying monotonically under cluster refinement in contrast to related proposals. Extending the information-theoretic framework beyond Shannon entropies, we furthermore derive set-matching and pair-based counterparts of the homogeneity and parsimony scores. These unify commonly used evaluation criteria and show that, in the pair-based setting, the homogeneity-parsimony trade-off recovers the receiver operating characteristic of binary classifiers. We demonstrate the framework's utility for feature selection and algorithm comparison, illustrating how considering scores jointly can clarify clustering operating points and identify Pareto-optimal solutions.

Image Clustering Cross Validation Performance Metric Binary Classification Trade-Off Information Bottleneck

Open paper Ask about it PDF

Abstract

Explore similar work

External Clustering Validation by the Homogeneity-Parsimony Trade-off

Associativity-Peakiness Metric for Contingency Tables

Central Description Length (CDL) Clustering Validation Index