cs.AINov 5, 2026

The Matthews Correlation Coefficient: Fifty years

Authors: Jesus S. Aguilar-Ruiz

Organizations: School of Engineering, Pablo de Olavide University, Seville, 41013, Spain

Abstract

Fifty years after its introduction, the Matthews Correlation Coefficient (MCC) remains central -and debated- in binary classification. This survey offers a comprehensive retrospective and an in-depth analysis of MCC, covering its historical development, mathematical foundations, and implications for practice. We clarify long-standing misconceptions, formalize its equivalences (to Pearson's correlation and to Chi2 in 2x2 tables), and examine its behavior under class imbalance. Beyond MCC, we develop analytical (derivatives) and graphical (contour plots) frameworks that generalize to any confusion-matrix-based metric. These frameworks expose structural properties -bias, robustness, sensitivity- that scalar summaries conceal, and they provide actionable guidance for selecting and interpreting performance measures in applications where reliability and fairness matter. Together, our results position MCC within a broader toolkit while clarifying when it is informative -and when alternatives may be preferable.

Figures & tables

Explore similar work

Apr 11, 2024

Robust performance metrics for imbalanced classification problems

We show that established performance metrics in binary classification, such as Matthews' correlation coefficient (MCC), Cohen's κκ, the F-score or the Jaccard similarity coefficient are not robust to class imbalance in the sense that if the proportion of the minority class tends to 00, the true positive rate (TPR) of the Bayes classifier under these metrics {becomes very small or even tends to 00}. Thus, in imbalanced classification problems, these metrics favour classifiers which ignore the minority class. To alleviate this issue we introduce robustified modifications of the MCC, of Cohen's κκ and of the F-score with an additional tuning parameter which allows to adapt the amount of robustness against class imbalance. As theoretical guarantee we show that the Bayes-optimal classifier for these robustified performance metrics, when expressed in terms of the density ratio f1/f0f_1/f_0 of the class-conditional densities fif_i, has a threshold parameter which is upper-bounded in terms of the tuning parameters. Therefore, even in strongly imbalanced settings, the TPR associated to this classifier will be bounded away from 00{, provided that f1/f0f_1/f_0 exceeds this bound with positive probability under the positive class}. We numerically illustrate the behaviour of the various performance metrics and the effect of the tuning parameters in simulations as well as on a credit default data set. We also discuss connections to the receiver operating characteristic and precision-recall curves, which provide an alternative perspective on the proposed notion of robustness, and give recommendations on how to combine their usage with performance metrics.
Aug 10, 2026

SoftMCC: An MCC-Brier Calibration Bridge for Threshold-Free Model Selection under Class Imbalance

Model selection for imbalanced binary classification often uses the Matthews correlation coefficient (MCC), but its thresholding step makes validation rankings depend on the chosen cutoff. SoftMCC is a post-training MCC validation framework built on established probability-valued confusion counts, coupling an MCC-specific calibrated identity with a tie-aware, shared-pool selection protocol. Its core score is a covariance-normalized probability-label association, reduces exactly to MCC for hard predictions, and is Pearson-bounded. Under perfect population calibration it equals the Brier skill score with identical candidate ordering, which is the calibrated case of Murphy's partition of the Brier score. Outside that regime the gap does not identify calibration error. Across 18 settings with 12 duplicate-safe grouped repeats, SoftMCC attains the best stability mean rank (2.31) and highest mean tie-corrected Kendall's W (0.659), with a significant Friedman test (p=0.004). Nemenyi analysis separates it from area under the precision-recall curve (AUPRC) and [email protected], and 14-source-family sensitivity retains both separations. For selected-model utility, none of six prespecified paired comparisons is significant before or after Holm correction (smallest raw p=0.093), and the dataset-level test is not significant (p=0.746). Label permutation lowers mean W to 0.092. Temperature scaling shifts SoftMCC rankings (mean Spearman 0.851) whereas rank-based and threshold-optimized metrics remain invariant. SoftMCC is a calibration-sensitive MCC-family selector with bounded stability evidence and inconclusive utility differences.
Nov 5, 2026

Performance metrics in machine learning: a scenario-based guide from the confusion matrix to calibration, regression and uncertainty

Performance metrics can yield different conclusions from the same predictions. This methodological tutorial explains the effects of class prevalence, decision thresholds, aggregation rules and software conventions through controlled scenarios. Twenty-two confusion-matrix metrics are organized in a prevalence-coded map. Fifteen scenarios cover binary and multi-class classification, score discrimination, probability calibration, regression, uncertainty and consistency checks. Inputs are fixed in released files, and one factor is varied in each scenario. All 205 verification checks passed; 202 were independent and 3 were internal. An always-negative rule achieved an accuracy of 0.990 at a prevalence of 0.010, with balanced accuracy of 0.500. At fixed sensitivity and specificity, precision fell from 0.889 to 0.008 as prevalence fell from 0.5 to 0.001. Three probability variants shared a ROC-AUC of 0.825 but had calibration slopes of 2.08, 1.04 and 0.52. With 10 participants, participant-majority accuracy of 0.700 had a 95% interval of 0.348-0.933. Fold-averaged scores were falsely flagged as inconsistent under an inappropriate pooled-test-set model. A goal-based metric table and a reporting checklist are provided. The scenarios illustrate mechanisms and do not estimate their frequency in published studies. A literature audit and a real-data demonstration are specified as future protocols and were not executed.