Overview

It studies how skewed class frequencies, long tails, and rare events bias models toward majority classes, a challenge in medical diagnosis, fraud detection, and computer vision. Common approaches rebalance data through over- or undersampling and synthetic examples, adjust objectives with class weights, focal or logit-adjusted losses, and improve representations with prototypes, contrastive learning, or class-aware sampling. Active learning, calibration, and federated or semi-supervised methods address settings with limited labels, distributed data, or shifting class priors. Results depend strongly on data structure and deployment conditions: reweighting and synthetic data can improve minority recall but may also harm performance or calibration, so evaluations need class-wise and operational metrics, not accuracy alone.

Latest papers

276 papers21 papers in the last four weeks, down 16% on the four weeks before. 0.2% of all new papers.

Jul 18, 2025

Are Kolmogorov-Arnold Networks (KANs) Suitable for Imbalanced Data Classification?

Kolmogorov-Arnold Networks (KANs) parameterize each connection as a learnable univariate B-spline, offering native feature-level interpretability. Whether this locality helps or hinders under class imbalance, where minority sparsity may leave spline edges without enough local observations to update reliably, remains unexamined. We evaluate KANs against capacity-matched Multilayer Perceptrons (MLPs) and XGBoost across ten imbalanced tabular datasets under three conditions: native class distribution, Focal Loss, and SMOTE-Tomek resampling, within a parameter-parity, two-phase framework. Under the native distribution, KAN spline activations collapse in minority-sparse regions, showing stronger majority-class bias than the matched MLP. Focal Loss yields negligible improvement, as it reweights gradients without providing the localized observations needed for discriminative curvature. Conversely, SMOTE-Tomek resampling lets spline activations recover structured minority-class representations, achieving the best cohort-wide rank and a large, statistically significant gain over unmitigated KANs, though not over resampled baselines. KAN's native spline interpretability directly reveals these dynamics via learned activation profiles, proving representational change rather than threshold recalibration without post-hoc attribution. However, these interpretability gains incur substantially higher training overhead than a parameter-matched MLP across all conditions, with no strategy-dependent latency reduction. Thus, KANs are conditionally suitable for imbalanced tabular classification: effective and interpretable with data-level resampling, yet costly relative to simpler alternatives without it.
May 19, 2025

When majority rules, minority loses: bias amplification of gradient descent

Despite growing empirical evidence of bias amplification in machine learning, its theoretical foundations remain poorly understood. We develop a formal framework for majority-minority learning tasks, showing how standard training can favor majority groups and produce stereotypical predictors that neglect minority-specific features. Assuming population and variance imbalance, our analysis reveals three key findings: (i) the close proximity between ``full-data'' and stereotypical predictors, (ii) the dominance of a region where training the entire model tends to merely learn the majority traits, and (iii) a lower bound on the additional training required. Our results are illustrated through experiments in deep learning for tabular and image classification tasks.
Sep 19, 2024

iCost: A Novel Instance-Complexity-Based Cost-Sensitive Learning Framework

Class imbalance poses a significant challenge in classification tasks, often causing standard learning algorithms to become biased toward the majority class. Cost-sensitive learning (CSL) addresses this issue by assigning higher penalties to minority-class misclassifications. However, conventional CSL typically applies a uniform penalty to all minority-class instances, ignoring the fact that minority samples may differ substantially in terms of local safety, overlap, boundary ambiguity, and outlier-like behavior. Uniform penalization can therefore introduce undue bias, increasing the number of misclassifications. In this study, we propose iCost, an instance-complexity-aware CSL framework that assigns adaptive penalties to minority-class samples according to their estimated learning difficulty. This fine-grained penalization strategy ensures fairer weighting, reduces unwarranted bias, and improves overall classification performance. Two complementary complexity estimation strategies are introduced: Neighbor-iCost, based on local neighborhood composition, and Gini-iCost, based on Gini-impurity-based feature-space partitioning. Extensive experiments on 65 binary and 10 multiclass imbalanced datasets show that iCost outperforms conventional CSL by a clear margin and remains highly competitive with widely used resampling methods. To support reproducibility and practical adoption, the proposed algorithm has been released as a scikit-learn-compatible Python package through PyPI. This work offers a fresh perspective on imbalanced learning by integrating instance-level data complexity into the learning process, opening new avenues for developing adaptive, complexity-aware strategies for imbalanced classification.
Aug 19, 2024

AIR: Analytic Imbalance Rectifier for Continual Learning

Continual learning (CL) agents incrementally learn from sequentially arriving data and adapt to the dynamic, ever-changing nature of real-world environments. However, many existing CL methods suffer performance degradation in evolving, imbalanced data streams due to limited adaptation to changing class frequencies or ineffective use of mixed data from new and previously observed classes. To deal with these challenges, we propose an analytic imbalance rectifier (AIR) algorithm for real-world CL. AIR is an online exemplar-free approach with a frozen backbone as the feature extractor and a closed-form incremental classifier whose weight equals the joint-learning weight for the same class-weighted ridge objective. AIR addresses class imbalance with an analytic reweighting module (ARM) that calculates a reweighting factor for each class in the loss function to equalize total sample weights across classes. Under long-tailed class-incremental learning, AIR leads 28 baselines in aggregate accuracy and exemplar-free methods in aggregate macro F1, gaining 3.21% accuracy and 2.14% macro F1 over the respective strongest exemplar-free baselines. Under the Si-Blurry setting with recurring classes, AIR leads 15 exemplar-based and exemplar-free baselines, gaining 2.32% aggregate accuracy and 1.27% aggregate macro F1 over the strongest baseline. One-sided paired tests support positive mean absolute gains in these four comparisons (Holm-adjusted p<0.006).
Apr 11, 2024

Robust performance metrics for imbalanced classification problems

We show that established performance metrics in binary classification, such as Matthews' correlation coefficient (MCC), Cohen's κκ, the F-score or the Jaccard similarity coefficient are not robust to class imbalance in the sense that if the proportion of the minority class tends to 00, the true positive rate (TPR) of the Bayes classifier under these metrics {becomes very small or even tends to 00}. Thus, in imbalanced classification problems, these metrics favour classifiers which ignore the minority class. To alleviate this issue we introduce robustified modifications of the MCC, of Cohen's κκ and of the F-score with an additional tuning parameter which allows to adapt the amount of robustness against class imbalance. As theoretical guarantee we show that the Bayes-optimal classifier for these robustified performance metrics, when expressed in terms of the density ratio f1/f0f_1/f_0 of the class-conditional densities fif_i, has a threshold parameter which is upper-bounded in terms of the tuning parameters. Therefore, even in strongly imbalanced settings, the TPR associated to this classifier will be bounded away from 00{, provided that f1/f0f_1/f_0 exceeds this bound with positive probability under the positive class}. We numerically illustrate the behaviour of the various performance metrics and the effect of the tuning parameters in simulations as well as on a credit default data set. We also discuss connections to the receiver operating characteristic and precision-recall curves, which provide an alternative perspective on the proposed notion of robustness, and give recommendations on how to combine their usage with performance metrics.
Aug 26, 2020

Multi-Label Sentiment Analysis on 100 Languages with Dynamic Weighting for Label Imbalance

We investigate cross-lingual sentiment analysis, which has attracted significant attention due to its applications in various areas including market research, politics and social sciences. In particular, we introduce a sentiment analysis framework in multi-label setting as it obeys Plutchik's wheel of emotions. We introduce a novel dynamic weighting method that balances the contribution from each class during training, unlike previous static weighting methods that assign non-changing weights based on their class frequency. Moreover, we adapt the focal loss that favors harder instances from single-label object recognition literature to our multi-label setting. Furthermore, we derive a method to choose optimal class-specific thresholds that maximize the macro-f1 score in linear time complexity. Through an extensive set of experiments, we show that our method obtains the state-of-the-art performance in 7 of 9 metrics in 3 different languages using a single model compared to the common baselines and the best-performing methods in the SemEval competition. We publicly share our code for our model, which can perform sentiment analysis in 100 languages, to facilitate further research.