Beyond Imbalance Ratio: Data Characteristics as Critical Moderators of Oversampling Method Selection

Yuwen Jiang; Songyun Ye

cs.LGApr 6, 2026

Beyond Imbalance Ratio: Data Characteristics as Critical Moderators of Oversampling Method Selection

Authors: Yuwen Jiang, Songyun Ye

Organizations: School of Artificial Intelligence, Guangzhou Institute of Science and Technology, Guangzhou, China

Abstract

The prevailing IR-threshold paradigm posits a positive correlation between imbalance ratio (IR) and oversampling effectiveness, yet this assumption remains empirically unsubstantiated through controlled experimentation. We conducted 12 controlled experiments (N > 100 dataset variants) that systematically manipulated IR while holding data characteristics (class separability, cluster structure) constant via algorithmic generation of Gaussian mixture datasets. Two additional validation experiments examined ceiling effects and metric-dependence. All methods were evaluated on 17 real-world datasets from OpenML. Upon controlling for confounding variables, IR exhibited a weak to moderate negative correlation with oversampling benefits. Class separability emerged as a substantially stronger moderator, accounting for significantly more variance in method effectiveness than IR alone. We propose a 'Context Matters' framework that integrates IR, class separability, and cluster structure to provide evidence-based selection criteria for practitioners.

Figures & tables

Panel of Figure 1 — Figure 1: Theoretical framework: Data characteristics as moderators of the IR-oversampling effectiveness relationship. Solid arrows indicate main effects; dashed arrows indicate moderation effects.

Dataset	Domain	IR	$n$	$d$	Minority	Type
sonar	Signal	1.14	208	60	97	Low IR
wdbc	Medical	1.68	569	30	212	Low IR
ionosphere	Radar	1.79	351	34	126	Low IR
breast-wisc	Medical	1.90	699	9	241	Low IR
iris	Biology	2.00	150	4	50	Low IR
credit-g	Finance	2.33	1000	7	300	Low IR

Table 1: Detailed Statistics of 17 Real Datasets Used in Experiments

Classifier	Strengths	Weaknesses	Type
Logistic Regression	Interpretable, fast, probabilistic output	Linear decision boundary, limited complexity	Linear
Decision Tree	Interpretable, handles non-linearity	Prone to overfitting, unstable	Tree-based
Random Forest	Robust, reduces overfitting, feature importance	Less interpretable, computationally expensive	Ensemble
Gradient Boosting	High accuracy, handles complex patterns	Sensitive to noise, requires tuning	Ensemble
SVM	Effective in high dimensions, memory efficient	Slow on large datasets, kernel selection	Kernel-based

Table 2: Comparison of Classifiers Used in Experiments

Dataset	Natural IR	Correlation $r$
wdbc	1.68	$-0.45$
glass	22.78	$-0.552$
churn	6.07	$-0.42$
pendigits	9.42	$-0.39$
letter	26.25	$-0.944$

Table 3: Within-Dataset Correlations: IR vs. Method Advantage

Panel of Figure 2 — Figure 2: Algorithmic workflow of the 12 controlled experiments. The three-stage design progresses from observational replication (Stage 1) through controlled manipulation (Stage 2) to moderation testing (Stage 3). B1: Within-dataset IR manipulation; B2: Generated data with controlled parameters; C1-C3: Moderation experiments for separability, cluster structure, and sample size.

Exp.	Name	Description	$r$	$p$
Stage 1: Observational Baseline
1	Cross-dataset analysis	17 real datasets from OpenML	$+0.9662$	$<0.001$
Stage 2: Controlled Manipulation
2–6	B1: Within-dataset (5 datasets)	glass, wdbc, churn, pendigits, letter	$-0.39$ to $-0.944$	$<0.05$
7	B2: Generated Gaussian mixtures	12 synthetic configurations	$-0.765$	$<0.001$
Stage 3: Moderation Testing

Table 4: Summary of 12 Controlled Experiments

Figure 3: Separability moderates oversampling effectiveness. Low separability data benefits more from oversampling.

Model Configuration	Mean AUC	$\Delta$ from Full
Full model (IR + Separability + Cluster + Sample Size)	0.892	—
w/o Sample Size control	0.887	$-0.005$
w/o Cluster Structure control	0.863	$-0.029$
w/o Separability control	0.834	$-0.058$
w/o IR control (baseline)	0.791	$-0.101$
IR only (observational)	0.756	$-0.136$

Table 5: Ablation Study: Contribution of Data Characteristics to Model Performance

Metric	Absolute r	Relative r	Interpretation
AUC-ROC	$-0.14$	$-0.41$	Weak-moderate negative
AUC-PR	$-0.08$	$-0.15$	Weak negative (ns)
F1-Score	$-0.53$	$\mathbf{-0.75}^{***}$	Strong negative
G-Mean	$-0.40$	$\mathbf{-0.73}^{***}$	Strong negative

Table 6: Ceiling Effect Control Analysis: Correlation between IR and Improvement

Metric	r	p-value	Effect Size
Recall	$\mathbf{-0.86}^{***}$	$<0.001$	Large
Specificity	$\mathbf{-0.89}^{***}$	$<0.001$	Large
Balanced-Accuracy	$\mathbf{-0.72}^{***}$	$<0.001$	Large
F1-Score	$\mathbf{-0.68}^{***}$	$<0.001$	Medium-Large
G-Mean	$\mathbf{-0.67}^{***}$	$<0.001$	Medium-Large
AUC-ROC	$-0.40^{***}$	$<0.001$	Medium

Table 7: Multi-Metric Validation: IR Correlation with Relative Improvement

Figure 4: Validation experiments demonstrating metric-dependence and ceiling effects in the IR-oversampling effectiveness relationship. Top row: Ceiling effect control showing absolute vs. relative improvement correlations. Bottom row: Multi-metric validation displaying effect sizes across eight evaluation metrics. Error bars represent 95% confidence intervals.

Method	sonar	wdbc	churn	glass	letter	mamm.	ecoli
Baseline	0.823	0.956	0.742	0.891	0.934	0.812	0.756
ROS	0.856	0.961	0.778	0.912	0.945	0.834	0.789
SMOTE	0.871	0.965	0.801	0.923	0.952	0.856	0.812
BL-SMOTE	0.868	0.963	0.795	0.931	0.948	0.871	0.805
ADASYN	0.865	0.964	0.789	0.918	0.950	0.849	0.798
Tomek	0.819	0.957	0.745	0.893	0.935	0.815	0.758

Table 8: Comprehensive Method Comparison (AUC-ROC)

Figure 5: Comparison of SMOTE and BorderlineSMOTE: SMOTE generates samples through linear interpolation (creating “bridge” structures), while BorderlineSMOTE focuses only on boundary regions.

Class-Imbalanced Learning

Class-imbalanced learning develops classifiers that recognize rare classes without sacrificing performance on common ones.

See all 256 papers

RUBRIC: Realism--Utility Balanced Ranking for Imbalanced Classification
Jul 10, 2026Yanxuan Yu, Dong Liu, Shu Wang +7Class-Imbalanced Learning Synthetic Data Augmentation
Columbia University · University of California, Los Angeles · University of Illinois Urbana-Champaign +3
K-IPO: Kendall-constrained Importance Preserving Oversampling for Imbalanced Tabular Data
Jul 17, 2026Marios Tyrovolas, Argiris Sofotasios, Dimitris Metaxakis +4Class-Imbalanced Learning Synthetic Data Augmentation
Department of Informatics and Telecommunications, University of Ioannina, Arta, Greece · Industrial Systems Institute, Athena Research Center, Patras, Greece · Archimedes Unit, Athena Research Center, Athens, Greece +3
QC-SMOTE: Quality-Controlled SMOTE for Imbalanced Classification
Jun 23, 2026Parth Upman, Shreyank N GowdaClass-Imbalanced Learning Synthetic Data Augmentation
School of Computer Science, University of Nottingham, Jubilee Campus, Wollaton Road, Nottingham, NG8 1BB, United Kingdom

Jul 10, 2026J/K move · Enter open · S save

RUBRIC: Realism--Utility Balanced Ranking for Imbalanced Classification

Yanxuan Yu, Dong Liu, Shu Wang, Wenxiao Zhao+6

Columbia University · University of California, Los Angeles · University of Illinois Urbana-Champaign+3

Class-Imbalanced Learning Synthetic Data Augmentation Synthetic Data Selection

Open paper AskPDF arXiv

Class imbalance poses a fundamental challenge in risk-sensitive applications such as fraud detection and medical diagnosis, where minority-class samples are scarce yet critical for accurate classification. Existing oversampling methods generate synthetic samples to rebalance class distributions; however, they often produce large numbers of low-quality candidates that distort decision boundaries or introduce artifacts, leading to overfitting and degraded generalization. In this work, we introduce RUBRIC, a generator-agnostic filtering framework that formulates synthetic sample selection as a quality-over-quantity optimization problem. RUBRIC ranks candidates using a realism-utility trade-off: realism is estimated via a neural density-ratio discriminator from each candidate's resemblance to real minority samples, while utility captures proximity to the decision boundary through a concave, margin-based scoring function

g_-(t)=-\log(1+e^{-t/τ})

. The discriminator uses the same architecture and training protocol on every benchmark and is fit independently to that dataset's real minority class versus its synthetic pool. We show that, under mild regularity conditions, the proposed filtering framework monotonically tightens the generalization bound for margin-based classifiers by jointly reducing distribution shift and suppressing near-negative tail contributions. Through extensive experiments on standard public imbalanced-classification benchmarks, we demonstrate that RUBRIC boosts minority-class recall while preserving overall discriminative ability across multiple data generators. Sensitivity analyses in

λ

and the selection budget

K

further characterize performance trade-offs oriented toward ranking quality.

Abstract

Class-Imbalanced Learning

Explore similar work

RUBRIC: Realism--Utility Balanced Ranking for Imbalanced Classification

K-IPO: Kendall-constrained Importance Preserving Oversampling for Imbalanced Tabular Data

QC-SMOTE: Quality-Controlled SMOTE for Imbalanced Classification