When Does Oversampling Help? A Decision Framework for Imbalanced Data Streams Under Label Delay and Concept Drift
Organizations: Faculty of Engineering & IT, The British University in Dubai, Dubai, UAE. · Faculty of Computers and Artificial Intelligence, Cairo University, Giza, Egypt.
Abstract
For imbalanced data streams with delayed labels, is it worth adding a streaming oversampler, or does continuous re-evaluation of earlier predictions already recover the cost of the delay? We address this through an extensive experimental comparison of streaming oversampling and continuous re-evaluation under joint class imbalance, concept drift, and label delay. Streaming oversamplers are typically evaluated under the test-then-train assumption of immediate labels; their behaviour under these conditions, assessed through re-evaluation, has not been systematically studied. We close this gap with a decision framework that predicts when oversampling adds value over re-evaluation, organised around three axes: drift density, minority-class learnability, and delay relative to the refinement window, and operationalised as a competition between a recovery channel (re-evaluation) and a minority-coverage channel (oversampling). Evaluating classifiers on synthetic and real streams, we find: re-evaluation recovers delay-induced loss on drift-active streams but is silent on stationary ones; oversampling's marginal contribution over re-evaluation is statistically detectable in only one real-data configuration; and cost--performance Pareto recommendations from synthetic streams do not transfer to real data, where fronts partition by drift regime. Method selection should therefore be conditioned on the stream's profile and re-estimated as it evolves, not inherited from synthetic benchmarks.
Figures & tables
| Learner | Overhead | G-mean | Pareto |
| (Pareto-optimal: 7/15) | |||
| ARF | 0.0 | 0.656 | Y |
| HAT | 0.0 | 0.656 | Y |
| CSMOTE | 99798.0 | 0.696 | Y |
| VFCSMOTE_KNN | 128797.3 | 0.777 | Y |
| VFCSMOTE_SWT | 130031.0 | 0.785 | Y |
| Oversampler | Bare learner | Excluded cells ( ) | mean DiD | |||
| VFCSMOTE | ARF | 16 | PAKDD@{0.1, 0.5} | 8/8/0 | 0.918 | |
| CSMOTE | ARF | 15 | PAKDD@{0.1, 0.5, 1.0} | 8/7/0 | 0.629 | |
| VFCSMOTE | HAT | 17 | PAKDD@{0.5} | 4/13/0 | 0.207 | |
| VFCSMOTE | NV | 15 | PAKDD@{0.1, 0.5, 1.0} | 15/0/0 | ||
| VFCSMOTE | KNN | 14 | Elec@{0.5}, PAKDD@{0.1, 0.5, 1.0} | 4/10/0 | 0.135 | |
| VFCSMOTE | SWT | 16 | PAKDD@{0.1, 1.0} | 9/7/0 | 0.214 |
| Learner | Overhead | G-mean | Pareto |
| Covertype (all ; Pareto-optimal: 2/15) | |||
| SWT | 0 | 0.893 | Y |
| OOB | 5,884,187 | 0.916 | Y |
| ARF | 0 | 0.890 | N |
| CreditCard (all ; 2/15) | |||
| NV | 0 | 0.897 | Y |
Class-Imbalanced Learning
Class-imbalanced learning develops classifiers that recognize rare classes without sacrificing performance on common ones.
See all 274 papers