cs.LGFeb 23, 2026
SaveDrift Localization using Conformal Predictions
Organizations: Bielefeld University—Faculty of Technology Inspiration 1, 33619 Bielefeld—Germany
Abstract
Concept drift -- the change of the distribution over time -- poses significant challenges for learning systems and is of central interest for monitoring. Understanding drift is thus paramount, and drift localization -- determining which samples are affected by the drift -- is essential. While several approaches exist, most rely on local testing schemes, which tend to fail in high-dimensional, low-signal settings. In this work, we consider a fundamentally different approach based on conformal predictions. We discuss and show the shortcomings of common approaches and demonstrate the performance of our approach on state-of-the-art image datasets.
Figures & tables
Figure 1: Effect of training-calibration/test-split on performance — median and -quantile. Decision tree, Fish-head dataset, 500 samples, 98 drifting. Plots differ in scaling of -axis only (test/calibration set size reduces as size of bootstrap increases)
Figure 2: Effect of number of bootstraps on ROC-AUC. Figure shows aggregation of 500 runs on Fish-Head dataset.
Figure 3: Experimental results. ROC-AUC (500 runs) for various drift localizes using windows of 250/60 samples with 98/10 drifting samples (in total).