What Changed? Drift Detection with Real, Virtual, and Incomparable Diagnosis
Organizations: Center for Management of Information Technologies, Kagoshima University
Abstract
Sharing a deep encoder does not, by itself, fix the central confound of task-comparison scores. We show that cross-evaluated heads on a frozen shared representation inherit the extrapolation confound of shallow exchange scores: pure input rotations with fixed labels inflate a deep exchange score from about 0 to 0.80, while representation-novelty scores are blind in the complementary direction (flat under label permutations that change the task completely). Transplanting a conditional two-discriminator discrepancy into the embedding space resolves both blind spots: the functional axis stays within +-0.001 under rotations and tracks label-permutation drift mass monotonically. Built into a mixture-of-heads lifecycle, the two-axis gate attains better decision quality with fewer heads than exchange or novelty triggers at a matched training budget. On generalized category discovery, the same chunk-level functional axis separates semantic novelty from photometric shift with AUROC 0.98-0.99 where per-input OOD scores (MSP, Energy, Mahalanobis, KNN) sit near chance for that distinction. All findings replicate across frozen ImageNet-21k ViT-B/16 and self-supervised DINOv2 backbones on CIFAR-100, and extend to residual adapter pools with recurrence, where a null-calibrated novelty trigger never fires on mechanism changes while the two-axis gate handles them with full recurrence reuse. We state explicitly the common-factoring condition under which embedding-space conclusions transfer to the original mechanism.
Figures & tables
| Detector | False alarms | Misses | Type acc. | Delay (detected) |
|---|---|---|---|---|
| ADWIN | 0.18 | 0.60 (SEA) | 0.5 | 56–5320 |
| DDM | 0.65 | 0.05 | 0.5 | 21–660 |
| EDDM | 1.00 | — | — | — |
| Page–Hinkley | 0.13 | 0.05 | 0.5 | 90–690 |
| D3 (input only) | 0.00 | 1.00 (concept) | 0.5 | 1000 |
| WATCH (reimpl.) | 0.20 | 0.60 (SEA) | 1.0 | 34–5506 |
Concept Drift
Concept drift is change over time in the relationship between data and target outcomes, which can erode the accuracy of deployed models.
See all 90 papers