When a deployed model degrades under concept drift, an operator can retrain it, raise an alert and monitor, or roll back. Yet most drift-adaptation methods consider only retraining. We present RAR (Retrain-Alert-Rollback), a state machine that gives each response an explicit trigger and cost. Retraining is gated on drift detection, label availability, and measured degradation. Every retrained model must pass a two-stage canary before promotion, and corruption-triggered and quality-triggered rollbacks enter different hold states. Under a prospectively specified protocol we compare RAR with six baselines, including a cost-aware retraining policy, on 13 streams: eight real public streams and five semi-synthetic streams derived from them. RAR does not dominate. At low retraining cost, retrain-heavy baselines lead, and no policy's rank differs significantly from RAR's. At higher cost, RAR beats detector-triggered retraining, periodic retraining, and no response simultaneously on four streams, each within a bounded cost window. Its advantage over detector-triggered retraining is consistent with a latency-feasibility condition stated in advance. Against the cost-aware policy, results are heterogeneous. RAR retrains before true drift onsets one eighth to one third as often as detector-triggered retraining, but at lower net utility on most semi-synthetic streams. On H2's designated corruption stream (a registered follow-up), rollbacks fire yet RAR loses to retraining baselines. At ρ=1, removing the canary, and thus rollback, raises utility on 11 of 13 streams. On a post-hoc follow-up stream, cause-specific hold release resumes adaptation where a single cooldown does not. We provide the harness, configurations, and per-cell results for reproduction.
Figures & tables
Figure 1: The RAR state machine. Retraining is admissible only from Serving when all three gates pass; every retrained model traverses the two-stage canary; rollbacks are typed by cause (R1 canary failure, R2 data corruption, R3 sanity failure) and may fire from either canary stage, entering a hard hold (corruption-triggered; thick border; no override) or a soft hold (quality-triggered; dashed border) with the severity-gated emergency override of Section 4.4 .
Stream
No-Resp.
Always-Retr.
Binary
Alert-Only
Rollback-Only
Full RAR
ELEC2
32,430
33,157
32,538
32,428
32,292
32,623 ± 1,233
Covertype
237,095
391,981
401,208
237,072
382,613
372,416 ± 4,777
Poker
513,697
513,318
513,697
513,697
513,697
513,697 ± 0
Airlines
289,503
324,869
330,968
289,483
321,438
302,233 ± 4,305
NOAA
12,454
12,585
12,446
12,453
12,505
12,452 ± 0
INS-abrupt
171,634
194,122
187,024
171,621
174,641
189,793 ± 4,902
Table 1: Mean net utility at the ratified ρ=1 over 30 seeds; best per stream in bold; Full RAR’s seed standard deviation in small type. Rollback-Only also varies by seed through its canary routing (its dispersion is reported in Table 16 ); the four canary-free policies are deterministic given the stream.
Stream
vs. Always-Retr.
vs. Binary
vs. Rollback-Only
vs. No-Resp.
ΔU(1)
ΔU(5)
ΔU(1)
ΔU(5)
ΔU(1)
ΔU(5)
ΔU(1)
ΔU(5)
ELEC2
− 534 ∗
− 492 ∗
+85
+171
+331
+349
+193
+171
Covertype
− 19,565 ∗
− 19,023 ∗
− 28,792 ∗
− 27,990 ∗
− 10,197 ∗
− 10,033 ∗
+135,321 ∗
+135,003 ∗
Poker
+379 ∗
+1,894 ∗
− 0.1 ∗
− 0.5 ∗
− 0.1 ∗
− 0.5 ∗
− 0.1 ∗
− 0.5 ∗
Airlines
− 22,636 ∗
− 21,946 ∗
− 28,735 ∗
− 28,673 ∗
− 19,205 ∗
− 19,186 ∗
+12,730 ∗
+12,624 ∗
NOAA
− 133 ∗
− 118 ∗
+6 ∗
+24 ∗
− 53 ∗
− 49 ∗
− 2 ∗
− 12 ∗
Table 2: Full RAR minus baseline, paired by seed, at the ratified ρ=1 and ρ=5 . ∗ : resolved at two standard errors, which indicates certainty under routing variation, not a meaningful effect (Poker’s − 0.1 ∗ and − 0.5 ∗ are sub-unit differences with zero variance).
ρ
No-Resp.
Always-Retr.
Binary
Alert-Only
Rollback-Only
Full RAR
Friedman
0.1
4.46
1.77
2.15
5.58
3.38
3.65
p<0.001
0.5
4.46
1.77
2.15
5.58
3.38
3.65
p<0.001
1
4.46
1.77
2.15
5.58
3.38
3.65
p<0.001
5
4.42
1.77
2.19
5.58
3.38
3.65
p<0.001
100
3.77
2.92
2.69
4.88
3.31
3.42
p=0.045
200
3.54
4.00
3.15
4.65
2.69
2.96
p=0.077
Table 3: Friedman mean ranks over the 13 stream blocks (rank 1 = best) at each ρ slice, with the tie-corrected omnibus test. Nemenyi critical difference at α=0.05 , k=6 , N=13 : CD=2.09 .
Figure 2: Critical-difference diagrams at ρ=1 (top) and ρ=100 (bottom); policies within one CD of each other are not separable. Full RAR is within one CD of every comparator at both prices.
Policy
Mean U(1)
Full RAR − policy
2 ⋅ SEM
+ / − seeds
Rollbacks/run
No-Resp.
291,741
+9,777 ∗
1,416
30/0
0.0
Always-Retr.
324,879
− 23,361 ∗
1,416
0/30
0.0
Binary
327,337
− 25,819 ∗
1,416
0/30
0.0
Alert-Only
291,721
+9,797 ∗
1,416
30/0
0.0
Rollback-Only
319,573
− 18,055 ∗
1,925
0/30
13.7
Full RAR
301,518
—
—
—
9.3
Table 4: E1: H2 on its specified subtle-corruption stream, 30 paired seeds, ρ=1 ( ∗ resolved). Rollbacks per run include all causes.
Stream
ρ=1
ρ=10
ρ=100
ρ=1000
RAR ahead from
ELEC2
− 77
− 323
+1,882 ∗
− 516
ρ=100
Covertype
− 32,812 ∗
− 29,956 ∗
− 11,886 ∗
+56,945 ∗
ρ=1000
Poker
− 0.1 ∗
− 1 ∗
− 10 ∗
− 100 ∗
never
Airlines
− 22,022 ∗
− 28,679 ∗
− 19,974 ∗
− 19,897 ∗
never
NOAA
+6 ∗
+48 ∗
+489 ∗
− 2,300 ∗
ρ=1
INS-abrupt
+2,769 ∗
+2,786 ∗
+2,956 ∗
+9,991 ∗
ρ=1
Table 5: Full RAR minus Cost-Aware Binary, paired by seed over 30 seeds, at the four real price points ( ∗ : resolved at two standard errors), and the first price at which Full RAR is ahead with a resolved difference. Cost-Aware Binary is not affine in ρ , so no analytic crossover is reported.
Stream
Onset
Always-Retr.
Binary
Rollback-Only
Full RAR
pre
frac
pre
frac
pre
frac
pre
frac
S-Abrupt
28,442
10.0
0.62
17.0
0.68
4.8
0.56
2.0
0.46
S-Gradual
364,701
134.0
0.62
158.0
0.55
66.9
0.62
36.3
0.61
S-Recurrent
11,398
4.0
0.67
3.0
0.60
1.8
0.56
1.0
0.35
S-Corrupt
338,570
125.0
0.63
42.0
1.00
20.9
1.00
13.1
1.00
S-RB-Drift
28,442
10.0
0.62
17.0
0.74
4.8
0.70
2.0
0.40
Table 6: Retrains before the first injected event on the five ground-truth streams: per-run mean count over 30 seeds (pre) and pooled pre-onset fraction (frac). An upper bound on false retrains; see text.
Stream group
Streams
Retrains
Alerts
Rollbacks
abrupt
2
5.8
24.7
2.2
gradual
2
6.6
46.9
2.3
recurrent
2
9.2
48.6
3.3
feature corruption
1
2.4
48.7
1.2
rollback then drift
1
11.0
38.3
4.4
real, type unspecified
3
7.9
34.7
3.4
Table 7: H4 on its specified quantity: Full RAR arm firings per 100k instances by drift-type group (mean over streams and seeds). Descriptive only; groups hold one to three streams.
Stream group
ρ=1
ρ=5
ρ=10
ρ=100
ρ=200
abrupt
AR
AR
AR
RAR h
RAR
gradual
AR/Bin =
AR/Bin =
AR/Bin =
AR
AR/RB =
recurrent
AR/Bin =
AR/Bin =
AR/Bin =
Bin h
Bin h
feature corruption
Bin
Bin
Bin
Bin
Bin
rollback then drift
AR
AR
AR
RAR
RAR
natural drift unspecified
AR/Bin =
AR h
AR h
Bin h
RB h
Table 8: Decision map v1 (drift type ×ρ→ best policy by mean within-stream rank). Bold: resolved and consistent across the group; = : tied rank winners; h : heterogeneous across the group’s streams. AR Always-Retrain, Bin Binary, RB Rollback-Only, RAR Full RAR, NR No-Response.
Stream
ΔU(1)
2 ⋅ SEM
Rollbacks/run
Drawdown RB
Drawdown Bin
ELEC2
− 246 ∗
126
6.8
0.572
0.600
Covertype
− 18,595 ∗
1,314
57.1
0.967
0.958
Poker
0
0
0.0
0.174
0.174
Airlines
− 9,530 ∗
1,351
13.7
0.626
0.636
NOAA
+59 ∗
35
1.5
0.249
0.324
INS-abrupt
− 12,383 ∗
1,667
2.6
0.629
0.610
Table 9: The canary as a one-factor change: Rollback-Only minus Binary, paired by seed, at ρ=1 ( ∗ resolved), with Rollback-Only’s mean rollbacks per run and both policies’ mean maximum accuracy drawdown.
Stream
ΔU(1)
2 ⋅ SEM
+ / −
Retr. RAR
Retr. no-can.
Rollb. no-can.
ELEC2
− 1,618 ∗
450
0/30
4.3
9.0
0.0
Covertype
− 14,242 ∗
1,744
0/30
60.6
99.0
0.0
Poker
0
0
0/0
0.0
0.0
0.0
Airlines
− 17,937 ∗
1,572
0/30
20.0
24.0
0.0
NOAA
− 167 ∗
0
0/30
2.0
2.0
0.0
INS-abrupt
− 11,854 ∗
1,790
0/30
6.9
11.0
0.0
Table 10: E2a: Full RAR minus Full RAR without the canary, paired by seed, ρ=1 ( ∗ resolved), with mean retrains per run and the no-canary policy’s rollbacks per run (zero everywhere: rollback fires only from canary states).
Cell
Phase A
Phase B
Cross-phase
canary.commit_z=0.0
+884
n.r.
0
n.r.
no sign
canary.delta=0.01
+853
n.r.
+4,548
R
same
canary.delta=0.05
+30
n.r.
− 263
n.r.
flip
canary.p1=0.1
+2,406
R
− 11,456
R
flip ∗
canary.p1=0.4
+996
n.r.
− 10,402
R
flip
canary.routing=iid
+1,430
R
+2,678
n.r.
same
Table 11: One-factor ablation of Full RAR, pooled paired ΔU(1) versus the ratified baseline. Phase A: ELEC2, NOAA, semi-corrupt, and rollback-then-drift, ten seeds. Phase B: INSECTS-abrupt and -recurrent, ten seeds. R: resolved at two standard errors; n.r.: not resolved; ∗ : sign flip resolved in both phases. Phase-B cells on which the two INSECTS streams disagree in sign are flagged in the artifact’s per-stream tables.
Stream
Mean ΔU
2⋅ SEM
Verdict
semi_corrupt_airlines
+12,384
2,005
RESOLVED
semi_rollback_then_drift
− 2,945
16
RESOLVED
river_elec2
+191
771
not resolved
noaa_weather
− 4.7
4.8
not resolved
Table 12: The sharpest heterogeneity case, inside Phase A alone: canary.p1 = 0.1 per stream, same ten seeds.
Stream
Policy
Stratified
i.i.d.
Paired Δ
Verdict
ELEC2
Full RAR
32,623 ± 1,233
32,952 ± 1,050
− 329 ± 289
n.r.
ELEC2
Rollback-Only
32,292 ± 345
32,123 ± 342
+169 ± 92
n.r.
INS-abrupt
Full RAR
189,793 ± 4,902
183,181 ± 2,070
+6,612 ± 1,037
resolved
INS-abrupt
Rollback-Only
174,641 ± 4,566
169,622 ± 8,959
+5,019 ± 1,734
resolved
S-RB-Drift
Full RAR
31,300 ± 16
30,310 ± 1,353
+990 ± 247
resolved
S-RB-Drift
Rollback-Only
28,087 ± 832
27,928 ± 662
+158 ± 188
n.r.
Table 13: Stage-1 routing: stratified (matrix default) versus i.i.d. at 30 seeds, paired by seed. Per-routing columns give mean ± s.d. over seeds; the paired difference gives mean ± SEM.
Metric
Two-tier
Single-tier
Paired Δ
Verdict
Accuracy
0.6909 ± 0.0003
0.6909 ± 0.0004
-0.0000 ± 0.0000
n.r.
Net utility ( ρ=1 )
31,300.4 ± 15.6
31,302.7 ± 17.8
-2.3 ± 2.1
n.r.
Retrains
5.00 ± 0.00
4.00 ± 0.00
+1.00 ± 0.00
resolved
Alerts
17.37 ± 0.49
16.00 ± 0.00
+1.37 ± 0.09
resolved
Rollbacks
2.00 ± 0.00
1.00 ± 0.00
+1.00 ± 0.00
resolved
Commits
2.00 ± 0.00
2.00 ± 0.00
+0.00 ± 0.00
n.r.
Table 14: H6 designated stream, two-tier (as designed) versus single-tier cooldown, 30 paired seeds. Primary metric — fraction of seeds hold-locked when the genuine drift arrives — is 0/30 for both variants. Per-variant columns give mean ± s.d. over seeds; the paired difference gives mean ± SEM.
Measure
Two-tier
Single-tier
Paired Δ
2 ⋅ SEM
Seeds + / − /0
Locked through genuine drift
0/30
30/30
—
—
primary
Accuracy
0.6695
0.6247
+0.0448 ∗
0.0123
30/0/0
Net utility at ρ=1
30,329
28,301
+2,027 ∗
558
30/0/0
Retrains
5.0
4.0
+1.0 ∗
0.0
30/0/0
Time below no-skill
13,313
16,391
− 3,078 ∗
988
0/30/0
Maximum accuracy drawdown
0.794
0.808
− 0.014 ∗
0.013
0/4/26
Table 15: Post-hoc H6 follow-up stream, two-tier versus naive single-tier, 30 paired seeds, regenerated from the per-cell payloads. ∗ denotes a difference resolved under the routing-variation rule; because the stream was constructed after the H6 null to activate the mechanism, these values are descriptive rather than confirmatory. The mechanism metric for this follow-up is the first row. The drawdown difference is resolved but present on only 4 of 30 seeds.
Stream
Full RAR
Binary
Always-Retr.
Rollback-Only
worst U
drawdown
worst U
drawdown
worst U
drawdown
worst U
drawdown
ELEC2
30,898
0.572
32,538
0.600
33,157
0.654
31,461
0.572
Covertype
362,827
0.889
401,208
0.958
391,981
0.988
375,214
0.967
Poker
513,697
0.174
513,697
0.174
513,318
0.174
513,697
0.174
Airlines
297,253
0.641
330,968
0.636
324,869
0.752
315,458
0.626
NOAA
12,452
0.250
12,446
0.324
12,585
0.304
12,428
0.249
Table 16: Tail metrics over 30 seeds: worst-seed net utility at ρ=1 and mean maximum rolling-accuracy drawdown (500-instance windows).
ρ=1
ρ=100
Stream
Feasibility
NB
HT
NB
HT
INS-abrupt
FEASIBLE
+4,472 ∗
+2,769 ∗
+4,593 ∗
+2,956 ∗
INS-recurr.
INFEASIBLE
− 38,970 ∗
− 19,094 ∗
− 38,967 ∗
− 19,510 ∗
S-Gradual
MIXED
− 16,863 ∗
− 28,020 ∗
+6,309 ∗
− 7,372 ∗
S-RB-Drift
FEASIBLE
− 4,050 ∗
+207 ∗
− 2,300 ∗
+1,864 ∗
S-Subtle
—
− 16,318 ∗
− 25,819 ∗
− 13,632 ∗
− 22,452 ∗
Table 17: E3: Full RAR minus Binary, paired by seed ( ∗ resolved), with Gaussian naive Bayes (NB) and the Hoeffding tree (HT) of the base matrix (E1 for the subtle stream).
Stream
Type
Onset lag
∣ bias ∣ , g=0
∣ bias ∣ , g=200
S-Abrupt
abrupt
191
0.081
0.003
S-Corrupt
corrupt
7935
0.231
0.222
S-Gradual
gradual
15
0.004
0.037
S-Recurrent
recurrent
767
0.423
0.313
S-RB-Drift
rollback_then_drift
639
0.208
0.151
S-Subtle
corrupt
3071
0.062
0.050
Table 18: Gate-3 input quality on the ground-truth streams (ADWIN, estimated-onset anchor): absolute bias of A^pre against the true pre-disturbance accuracy, with and without the guard gap.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Window anatomy on the worked example of Section 4 : t^d=9,500 , τ=800 , g=200 , evaluated at t=12,189 . Wref (1,000 instances, ending at t^d−g ) and Wcur (the newest 500 labeled instances) are frontier-clocked measurement windows; Wtrain (anchored at t^d , 1,890 labeled of 2,690) is the learning window; Wmon is a live-edge vigilance timer started by an alert at t=10,300 . Dashed gray spans are excluded from, or not yet available to, label-dependent computation.
ψ=0.05
ψ=0.10
ψ=0.20
Required labeled window n
δ=0.05
67
135
270
δ=0.03
187
376
753
δ=0.01
1,695
3,391
6,782
Detectable δ at window n
n=150
0.033
0.047
0.067
Appendix
Table 19: Paired (McNemar) power for the canary commit bar at the deployed zcommit=1 (one-sided α≈0.16 ), power 0.8, as a function of discordance ψ .