cs.LGNov 6, 2026

Multi-objective drift-aware MLOps agent for model retraining

Authors: Kumuda Valli, Anli Ji

Organizations: Computer Science California State University, Fullerton Fullerton, United States of America

Abstract

Deployed ML models face non-stationary data and delayed labels, yet most retraining strategies rely on fixed schedules or brittle drift thresholds that ignore operational cost. In this setting, retraining is a sequential decision problem rather than a one-time threshold choice. We present DARE (Drift-Aware Retraining Engine), an offline, event-driven retraining framework that separates ingestion, monitoring, policy decisions, training, and logging through explicit schemas and deterministic replay. We compare DARE against practical heuristic retraining baselines, including fixed-interval, threshold-persistence, and action-rate-matched fixed scheduling policies. On Airline, the locked LinUCB policy achieves a higher mean F1 while substantially reducing intervention frequency relative to heuristic and action-rate-matched baselines. Additional runtime-calibrated evaluation shows that these gains persist when measured using the cumulative update wall-clock time. These results suggest that conservative sequential retraining policies can improve performance while reducing unnecessary operational overhead under delayed feedback.

Figures & tables

Explore similar work

Aug 19, 2026

When to Retrain: An Empirical Study of Retraining Policies for Streaming ML Under Concept Drift, Budget, and Latency Constraints

Production machine learning systems degrade under concept drift, yet practitioners have little principled guidance on when to retrain. Retraining is costly, retraining budgets are finite, and a retrained model does not take effect instantly: training and deployment latency leave a stale model serving predictions while the data continues to move. We present a controlled empirical study of three practical model-refresh policies (periodic retraining, error-threshold triggering, and statistical drift-triggered retraining with ADWIN) against a no-retrain baseline, evaluated under a unified system model that makes retraining budgets and training-plus-deployment latency explicit. Across 3,933 experiment runs spanning three drift regimes, three budget levels, up to five latency levels, three datasets, and two learning modes, we find that the single most consequential design decision is not the retraining policy but whether the deployed model learns incrementally. With per-sample incremental updates, and for the linear online learner with immediate labels studied here, no policy differs from the no-retrain baseline by a practically significant margin in any of 54 paired comparisons, even at extreme latency. Without incremental updates, policy choice separates outcomes by 15-55 percentage points of post-drift accuracy, and simple periodic retraining significantly outperforms both reactive policies under abrupt and gradual drift, while reactive policies retain an advantage only under recurring drift. We document systematic failure modes of reactive policies and a latency-budget queueing interaction that silently halves effective retraining budgets, and release the full simulator, dataset pipelines, and per-run artifacts for reproducibility.
Nov 1, 2026

Retrain, Alert, or Rollback? A Three-Response Decision Framework for Production ML Under Concept Drift

When a deployed model degrades under concept drift, an operator can retrain it, raise an alert and monitor, or roll back. Yet most drift-adaptation methods consider only retraining. We present RAR (Retrain-Alert-Rollback), a state machine that gives each response an explicit trigger and cost. Retraining is gated on drift detection, label availability, and measured degradation. Every retrained model must pass a two-stage canary before promotion, and corruption-triggered and quality-triggered rollbacks enter different hold states. Under a prospectively specified protocol we compare RAR with six baselines, including a cost-aware retraining policy, on 13 streams: eight real public streams and five semi-synthetic streams derived from them. RAR does not dominate. At low retraining cost, retrain-heavy baselines lead, and no policy's rank differs significantly from RAR's. At higher cost, RAR beats detector-triggered retraining, periodic retraining, and no response simultaneously on four streams, each within a bounded cost window. Its advantage over detector-triggered retraining is consistent with a latency-feasibility condition stated in advance. Against the cost-aware policy, results are heterogeneous. RAR retrains before true drift onsets one eighth to one third as often as detector-triggered retraining, but at lower net utility on most semi-synthetic streams. On H2's designated corruption stream (a registered follow-up), rollbacks fire yet RAR loses to retraining baselines. At ρ=1ρ= 1, removing the canary, and thus rollback, raises utility on 11 of 13 streams. On a post-hoc follow-up stream, cause-specific hold release resumes adaptation where a single cooldown does not. We provide the harness, configurations, and per-cell results for reproduction.
Jul 9, 2026

ADORN: Adaptive Drift handling for Open RAN using Reinforcement Learning

Dynamic traffic variations in Open Radio Access Networks (O-RAN) lead to drift, which degrades the performance of Artificial Intelligence/Machine Learning (AI/ML) models. Traditional retraining approaches maintain forecasting accuracy but incur high computational cost and may lead to violations of Service Level Agreements (SLAs). This work proposes a Q-learning-based adaptive retraining approach that formulates the retraining decision as a Markov Decision Process (MDP), where a Reinforcement Learning (RL) agent learns a policy that balances forecasting accuracy and retraining cost. The proposed approach incorporates a multi-expert Long Short-Term Memory (LSTM) ensemble to mitigate catastrophic forgetting and improve robustness across diverse traffic conditions. Experimental results show that the proposed approach effectively reduces retraining overhead compared to greedy and random baselines, while maintaining system performance within predefined limits.