Towards Differentiating Between Failures and Domain Shifts in Industrial Data Streams
Authors: Natalia Wojak-Strzelecka, Szymon Bobek, Grzegorz J. Nalepa, Jerzy Stefanowski
Organizations: 1Jagiellonian Human-Centered AI Lab, Mark Kac Center for Complex Systems Research, Institute of Applied Computer Science, Faculty of Physics, Astronomy and Applied Computer Science, Jagiellonian University, ul. prof. Stanisława Łojasiewicza 11, 30-348 Krakow, Poland · Institute of Computing Science, Poznań University of Technology, 60-965 Poznań, Poland
Anomaly and failure detection methods are crucial in identifying deviations from normal system operational conditions, which allows for actions to be taken in advance, usually preventing more serious damages. Long-lasting deviations indicate failures, while sudden, isolated changes in the data indicate anomalies. However, in many practical applications, changes in the data do not always represent abnormal system states. Such changes may be recognized incorrectly as failures, while being a normal evolution of the system, e.g. referring to characteristics of starting the processing of a new product, i.e. realizing a domain shift. Therefore, distinguishing between failures and such ''healthy'' changes in data distribution is critical to ensure the practical robustness of the system. In this paper, we propose a method that not only detects changes in the data distribution and anomalies but also allows us to distinguish between failures and normal domain shifts inherent to a given process. The proposed method consists of a modified Page-Hinkley changepoint detector for identification of the domain shift and possible failures and supervised domain-adaptation-based algorithms for fast, online anomaly detection. These two are coupled with an explainable artificial intelligence (XAI) component that aims at helping the human operator to finally differentiate between domain shifts and failures. The method is illustrated by an experiment on a data stream from the steel factory.
Figures & tables
Figure 1: The flow chart for differentiating between failures and domain shifts. The changepoint detection algorithm marks possible domain shifts and failures. After that, the domain adaptation algorithm updates the model that detects anomalies. Simultaneously human operator accompanied by the XAI algorithm decides if the change in the data represents a healthy domain shift or a failure.
Figure 2: Raw signal for current_2 on the left, SHAP values plotted in batches for current_2 on the right.
Figure 3: The schematic diagram of four stand cold rollling mill.
Feature
Description
Unit
ithick/othick
coil thickness at the beginning/end of rolling
mm
width
coil width
mm
ys0/ys1
yield strength at the beginning/end of rolling
MPa
work_roll_diam
diameter of roll
mm
work_roll_mileage
mileage of roll
mm
reduction
reduction on specific stand
%
Table 1: Description of cold rolling mill dataset.
Figure 4: Anomaly detection algorithms evaluation on target products on important signals from rolling stand 2.
Figure 5: SHAP values median for: a) all parameters (on left), b) for important parameters for anomaly detection (on right).
Figure 6: Detected changepoints in the dataset.
Figure 7: Boxplots of SHAP values important for bearing diagnosis.
A reading departs from what a monitoring system expects for one of three reasons: the physical process changed, the instrument is failing, or the value was corrupted between the instrument and the software. The three call for different responses and look alike in a single sample. We state the three-cause problem, prove which pairs of causes one channel cannot separate, and propose a bank of online detectors - a Kalman tracker with CUSUM and EWMA charts on its innovation, a Hampel median with generalized-ESD confirmation, a robust noise statistic with ADWIN, exact-value checks and an optional two-channel complementary filter - combined by a deterministic, inspectable attribution rule. The detectors were merged into a public algorithm library. On 2,942 injected events of thirteen fault types the method detects 96.3% (95% interval 95.7-97.0) at 0.22 false alarms per hour and attributes 91.2% (90.6-91.8) of detected events correctly from one channel and 99.0% (98.6-99.4) with a second, rate channel; from one channel only 2 of 207 detected sensor drifts are attributed to the instrument, as the problem statement predicts. Injected into recorded industrial signals, the same catalogue is detected at 80.8% and attributed correctly at 84.5%, at 3.6 false alarms per hour. On the Skoltech Anomaly Benchmark the method detects 32 of 33 experiments with a median delay of 15 s, at 46 false alarms per hour against 11 for a rolling z-score.
Effective anomaly detection frameworks are a central pillar of the Industry 4.0 paradigm. In this paper, we introduce an Optimal Transport (OT)-based framework for anomaly detection, designed to detect deviations from normal behaviour in time-series sensor data. The OT-based method requires minimal user input and adapts to real-time data without the need for labelled training data. Our method effectively addresses existing limitations related to data labelling, generalisability, and scalability, demonstrating resilience against short-term fluctuations, noise, and data gaps - common challenges in industrial environments. Additionally, our method provides counterfactual explanations improving the auditability of the approach when deployed in industrial settings. The proposed method learns the mapping between normal and observed operating conditions through a sliding reference window that adapts to the dynamicity of the data. We evaluate our approach on three industrial datasets, from shipping, industrial HVAC systems, and publicly available benchmark data. The method was highly effective in identifying anomalies and reducing false positives, outperforming traditional methods, while maintaining computational efficiency and ease of configuration.
Abigail Langbridge, Fearghal O'Donncha, James T Rayfield +1
Dyson School of Design Engineering Imperial College London* United Kingdom · AI for Sustainable Industry IBM Research Europe Dublin, Ireland · IBM T.J. Watson Research Center IBM Research NY, USA
We present a collaborative streaming anomaly detection system for high-speed data streams that explicitly integrates human analysts into the decision loop. The system combines heterogeneous detectors and aggregates their outputs through a normalization-based weighted consensus, complemented by artifact-aware rules to stabilize anomaly scoring under deployment. To improve interpretability, it derives surrogate models that approximate the ensemble consensus and expose human-readable sensor conditions associated with anomalous behavior. Analysts can actively intervene by reviewing anomaly episodes, adjusting consensus behavior, and refining surrogate rules used for anomaly prediction, producing a human-adjusted ensemble. We evaluate the approach on an industrial stream with 260,000 events and 3 anomalous episodes, showing robust detection and actionable human-AI interaction.
Diogo Risca, Afonso Lourenço, Ricardo Martins +1
GECAD, ISEP, Polytechnic of Porto, Portugal · SISTRADE Software Consulting, Porto, Portugal