Predictive maintenance in Industry 4.0 refers to using data from sensors, machines, and production systems to estimate when equipment is likely to fail, so maintenance can be planned before a breakdown occurs [1]. However, a model that predicts maintenance may work perfectly in the lab but fail unexpectedly when applied to real factory data [2]. To solve this "reliability" gap, we evaluated six deep learning architectures across more than 700 experimental runs. We focused on the two dominant approaches in the field: Recurrent Neural Networks (RNNs), which process data step-by-step, like reading a sentence [3], and Transformers, a recent dominant approach, which look at the entire sequence at once to spot important connections [4]. We examined whether Transformers still outperform recurrent neural networks (RNNs) when the data includes noise [5]. We found that while Transformers excelled at tracking stable, slow-moving processes, they tend to overreact to chaotic data, mistakenly taking sensor noise for meaningful signals [6]. We also found that the hybrid method that combines a Long Short-Term Memory (LSTM) layer with a Transformer layer is more resilient to noisy data from factory shops [7]. Functioning as a noise filter, the LSTM smooths out data volatility, allowing the Transformer to focus on the bigger picture without being distracted [8]. The hybrid model did not just improve accuracy; it proved to be significantly more consistent than complex models, delivering reliable predictions regardless of how chaotic the underlying system became.
Figures & tables
Fig. 1: ACF Comparison. (a) Manufacturing data show high inertia. (b) IoT data show limited temporal dependence.
Fig. 2: Predictability Comparison. (a): The tight diagonal relationship indicates a predictable system. (b): The scattered cloud indicates a noisy, chaotic system.
Fig. 3: Stability Comparison. (a): Manufacturing data show high state persistence. (b): IoT data show state degradation.
Fig. 4: Architecture Comparison. Data flow (Bottom-to-Top) for the six evaluated models. (a) CNN (b) LSTM (c) CNN-LSTM (d) Pure Attention. (e) Our proposed hybrid (LSTM+Attention). (f) Parallel “Wide & Deep”.
Model
R2 Score (Mean ± Std)
RMSE
MAE
Transformer
0.644 ± 0.002
2.392
1.938
LSTM-Trans
0.644 ± 0.002
2.393
1.943
LSTM
0.643 ± 0.002
2.396
1.943
CNN-LSTM
0.642 ± 0.003
2.399
1.946
Tri-Hybrid
0.640 ± 0.003
2.406
1.951
CNN
0.633 ± 0.005
2.428
1.967
TABLE I: Manufacturing Dataset Results (20 Runs)
Fig. 5: Stability Analysis. The boxplots reveal that LSTM or Attention-based models have significantly lower variance than the Convolutional baseline.
Fig. 6: Learning Curves. Comparison of training (Blue) and validation (Orange) loss. All models show stable convergence with minimal overfitting.
Fig. 7: Prediction Analysis (Best Run). While all models track well in their best attempts, the CNN lacks the consistency of memory-based models across repeated trials.
Machine
Model
R2 (Mean ± Std)
RMSE
MAE
M001
LSTM
0.450 ± 0.004
8.054
7.612
(Chaotic)
Tri-Hybrid
0.448 ± 0.006
8.066
7.614
LSTM-Trans
0.448 ± 0.005
8.067
7.591
CNN-LSTM
0.446 ± 0.006
8.083
7.615
Transformer
0.443 ± 0.005
8.102
7.653
ResNet
0.437 ± 0.008
8.143
7.679
TABLE II: Model Performance by Machine Type
Fig. 8: Analysis of Chaotic Regime (M001). (a) Only LSTM-based models maintain tight variance; others show significant instability. (b) In the “Best Run” scenario, performance is visually identical, highlighting the need for stability metrics.
Model
R2 Score (Mean ± Std)
RMSE
MAE
LSTM-Trans
0.472 ± 0.004
7.823
7.323
CNN-LSTM
0.472 ± 0.003
7.825
7.330
Tri-Hybrid
0.472 ± 0.004
7.828
7.334
LSTM
0.471 ± 0.002
7.833
7.347
Transformer
0.471 ± 0.004
7.837
7.350
ResNet
0.470 ± 0.004
7.843
7.345
TABLE III: Global Dataset Results
Fig. 9: Analysis of Global Test. (a) The mixed dataset increases variance for all models, but the LSTM-Transformer remains the most stable. (b) The “Best Run” plots are virtually identical, confirming that capacity is not the bottleneck—consistency is.
Providing reliable predictive maintenance is a critical industrial AI service essential for ensuring the high availability of manufacturing devices. Existing deep-learning methods present competitive results on such tasks but lack a general service-oriented framework to capture complex dependencies in industrial IoT sensor data. While Transformer-based models show strong sequence modeling capabilities, their direct deployment as robust AI services faces significant bottlenecks. Specifically, streaming sensor data collected in real-world service environments often exhibits multi-scale temporal correlations driven by machine working principles. Besides, the datasets available for training time-to-failure predictive services are typically limited in size. These issues pose significant challenges for directly applying existing models as robust predictive services. To address these challenges, we propose MsFormer, a lightweight Multi-scale Transformer designed as a unified AI service model for reliable industrial predictive maintenance. MsFormer incorporates a Multi-scale Sampling (MS) module and a tailored position encoding mechanism to capture sequential correlations across multi-streaming service data. Additionally, to accommodate data-scarce service environments, MsFormer adopts a lightweight attention mechanism with straightforward pooling operations instead of self-attention. Extensive experiments on real-world datasets demonstrate that the proposed framework achieves significant performance improvements over state-of-the-art methods. Furthermore, MsFormer outperforms across industrial devices and operating conditions, demonstrating strong generalizability while maintaining a highly reliable Quality of Service (QoS).
Jiahui Zhou, Dan Li, Ruibing Jin +5
School of Software Engineering, Sun Yat-Sen University, China · Agency for Science, Technology and Research (A*STAR), Singapore · School of the Art Institute of Chicago, Chicago +1
Deep learning predictive maintenance models suffer from poor transferability across machines and operating conditions, especially when labelled data are scarce and signals span five orders of magnitude in sampling frequency (1 Hz to ~100 kHz). We propose FreqCondNorm, a Transformer-based architecture that introduces a FiLM-style frequency-conditioned normalization layer to unify heterogeneous time-series within a single model. The architecture is pretrained on five public predictive maintenance datasets (CWRU, MFPT, UOC18, PRONOSTIA, CMAPSS) using masked auto-encoding and contrastive learning with balanced domain sampling. On fault diagnosis, the model achieves 99.2% accuracy on CWRU (+6.4 pp over CNN) and 82.1% zero-shot accuracy on MFPT, demonstrating strong transfer across sampling frequencies. However, the approach does not improve remaining useful life prediction, suggesting a mismatch between pretraining and RUL objectives that warrants future investigation.
Zaynab Raounak, Camille LHermine, Zhiguo Zeng
Laboratoire Génie Industriel, CentraleSupélec, Université Paris-Saclay, France
Unplanned network hardware malfunctions can interrupt services and result in expensive downtime in data centers. A deep learning-based predictive maintenance strategy is presented that utilizes thermal imaging and power sensor data to detect early indicators of equipment breakdown in routers, switches, and servers. A simulated dataset was generated comprising annotated thermal pictures and power readings indicative of three operating states: Normal, Warning, and Critical. Three ImageNet-pretrained convolutional neural network (CNN) models ResNet-50, InceptionV3, and VGG16 were assessed together with a multi-modal CNN-LSTM fusion model that integrates visual and sensor time-series information. Experiments were performed with and without pre-processing procedures, including region-of-interest (ROI) extraction and normalization. In the absence of pre-processing, CNNs attained moderate accuracy (e.g., ResNet-50 at 52%), but ROI-based pre-processing significantly enhanced performance (ResNet-50 accuracy reaching 91%). The CNN-LSTM model attained the greatest accuracy of 94%, with precision and recall approaching 95%, illustrating the effectiveness of multi-modal fusion. The results validate that domain-specific pre-processing and sensor fusion substantially improve early failure prediction, providing a potential foundation for proactive maintenance of network hardware through non-intrusive monitoring.