MsFormer: Enabling Robust Predictive Maintenance Services for Industrial Devices

Jiahui Zhou; Dan Li; Ruibing Jin; Jian Lou; Yanran Zhao; Zhenghua Chen; Zigui Jiang; See-Kiong Ng

cs.LGMar 24, 2026

MsFormer: Enabling Robust Predictive Maintenance Services for Industrial Devices

Authors: Jiahui Zhou, Dan Li, Ruibing Jin, Jian Lou, Yanran Zhao, Zhenghua Chen, Zigui Jiang, See-Kiong Ng

Organizations: School of Software Engineering, Sun Yat-Sen University, China · Agency for Science, Technology and Research (A*STAR), Singapore · School of the Art Institute of Chicago, Chicago · Institute of Data Science and School of Computing, National University of Singapore, Singapore

Abstract

Providing reliable predictive maintenance is a critical industrial AI service essential for ensuring the high availability of manufacturing devices. Existing deep-learning methods present competitive results on such tasks but lack a general service-oriented framework to capture complex dependencies in industrial IoT sensor data. While Transformer-based models show strong sequence modeling capabilities, their direct deployment as robust AI services faces significant bottlenecks. Specifically, streaming sensor data collected in real-world service environments often exhibits multi-scale temporal correlations driven by machine working principles. Besides, the datasets available for training time-to-failure predictive services are typically limited in size. These issues pose significant challenges for directly applying existing models as robust predictive services. To address these challenges, we propose MsFormer, a lightweight Multi-scale Transformer designed as a unified AI service model for reliable industrial predictive maintenance. MsFormer incorporates a Multi-scale Sampling (MS) module and a tailored position encoding mechanism to capture sequential correlations across multi-streaming service data. Additionally, to accommodate data-scarce service environments, MsFormer adopts a lightweight attention mechanism with straightforward pooling operations instead of self-attention. Extensive experiments on real-world datasets demonstrate that the proposed framework achieves significant performance improvements over state-of-the-art methods. Furthermore, MsFormer outperforms across industrial devices and operating conditions, demonstrating strong generalizability while maintaining a highly reliable Quality of Service (QoS).

Figures & tables

Figure 1: The overall architecture of the MsFormer.

Figure 2: The illustration for the multi-scale position encoding mechanism.

Subsets	FD001	FD002	FD003	FD004
Operation conditions	1	6	1	6
Fault modes	1	1	2	2
Training trajectories	100	260	100	248
Testing trajectories	100	259	100	249

Table 1: Details of the C-MAPSS Dataset.

Dataset	FD001		FD002		FD003		FD004
Evaluation	RMSE	Score	RMSE	Score	RMSE	Score	RMSE	Score
BiLSTM [ 27 ]	13.65	295.00	23.18	4130.00	13.74	317.00	24.86	5430.00
KDnet [ 32 ]	13.68	362.08	14.47	929.20	12.95	327.27	15.96	1303.19
PE-Net [ 12 ]	13.98	280.87	14.69	881.73	12.33	272.85	15.40	1103.18
TF-SCN [ 13 ]	12.05	219.00	14.71	1358.00	12.11	238.00	16.95	1367.00
IMDSSN [ 36 ]	12.14	206.11	17.40	1775.15	12.35	229.54	19.78	2852.81

Table 2: Performance comparison of other methods on the C-MAPSS dataset.

Dataset	B5		B6		B7		B18		Average
Evaluation	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE
DeTransformer [ 3 ]	6.08	5.26	14.84	13.67	3.86	3.44	9.99	8.13	8.69	7.63
EM-UPF-W [ 35 ]	5.65	4.58	10.10	7.51	7.06	5.21	8.69	6.38	7.88	5.92
IMDSSN [ 36 ]	8.54	7.36	12.03	10.67	9.46	8.51	5.65	4.63	8.92	7.79
Bao et al. [ 2 ]	7.89	6.80	14.13	11.30	4.30	3.67	13.18	11.01	9.87	8.19
Dual-Mixer [ 7 ]	7.90	7.20	11.17	10.15	7.75	6.12	6.30	5.47	8.28	7.23

Table 3: Performance comparison on the NASA Dataset, with values scaled by

10^{2}

Stage 1	Stage 2	Stage 3	RMSE	Score
			13.98	897.28
✓			14.07	876.18
	✓		14.13	885.58
		✓	13.91	879.78
✓	✓		14.03	848.40
✓		✓	14.12	825.45

Table 4: Results of the MS module applied at different stages on FD002.

Figure 3: Performance comparison with fixed dilation factor $\lambda_{w}$ in the MS module.

Figure 4: Results of various dilation factors $\lambda_{w}$ in the MS module on FD002.

Dataset	FD001		FD002		FD003		FD004
Evaluation	RMSE	Score	RMSE	Score	RMSE	Score	RMSE	Score
w/o RPE	12.31	212.10	14.06	941.34	12.60	277.92	15.00	1021.74
Shaw et al. [ 26 ]	13.84	285.10	14.41	941.09	12.60	274.91	15.13	1004.25
DIET [ 5 ]	10.99	169.45	13.57	854.14	12.55	263.21	14.83	1037.11
T5’s RPE [ 24 ]	11.87	194.43	14.04	887.36	12.18	284.51	14.83	1045.46
MsFormer( ours )	10.94	178.50	13.64	766.70	11.62	236.91	14.81	961.42

Table 5: Results for position encoding mechanisms applied in the RPE block.

Figure 5: Sensitivity analysis for parameter $c_{1}$ in position encoding mechanisms.

Attention	Stages				Score
Layers	1	2	3	4	FD001	FD002	FD003	FD004
4	LA	LA	RPE(2)	RPE(2)	186.46	867.68	309.12	1017.78
4	LA	LA	RPE(3)	RPE	173.13	874.85	345.00	1040.54
4	LA	RPE	RPE(2)	PE	258.08	847.94	309.22	935.33
5	RPE	RPE	RPE(2)	PE	229.77	922.54	319.53	974.72
3( ours )	LA	LA	RPE(2)	RPE	178.50	766.70	236.91	961.42

Table 6: Different attention layer setting strategies across stages of MsFormer. We diversely add attention layers to the RPE (Relative Position Encoding) block or replace the the LA (Lightweight Attention) blocks with attention layers from the RPE block.

Evaluation	Params(M)
FEDformer	>114
Informer	>11
Autoformer	>10
Pyraformer	>3
Metaformer	>12
Swin Transformer	>28

Table 7: Complexity comparison.

Figure 6: Visualization of time-to-failure prediction on C-MAPSS and NASA Datasets.

Explore similar work

Benchmarking Hybrid Deep Learning Architectures for Predictive Maintenance in Industry 4.0
Sep 18, 2026Zhengyang, Gu, Joseph E. Hernandez +4Predictive Maintenance Time Series Transformers
Cissy · Data Science, Liveline Technologies Livonia, MI, USA · Engineering, Liveline Technologies Livonia, MI, USA
Self-Evolving Multi-Agent Network for Industrial IoT Predictive Maintenance
Feb 17, 2026Rebin Saleh, Khanh Pham Dinh, Balázs Villányi +1Predictive Maintenance Industrial Anomaly Detection
Department of Electronics Technology, Faculty of Electrical Engineering and Informatics, Budapest University of Technology and Economics, Műegyetem rkp. 3, H-1111 Budapest, Hungary · DataScienceWorld.Kan, Hanoi, Vietnam · Department of Computer Science, The University of Alabama at Birmingham, 1720 University Blvd, Birmingham, AL 35294 USA
Time-Series Foundation Model Embeddings for Remaining Useful Life Estimation
Jun 10, 2026Amir El-Ghoussani, Michele De Vita, Ronald Naumann +1RUL Estimation Predictive Maintenance
Friedrich-Alexander-Universität Erlangen–Nürnberg (FAU), Germany · Nokia Solutions and Networks GmbH & Co. KG, Germany

Sep 18, 2026J/K move · Enter open · S save

Benchmarking Hybrid Deep Learning Architectures for Predictive Maintenance in Industry 4.0

Zhengyang, Gu, Joseph E. Hernandez, Thomas Cook+3

Cissy · Data Science, Liveline Technologies Livonia, MI, USA · Engineering, Liveline Technologies Livonia, MI, USA

Predictive Maintenance Time Series Transformers Neural Network Robustness Recurrent Neural Networks Recurrent Transformers

Open paper AskPDF arXiv

Predictive maintenance in Industry 4.0 refers to using data from sensors, machines, and production systems to estimate when equipment is likely to fail, so maintenance can be planned before a breakdown occurs [1]. However, a model that predicts maintenance may work perfectly in the lab but fail unexpectedly when applied to real factory data [2]. To solve this "reliability" gap, we evaluated six deep learning architectures across more than 700 experimental runs. We focused on the two dominant approaches in the field: Recurrent Neural Networks (RNNs), which process data step-by-step, like reading a sentence [3], and Transformers, a recent dominant approach, which look at the entire sequence at once to spot important connections [4]. We examined whether Transformers still outperform recurrent neural networks (RNNs) when the data includes noise [5]. We found that while Transformers excelled at tracking stable, slow-moving processes, they tend to overreact to chaotic data, mistakenly taking sensor noise for meaningful signals [6]. We also found that the hybrid method that combines a Long Short-Term Memory (LSTM) layer with a Transformer layer is more resilient to noisy data from factory shops [7]. Functioning as a noise filter, the LSTM smooths out data volatility, allowing the Transformer to focus on the bigger picture without being distracted [8]. The hybrid model did not just improve accuracy; it proved to be significantly more consistent than complex models, delivering reliable predictions regardless of how chaotic the underlying system became.

Abstract

Explore similar work

Benchmarking Hybrid Deep Learning Architectures for Predictive Maintenance in Industry 4.0

Self-Evolving Multi-Agent Network for Industrial IoT Predictive Maintenance

Time-Series Foundation Model Embeddings for Remaining Useful Life Estimation