Do RUL explanations hold up? Faithfulness and stability of attributions on C-MAPSS
Organizations: AI Lab, Phuong Hai JSC, Vietnam · International College of Management Sydney, Australia · School of Science, Engineering, and Technology, RMIT, Vietnam · FPT, Vietnam
Abstract
Deep remaining-useful-life (RUL) models on NASA C-MAPSS are now routine, and so are heatmaps that colour sensors and timesteps. A heatmap that looks mechanical is not the same as an explanation an engineer can act on. We train three standard architectures - a 1D CNN, an LSTM, and a small Transformer encoder - on the official FD001 and FD003 splits with the piecewise RUL cap of 125 cycles and the official PHM08 asymmetric score. We then attach three attribution maps (Integrated Gradients, occlusion, last-layer attention) and evaluate them with the checks the XAI-for-PdM literature still under-reports: deletion/insertion faithfulness, Spearman stability under sensor-scale noise, agreement across training seeds, and cosine consistency inside RUL bins. Prediction error is a prerequisite, not the claim. The headline is which explanation method moves the RUL output when its top cells are removed, and which map survives a 5% input perturbation. Integrated Gradients and occlusion are similarly faithful on the LSTM; Transformer attention is cheap and temporally smooth but weakly faithful. All three maps are almost unchanged under 5% input noise, yet IG/occlusion agree only moderately across two LSTM seeds - stability to sensor jitter is not the same as stability to retraining. A secondary tabular check on the AI4I 2020 failure dataset shows the same deletion pattern for tree importances. We recommend occlusion or IG for any C-MAPSS-style report that will be read by a maintenance engineer, and we treat raw attention weights as a visualisation only.
Figures & tables
| Subset | Model | RMSE | C-MAPSS score |
|---|---|---|---|
| FD001 | cnn | 18.70 | 905.1 |
| FD001 | lstm | 15.54 | 436.1 |
| FD001 | transformer | 16.13 | 807.0 |
| FD003 | cnn | 19.65 | 1039.5 |
| FD003 | lstm | 13.67 | 324.5 |
| FD003 | transformer | 15.88 | 608.2 |
| Method | Del@20% | Ins@20% | Stability | Seed |
|---|---|---|---|---|
| Integrated Gradients | 35.79 | 11.68 | 0.981 | 0.634 |
| Occlusion | 35.26 | 10.68 | 0.980 | 0.559 |
| Attention | 12.61 | 39.13 | 0.982 | — |
| Subset | RMSE | Score | IG @20% | Occ @20% |
|---|---|---|---|---|
| FD002 | 17.90 | 1845 | 41.5 | 60.8 |
| FD004 | 20.43 | 4602 | 35.1 | 51.6 |
Feature Attribution
Feature attribution estimates which input features or internal components contribute to a model’s predictions.
See all 229 papers