Planning or Learning: Reliability and Cost in Multi-Asset Maintenance

Xian Yeow Lee; Chandrasekar Venkatraman; Ahmed Farahat

cs.AISep 11, 2026

Planning or Learning: Reliability and Cost in Multi-Asset Maintenance

Authors: Xian Yeow Lee, Chandrasekar Venkatraman, Ahmed Farahat

Organizations: Industrial AI Lab, Hitachi America Ltd. Santa Clara, U.S.A.

Abstract

Industrial maintenance systems involve multiple interacting assets and shared resources, making it challenging to balance reliability and operational cost using a single decision framework. While recent work has focused on reinforcement learning (RL) for maintenance scheduling, direct comparisons with planning approaches under identical settings remain limited. In this work, we empirically compare planning and RL for multi-asset bearing maintenance using run-to-failure data. We examine how these methods behave when balancing preventive maintenance against tolerable failures across a range of failure penalty scenarios. We observed a consistent behavioral difference driven by objective formulation. Planning enforces reliability as a hard constraint and produces zero-failure policies whose total cost is largely insensitive to the magnitude of failure penalties. RL agents optimize expected cost and often trade off preventive maintenance against occasional failures as penalties vary, resulting in lower costs under low-penalty regimes but persistent non-zero failures even when penalties are high. We also investigate lightweight constraint mechanisms, including reward shaping and action masking, to encourage RL's reliability. From a practical perspective, planning may be more suitable when strict reliability is required and deployment horizons are short, whereas RL may provide cost-efficient policies when limited failures are acceptable and long-run operational efficiency is prioritized. Overall, this study clarifies the trade-offs between reliability and cost in multi-asset maintenance and suggests that planning and RL are complementary approaches. Beyond these findings, the controlled benchmark protocol itself that unifies environment, cost model, and evaluation across paradigms, offers a reusable template for comparing decision-making approaches in other maintenance settings.

Figures & tables

Figure 1: (a) Pipeline from raw vibration signals to health indicators to maintenance decisions. (b) Raw horizontal (blue) and vertical (red) accelerometer signals for a bearing at early stages (row 1), mid-life (row 2), and near-failure stages (row 3). (c) Composite RMS health indicator trajectories for all six bearings, each following a distinct degradation profile before failure.

Figure 2: Pareto tradeoff between cost and failures across all methods and failure penalties $c_{f}$ , where marker size $\propto c_{f}$ and filled = oracle dynamics, hollow = estimated dynamics. Planning and heuristic are anchored at zero failures; RL methods cluster in the high-failure, low-cost region regardless of $c_{f}$ , reflecting the hard-constraint vs. soft-cost formulation gap.

Oracle dynamics
Method	$c_{\text{f}}=100$		$c_{\text{f}}=500$		$c_{\text{f}}=1000$		$c_{\text{f}}=5000$		$c_{\text{f}}=10000$
Method	Cost	Failures	Cost	Failures	Cost	Failures	Cost	Failures	Cost	Failures
Threshold	231336	0.0	231336	0.0	231336	0.0	231336	0.0	231336	0.0
Dijkstra	49176	0.0	49176	0.0	49176	0.0	49176	0.0	49176	0.0
DQN	8134 $\pm$ 0	6.0 $\pm$ 0.0	10534 $\pm$ 0	6.0 $\pm$ 0.0	13534 $\pm$ 0	6.0 $\pm$ 0.0	37534 $\pm$ 0	6.0 $\pm$ 0.0	66856 $\pm$ 959	5.7 $\pm$ 0.5
PPO	15918 $\pm$ 2561	5.2 $\pm$ 0.4	32973 $\pm$ 3751	3.4 $\pm$ 0.4	25330 $\pm$ 4721	5.1 $\pm$ 0.8	47427 $\pm$ 3293	2.6 $\pm$ 1.4	54780 $\pm$ 2542	1.2 $\pm$ 0.8

Table 1: Main results for planning, heuristic, and vanilla RL methods across all failure penalty levels. Cost is the mean total episode cost; Failures is the mean number of bearing failures per episode. Standard deviation across three seeds is shown for RL methods. Dijkstra and Threshold are deterministic (no standard deviation).

Method	Training	Inference (per episode)
Threshold	—	28 s
Dijkstra	—	$\sim$ 90 min
DQN	$\sim$ 40 min	9 s
PPO	$\sim$ 68 min	10 s

Table 2: Training and inference runtimes for all methods.

Oracle dynamics
Method	$c_{\text{f}}=100$		$c_{\text{f}}=500$		$c_{\text{f}}=1000$		$c_{\text{f}}=5000$		$c_{\text{f}}=10000$
Method	Cost	Fail	Cost	Fail	Cost	Fail	Cost	Fail	Cost	Fail
DQN shaped	8134 $\pm$ 0	6.0 $\pm$ 0.0	10534 $\pm$ 0	6.0 $\pm$ 0.0	13534 $\pm$ 0	6.0 $\pm$ 0.0	37534 $\pm$ 0	6.0 $\pm$ 0.0	65744 $\pm$ 1373	4.7 $\pm$ 1.2
DQN masked	33703 $\pm$ 0	5.0 $\pm$ 0.0	35703 $\pm$ 0	5.0 $\pm$ 0.0	40928 $\pm$ 3853	5.0 $\pm$ 0.0	58577 $\pm$ 529	4.7 $\pm$ 0.5	83458 $\pm$ 361	4.3 $\pm$ 0.9
PPO shaped	20710 $\pm$ 5377	5.1 $\pm$ 0.8	30087 $\pm$ 4023	4.1 $\pm$ 0.2	28033 $\pm$ 3620	3.9 $\pm$ 0.7	47048 $\pm$ 6364	2.2 $\pm$ 2.1	55429 $\pm$ 4298	1.5 $\pm$ 1.7
PPO masked	56810 $\pm$ 8411	2.0 $\pm$ 1.6	56979 $\pm$ 8237	2.2 $\pm$ 0.8	54924 $\pm$ 1549	2.3 $\pm$ 0.9	58846 $\pm$ 2265	0.0 $\pm$ 0.0	73921 $\pm$ 15265	1.1 $\pm$ 1.4

Table 3: DQN and PPO performance with constraint mechanisms: reward shaping and safety mask, under oracle and estimated dynamics.

Oracle dynamics
Method	$c_{\text{f}}=100$		$c_{\text{f}}=500$		$c_{\text{f}}=1000$		$c_{\text{f}}=5000$		$c_{\text{f}}=10000$
Method	Cost	Fail	Cost	Fail	Cost	Fail	Cost	Fail	Cost	Fail
DQN cont. $\gamma=0.99$	8134 $\pm$ 0	6.0 $\pm$ 0.0	10534 $\pm$ 0	6.0 $\pm$ 0.0	13534 $\pm$ 0	6.0 $\pm$ 0.0	37534 $\pm$ 0	6.0 $\pm$ 0.0	66856 $\pm$ 959	5.7 $\pm$ 0.5
DQN cont. $\gamma=0.999$	8134 $\pm$ 0	6.0 $\pm$ 0.0	10534 $\pm$ 0	6.0 $\pm$ 0.0	13534 $\pm$ 0	6.0 $\pm$ 0.0	37534 $\pm$ 0	6.0 $\pm$ 0.0	66795 $\pm$ 578	5.3 $\pm$ 0.5
DQN disc. $\gamma=0.99$	8134 $\pm$ 0	6.0 $\pm$ 0.0	10534 $\pm$ 0	6.0 $\pm$ 0.0	13534 $\pm$ 0	6.0 $\pm$ 0.0	37534 $\pm$ 0	6.0 $\pm$ 0.0	65895 $\pm$ 1159	5.3 $\pm$ 0.5
DQN disc. $\gamma=0.999$	8134 $\pm$ 0	6.0 $\pm$ 0.0	10534 $\pm$ 0	6.0 $\pm$ 0.0	13534 $\pm$ 0	6.0 $\pm$ 0.0	37534 $\pm$ 0	6.0 $\pm$ 0.0	67534 $\pm$ 0	6.0 $\pm$ 0.0

Table 4: Ablation study: Effect of discount factor on DQN with continuous and discrete observations under oracle dynamics.

Predictive Maintenance

Predictive maintenance uses data and models to detect equipment faults, estimate degradation and remaining life, and plan timely repairs.

See all 118 papers

Mar 12, 2026

The price of decentralization in managing engineering systems through multi-agent reinforcement learning

Inspection and maintenance (I&M) planning involves sequential decision making under uncertainties and incomplete information, and can be modeled as a partially observable Markov decision process (POMDP). While single-agent deep reinforcement learning provides approximate solutions to POMDPs, it does not scale well in multi-component systems. Scalability can be achieved through multi-agent deep reinforcement learning (MADRL), which decentralizes decision-making across multiple agents, locally controlling individual components. However, this decentralization can induce cooperation pathologies that degrade the optimality of the learned policies. To examine these effects in I&M planning, we introduce a set of deteriorating systems in which redundancy is varied systematically. These benchmark environments are designed such that computation of centralized (near-)optimal policies remains tractable, enabling direct comparison of solution methods. We implement and benchmark a broad set of MADRL algorithms spanning fully centralized and decentralized training paradigms, from value-factorization to actor-critic methods. Our results show a clear effect of redundancy on coordination: MADRL algorithms achieve near-optimal performance in series-like settings, whereas increasing redundancy amplifies coordination challenges and can lead to optimality losses. Nonetheless, decentralized agents learn structured policies that consistently outperform optimized heuristic baselines, highlighting both the promise and current limitations of decentralized learning for scalable maintenance planning.

Prateek Bhustali, Pablo G. Morato, Konstantinos G. Papakonstantinou +1

Faculty of Architecture and the Built Environment, Delft University of Technology, Delft, 2628 BL, The Netherlands · Engineering Risk Analysis Group, Technical University of Munich, Munich, 80333, Germany · Department of Civil & Environmental Engineering, The Pennsylvania State University, University Park, PA, 16802, USA

Decentralized MARL Multi-Agent Coordination Cooperative MARL

Ask PDF

Mar 9, 2026

Evidence-Driven Reasoning for Industrial Maintenance Using Heterogeneous Data

Industrial maintenance platforms contain rich but fragmented evidence, including free-text work orders, heterogeneous operational sensors or indicators, and structured failure knowledge. These sources are often analyzed in isolation, producing alerts or forecasts that do not support conditional decision-making: given this asset history and behavior, what is happening and what action is warranted? We present Condition Insight Agent, a deployed decision-support framework that integrates maintenance language, behavioral abstractions of operational data, and engineering failure semantics to produce evidence-grounded explanations and advisory actions. The system constrains reasoning through deterministic evidence construction and structured failure knowledge, and applies a rule-based verification loop to suppress unsupported conclusions. Case studies from production CMMS deployments show that this verification-first design operates reliably under heterogeneous and incomplete data while preserving human oversight. Our results demonstrate how constrained LLM-based reasoning can function as a governed decision-support layer for industrial maintenance.

Fearghal O'Donncha, Nianjun Zhou, Natalia Martinez +4

IBM Research Europe, Ireland · IBM Research Yorktown, USA · Imperial College London, UK

Evidence-Grounded Reasoning AI-Assisted Decision Making LLM Decision-Making Fault Diagnosis

Ask PDF

Aug 26, 2026

Uncertainty and Business-Aware Remaining Useful Life Estimation for Semiconductor Manufacturing

Semiconductor manufacturing relies on tightly interconnected components, so early identification of the assets most likely to fail is essential to prevent a single breakdown from disrupting the entire production pipeline. Maintenance planning must therefore balance unexpected failures against prematurely interrupted operating life. We present a Predictive Maintenance (PdM) framework combining Deep Learning (DL) sequence models and Simoultaneous Quantile Regression (SQR) for uncertainty-aware Remaining Useful Life (RUL) estimation and risk-aware maintenance decisions. Several architectures are compared on ion-milling data from the 2018 PHM Data Challenge (PHM18), including architectures based on State Space Models (SSM), using prediction and business metrics: Unexpected Breaks (UB), Unexploited Lifetime (UL), and a cost-weighted objective. Diagonal State Spaces (S4D) delivers the best Remaining Useful Life (RUL) estimates across quantiles and, relative to Preventive Maintenance (PvM) baselines, substantially lowers business cost by avoiding systematically early interventions. The results support uncertainty-aware, cost-sensitive maintenance planning in semiconductor production.

Davide Frizzo, Francesco Borsatti, Gian Antonio Susto

Department of Information Engineering, University of Padova

RUL Estimation Predictive Maintenance Semiconductor Manufacturing Uncertainty Quantification

Ask PDF

Abstract

Predictive Maintenance

Explore similar work

The price of decentralization in managing engineering systems through multi-agent reinforcement learning

Evidence-Driven Reasoning for Industrial Maintenance Using Heterogeneous Data

Uncertainty and Business-Aware Remaining Useful Life Estimation for Semiconductor Manufacturing