Planning or Learning: Reliability and Cost in Multi-Asset Maintenance

Xian Yeow Lee; Chandrasekar Venkatraman; Ahmed Farahat

cs.AISep 11, 2026

Planning or Learning: Reliability and Cost in Multi-Asset Maintenance

Authors: Xian Yeow Lee, Chandrasekar Venkatraman, Ahmed Farahat

Organizations: Industrial AI Lab, Hitachi America Ltd. Santa Clara, U.S.A.

Abstract

Industrial maintenance systems involve multiple interacting assets and shared resources, making it challenging to balance reliability and operational cost using a single decision framework. While recent work has focused on reinforcement learning (RL) for maintenance scheduling, direct comparisons with planning approaches under identical settings remain limited. In this work, we empirically compare planning and RL for multi-asset bearing maintenance using run-to-failure data. We examine how these methods behave when balancing preventive maintenance against tolerable failures across a range of failure penalty scenarios. We observed a consistent behavioral difference driven by objective formulation. Planning enforces reliability as a hard constraint and produces zero-failure policies whose total cost is largely insensitive to the magnitude of failure penalties. RL agents optimize expected cost and often trade off preventive maintenance against occasional failures as penalties vary, resulting in lower costs under low-penalty regimes but persistent non-zero failures even when penalties are high. We also investigate lightweight constraint mechanisms, including reward shaping and action masking, to encourage RL's reliability. From a practical perspective, planning may be more suitable when strict reliability is required and deployment horizons are short, whereas RL may provide cost-efficient policies when limited failures are acceptable and long-run operational efficiency is prioritized. Overall, this study clarifies the trade-offs between reliability and cost in multi-asset maintenance and suggests that planning and RL are complementary approaches. Beyond these findings, the controlled benchmark protocol itself that unifies environment, cost model, and evaluation across paradigms, offers a reusable template for comparing decision-making approaches in other maintenance settings.

Figures & tables

Figure 1: (a) Pipeline from raw vibration signals to health indicators to maintenance decisions. (b) Raw horizontal (blue) and vertical (red) accelerometer signals for a bearing at early stages (row 1), mid-life (row 2), and near-failure stages (row 3). (c) Composite RMS health indicator trajectories for all six bearings, each following a distinct degradation profile before failure.

Figure 2: Pareto tradeoff between cost and failures across all methods and failure penalties $c_{f}$ , where marker size $\propto c_{f}$ and filled = oracle dynamics, hollow = estimated dynamics. Planning and heuristic are anchored at zero failures; RL methods cluster in the high-failure, low-cost region regardless of $c_{f}$ , reflecting the hard-constraint vs. soft-cost formulation gap.

Oracle dynamics
Method	$c_{\text{f}}=100$		$c_{\text{f}}=500$		$c_{\text{f}}=1000$		$c_{\text{f}}=5000$		$c_{\text{f}}=10000$
Method	Cost	Failures	Cost	Failures	Cost	Failures	Cost	Failures	Cost	Failures
Threshold	231336	0.0	231336	0.0	231336	0.0	231336	0.0	231336	0.0
Dijkstra	49176	0.0	49176	0.0	49176	0.0	49176	0.0	49176	0.0
DQN	8134 $\pm$ 0	6.0 $\pm$ 0.0	10534 $\pm$ 0	6.0 $\pm$ 0.0	13534 $\pm$ 0	6.0 $\pm$ 0.0	37534 $\pm$ 0	6.0 $\pm$ 0.0	66856 $\pm$ 959	5.7 $\pm$ 0.5
PPO	15918 $\pm$ 2561	5.2 $\pm$ 0.4	32973 $\pm$ 3751	3.4 $\pm$ 0.4	25330 $\pm$ 4721	5.1 $\pm$ 0.8	47427 $\pm$ 3293	2.6 $\pm$ 1.4	54780 $\pm$ 2542	1.2 $\pm$ 0.8

Table 1: Main results for planning, heuristic, and vanilla RL methods across all failure penalty levels. Cost is the mean total episode cost; Failures is the mean number of bearing failures per episode. Standard deviation across three seeds is shown for RL methods. Dijkstra and Threshold are deterministic (no standard deviation).

Method	Training	Inference (per episode)
Threshold	—	28 s
Dijkstra	—	$\sim$ 90 min
DQN	$\sim$ 40 min	9 s
PPO	$\sim$ 68 min	10 s

Table 2: Training and inference runtimes for all methods.

Oracle dynamics
Method	$c_{\text{f}}=100$		$c_{\text{f}}=500$		$c_{\text{f}}=1000$		$c_{\text{f}}=5000$		$c_{\text{f}}=10000$
Method	Cost	Fail	Cost	Fail	Cost	Fail	Cost	Fail	Cost	Fail
DQN shaped	8134 $\pm$ 0	6.0 $\pm$ 0.0	10534 $\pm$ 0	6.0 $\pm$ 0.0	13534 $\pm$ 0	6.0 $\pm$ 0.0	37534 $\pm$ 0	6.0 $\pm$ 0.0	65744 $\pm$ 1373	4.7 $\pm$ 1.2
DQN masked	33703 $\pm$ 0	5.0 $\pm$ 0.0	35703 $\pm$ 0	5.0 $\pm$ 0.0	40928 $\pm$ 3853	5.0 $\pm$ 0.0	58577 $\pm$ 529	4.7 $\pm$ 0.5	83458 $\pm$ 361	4.3 $\pm$ 0.9
PPO shaped	20710 $\pm$ 5377	5.1 $\pm$ 0.8	30087 $\pm$ 4023	4.1 $\pm$ 0.2	28033 $\pm$ 3620	3.9 $\pm$ 0.7	47048 $\pm$ 6364	2.2 $\pm$ 2.1	55429 $\pm$ 4298	1.5 $\pm$ 1.7
PPO masked	56810 $\pm$ 8411	2.0 $\pm$ 1.6	56979 $\pm$ 8237	2.2 $\pm$ 0.8	54924 $\pm$ 1549	2.3 $\pm$ 0.9	58846 $\pm$ 2265	0.0 $\pm$ 0.0	73921 $\pm$ 15265	1.1 $\pm$ 1.4

Table 3: DQN and PPO performance with constraint mechanisms: reward shaping and safety mask, under oracle and estimated dynamics.

Oracle dynamics
Method	$c_{\text{f}}=100$		$c_{\text{f}}=500$		$c_{\text{f}}=1000$		$c_{\text{f}}=5000$		$c_{\text{f}}=10000$
Method	Cost	Fail	Cost	Fail	Cost	Fail	Cost	Fail	Cost	Fail
DQN cont. $\gamma=0.99$	8134 $\pm$ 0	6.0 $\pm$ 0.0	10534 $\pm$ 0	6.0 $\pm$ 0.0	13534 $\pm$ 0	6.0 $\pm$ 0.0	37534 $\pm$ 0	6.0 $\pm$ 0.0	66856 $\pm$ 959	5.7 $\pm$ 0.5
DQN cont. $\gamma=0.999$	8134 $\pm$ 0	6.0 $\pm$ 0.0	10534 $\pm$ 0	6.0 $\pm$ 0.0	13534 $\pm$ 0	6.0 $\pm$ 0.0	37534 $\pm$ 0	6.0 $\pm$ 0.0	66795 $\pm$ 578	5.3 $\pm$ 0.5
DQN disc. $\gamma=0.99$	8134 $\pm$ 0	6.0 $\pm$ 0.0	10534 $\pm$ 0	6.0 $\pm$ 0.0	13534 $\pm$ 0	6.0 $\pm$ 0.0	37534 $\pm$ 0	6.0 $\pm$ 0.0	65895 $\pm$ 1159	5.3 $\pm$ 0.5
DQN disc. $\gamma=0.999$	8134 $\pm$ 0	6.0 $\pm$ 0.0	10534 $\pm$ 0	6.0 $\pm$ 0.0	13534 $\pm$ 0	6.0 $\pm$ 0.0	37534 $\pm$ 0	6.0 $\pm$ 0.0	67534 $\pm$ 0	6.0 $\pm$ 0.0

Table 4: Ablation study: Effect of discount factor on DQN with continuous and discrete observations under oracle dynamics.

Predictive Maintenance

Predictive maintenance uses data and models to detect equipment faults, estimate degradation and remaining life, and plan timely repairs.

See all 118 papers

The price of decentralization in managing engineering systems through multi-agent reinforcement learning
Mar 12, 2026Prateek Bhustali, Pablo G. Morato, Konstantinos G. Papakonstantinou +1Decentralized MARL Multi-Agent Coordination
Faculty of Architecture and the Built Environment, Delft University of Technology, Delft, 2628 BL, The Netherlands · Engineering Risk Analysis Group, Technical University of Munich, Munich, 80333, Germany · Department of Civil & Environmental Engineering, The Pennsylvania State University, University Park, PA, 16802, USA
Evidence-Driven Reasoning for Industrial Maintenance Using Heterogeneous Data
Mar 9, 2026Fearghal O'Donncha, Nianjun Zhou, Natalia Martinez +4Evidence-Grounded Reasoning AI-Assisted Decision Making
IBM Research Europe, Ireland · IBM Research Yorktown, USA · Imperial College London, UK
Uncertainty and Business-Aware Remaining Useful Life Estimation for Semiconductor Manufacturing
Aug 26, 2026Davide Frizzo, Francesco Borsatti, Gian Antonio SustoRUL Estimation Predictive Maintenance
Department of Information Engineering, University of Padova

Mar 12, 2026J/K move · Enter open · S save

The price of decentralization in managing engineering systems through multi-agent reinforcement learning

Prateek Bhustali, Pablo G. Morato, Konstantinos G. Papakonstantinou, Charalampos P. Andriotis

Faculty of Architecture and the Built Environment, Delft University of Technology, Delft, 2628 BL, The Netherlands · Engineering Risk Analysis Group, Technical University of Munich, Munich, 80333, Germany · Department of Civil & Environmental Engineering, The Pennsylvania State University, University Park, PA, 16802, USA

Decentralized MARL Multi-Agent Coordination Cooperative MARL

Open paper AskPDF arXiv

Inspection and maintenance (I&M) planning involves sequential decision making under uncertainties and incomplete information, and can be modeled as a partially observable Markov decision process (POMDP). While single-agent deep reinforcement learning provides approximate solutions to POMDPs, it does not scale well in multi-component systems. Scalability can be achieved through multi-agent deep reinforcement learning (MADRL), which decentralizes decision-making across multiple agents, locally controlling individual components. However, this decentralization can induce cooperation pathologies that degrade the optimality of the learned policies. To examine these effects in I&M planning, we introduce a set of deteriorating systems in which redundancy is varied systematically. These benchmark environments are designed such that computation of centralized (near-)optimal policies remains tractable, enabling direct comparison of solution methods. We implement and benchmark a broad set of MADRL algorithms spanning fully centralized and decentralized training paradigms, from value-factorization to actor-critic methods. Our results show a clear effect of redundancy on coordination: MADRL algorithms achieve near-optimal performance in series-like settings, whereas increasing redundancy amplifies coordination challenges and can lead to optimality losses. Nonetheless, decentralized agents learn structured policies that consistently outperform optimized heuristic baselines, highlighting both the promise and current limitations of decentralized learning for scalable maintenance planning.

Abstract

Predictive Maintenance

Explore similar work

The price of decentralization in managing engineering systems through multi-agent reinforcement learning

Evidence-Driven Reasoning for Industrial Maintenance Using Heterogeneous Data

Uncertainty and Business-Aware Remaining Useful Life Estimation for Semiconductor Manufacturing