DEPLOY-RL: Active Boundary Discovery and Conservative Certification for Deployable Reinforcement Learning in Safety-Critical Continuous Processes
Organizations: Hanbat National University, Republic of Korea · Korea Advanced Institute of Science and Technology, Republic of Korea
Abstract
Reinforcement learning (RL) policies often outperform classical controllers in simulation, yet rarely reach production in safety-critical processes. The barrier is that there is no principled way to answer "Is this policy safe to deploy?" with statistical guarantees. We introduce DEPLOY-RL, a post-training certification framework built on one key insight: deployment certification requires discovering where failures occur (boundary discovery), not measuring how much everywhere (uniform reconstruction). Our contributions: (1) a contract-coupled acquisition function that concentrates sampling on certification-critical boundaries, achieving approximately 2x sample efficiency with a semi-empirical ambiguity reduction bound (domain-calibrated convergence guarantee); (2) conformal risk control providing finite-sample false-go guarantees (<= alpha) under explicit deployment contracts; (3) a three-way decision framework (Deploy/No-Deploy/Abstain) with fail-safe PID/MPC fallback. In simulations on papermaking (industrial digital twin) and Tennessee Eastman (public benchmark), DEPLOY-RL achieves 4.4% false-go rate (vs. 8.6% for the best baseline) while retaining 88.6% policy coverage, the only method achieving <5% false-go with >85% coverage among 14 baselines under our evaluation protocol.
Figures & tables
| Cat. | Method | FG | Cov. | Rec. | AUC |
|---|---|---|---|---|---|
| Ctrl | PID | 22.3 1.6 | 94.1 0.9 | .46 | .68 |
| MPC | 18.7 1.4 | 93.2 1.0 | .42 | .72 | |
| Robust MPC | 11.2 1.2 | 71.4 1.8 | .48 | .78 | |
| CC-MPC | 14.6 1.3 | 82.3 1.4 | .44 | .76 | |
| Safe | PPO | 29.4 1.8 | 97.2 0.7 | .32 | .67 |
| CPO | 12.8 1.3 | 89.4 1.2 | .35 | .81 |
| Method | FG@C | CC | FB |
|---|---|---|---|
| Robust MPC | 14.2 1.4 | — | 28.6 2.1 |
| CPO | 11.8 1.2 | — | 10.6 1.4 |
| Recovery RL | 9.8 1.1 | — | 14.8 1.6 |
| Deep Ensemble Gate | 8.2 1.0 | 320 24 | 20.2 1.8 |
| AST + Gate | 8.8 1.1 | 280 22 | 18.6 1.6 |
| Straddle + Gate | 7.2 0.9 | 185 20 | 16.4 1.5 |
Safe RL
Safe reinforcement learning studies how agents can learn and act while respecting safety constraints and limiting harmful outcomes.
See all 239 papers