Shields to Guarantee Probabilistic Safety in MDPs

Linus Heck; Filip Macák; Roman Andriushchenko; Milan Češka; Sebastian Junges

cs.LOMay 11, 2026

Shields to Guarantee Probabilistic Safety in MDPs

Authors: Linus Heck, Filip Macák, Roman Andriushchenko, Milan Češka, Sebastian Junges

Organizations: Radboud University, Nijmegen, the Netherlands · Brno University of Technology, Czechia

Abstract

Shielding is a prominent model-based technique to ensure safety of autonomous agents. Classical shielding aims to ensure that nothing bad ever happens and comes with strong guarantees about safety and maximal permissiveness. However, shielding systems for probabilistic safety, where something bad is allowed to happen with an acceptable probability, has proven to be more intricate. This paper presents a formal framework that conservatively extends classical shields to probabilistic safety. In this framework, we (i) demonstrate the impossibility of preserving the strong guarantees on safety and permissiveness, (ii) provide natural shields with weaker guarantees, and (iii) introduce offline and online shield constructions ensuring strong safety guarantees. The empirical evaluation highlights the practical advantages of the new shields, as well as their computational feasibility.

Explore similar work

Robust Shielding for Safe Reinforcement Learning
May 29, 2026Edwin Hamel-De le Court, Thom Badings, Alessandro Abate +2Shielding Markov Decision Processes
Interval POMDP Shielding for Imperfect-Perception Agents
Apr 22, 2026William Scarbro, Ravi MangalShielding Partially Observable Markov Decision Process
Robust Probabilistic Shielding for Safe Offline Reinforcement Learning
May 11, 2026Maris F. L. Galesloot, Thomas Rhemrev, Nils JansenOffline Reinforcement Learning Shielding

May 29, 2026 · cs.AIJ/K move · Enter open · S save

Robust Shielding for Safe Reinforcement Learning

Edwin Hamel-De le Court, Thom Badings, Alessandro Abate, Francesco Belardinelli+1

Department of Computer Science, University of Manchester, United Kingdom · Faculty of Computer Science & DSME, RWTH Aachen University, Germany · Department of Computer Science, University of Oxford, United Kingdom · Department of Computing, Imperial College London, United Kingdom

Shielding is an effective approach to formally guarantee the safety of reinforcement learning agents in Markov decision processes (MDPs). However, existing shielding techniques typically assume knowledge of the safety-relevant transition dynamics - a requirement that is seldom met in practice. To address this limitation, we introduce a novel shielding framework for robust MDPs (RMDPs), i.e., MDPs with sets of transition probabilities. We define safety as the satisfaction of a linear temporal logic (LTL) formula with a certain threshold probability under the worst-case transition probabilities of the RMDP. We prove that our shielding framework is both sound and optimal for the RMDP: every policy admissible by the shield is safe, and conversely, every safe RMDP policy is admissible by the shield. We combine our approach with existing sampling methods for learning transition probabilities of MDPs with probably approximately correct (PAC) guarantees. This combination enables the construction of shields for MDPs that, with high confidence, guarantee safety while remaining minimally restrictive. Our experiments show that our shields for learned RMDPs guarantee safety in unknown MDPs while recovering strong expected return as the number of samples increases.

Shielding Markov Decision Processes Deep Reinforcement Learning Safety Constraints Linear Temporal Logics Transitions

Open paper Ask about it PDF

Abstract

Explore similar work

Robust Shielding for Safe Reinforcement Learning

Interval POMDP Shielding for Imperfect-Perception Agents

Robust Probabilistic Shielding for Safe Offline Reinforcement Learning