Interval Markov Decision Process

Interval Markov Decision Processes (IMDPs) are mathematical models used to represent and solve decision-making problems under uncertainty, where transition probabilities are only known to lie within specified intervals rather than being precisely defined. Current research focuses on developing efficient algorithms for synthesizing robust and optimal policies within IMDP frameworks, particularly addressing challenges posed by continuous action spaces and limited data availability, often employing techniques like Gaussian processes and scenario approaches to quantify uncertainty. This work is significant for its applications in various fields requiring robust control under uncertainty, including robotics, autonomous systems, and safety-critical systems, enabling the design of controllers with provable performance guarantees even with incomplete knowledge of the system dynamics.

Papers

August 6, 2024

Certifiably Robust Policies for Uncertain Parametric Environments
Yannik Schnitzer, Alessandro Abate, David Parker
Robust Policy Stochastic Environment POMDP Policy Uncertain Dynamic Environment Interval Markov Decision Process

July 15, 2024

Data-Driven Abstractions via Binary-Tree Gaussian Processes for Formal Verification
Oliver Schön, Shammakh Naseer, Ben Wooding, Sadegh Soudjani
Gaussian Process Formal Verification Stochastic System Data Driven Abstraction Interval Markov Decision Process

March 22, 2023

Strategy Synthesis in Markov Decision Processes Under Limited Sampling Access
Christel Baier, Clemens Dubslaff, Patrick Wienhöft, Stefan J. Kiebel
Reinforcement Learning Markov Decision Process Limited Sample Strategy Synthesis Interval Markov Decision Process Reward Maximizing

January 4, 2023

Robust Control for Dynamical Systems With Non-Gaussian Noise via Formal Abstractions
Thom Badings, Licio Romao, Alessandro Abate, David Parker, Hasan A. Poonawala, Marielle Stoelinga, Nils Jansen
Dynamical System Robust Control Non Gaussian Transition Probability Probabilistic Guarantee Real World Control Stochastic Disturbance Formal Abstraction Interval Markov Decision Process

November 2, 2022

Interval Markov Decision Processes with Continuous Action-Spaces
Giannis Delimpaltadakis, Morteza Lahijanian, Manuel Mazo, Luca Laurenti
Markov Decision Process Action Space Interval Markov Decision Process

October 12, 2022

Probabilities Are Not Enough: Formal Controller Synthesis for Stochastic Dynamical Models with Epistemic Uncertainty
Thom Badings, Licio Romao, Alessandro Abate, Nils Jansen
Epistemic Uncertainty Aleatoric Uncertainty Relative Probability Controller Synthesis Interval Markov Decision Process Stochastic Dynamic Model