Probabilistic Model Checking

Probabilistic model checking is a formal verification technique used to analyze systems exhibiting stochastic behavior, aiming to determine the probability of satisfying given properties. Current research focuses on extending its application to complex scenarios, including multi-agent systems and the verification of reinforcement learning policies, often employing Markov Decision Processes (MDPs) and probabilistic computation tree logic (PCTL) for modeling and analysis. This methodology is increasingly important for ensuring the reliability and safety of autonomous systems, particularly in areas like robotics and AI, by providing rigorous guarantees about system behavior under uncertainty. Improved scalability and explainability are key challenges driving ongoing advancements.

Papers

October 23, 2024

1-2-3-Go! Policy Synthesis for Parameterized Markov Decision Processes via Decision-Tree Learning and Generalization
Muqsit Azeem, Debraj Chakraborty, Sudeep Kanav, Jan Kretinsky, Mohammadsadegh Mohagheghi, Stefanie Mohr, Maximilian Weininger
Strong Generalization Decision Tree Optimal Policy State Space Policy Synthesis Probabilistic Model Checking

March 27, 2024

Probabilistic Model Checking of Stochastic Reinforcement Learning Policies
Dennis Gross, Helge Spieker
Markov Decision Process Model Checking Reinforcement Learning Policy Computation Tree Logic Probabilistic Model Checking

August 5, 2023

Multi-Agent Verification and Control with Probabilistic Model Checking
David Parker
Multi Agent External Control Game Theory Stochastic Game Multi Agent Setting Rational Agent Probabilistic Model Checking

July 15, 2023

Probabilistic Black-Box Checking via Active MDP Learning
Junya Shijubo, Masaki Waga, Kohei Suenaga
Black Box Learning Automaton Stochastic Behavior Probabilistic Model Checking

May 26, 2023

MULTIGAIN 2.0: MDP controller synthesis for multiple mean-payoff, LTL and steady-state constraints
Severin Bals, Alexandros Evangelidis, Jan Křetínský, Jakob Waibel
Payoff Function Controller Synthesis Probabilistic Model Checking Probabilistic System

May 17, 2023

Explainable Multi-Agent Reinforcement Learning for Temporal Queries
Kayla Boggess, Sarit Kraus, Lu Feng
Multi Agent Reinforcement Learning Emergent Behavior Contrastive Explanation Multi Agent Behavior Probabilistic Model Checking Temporal Query

May 1, 2023

Probabilistic Formal Modelling to Uncover and Interpret Interaction Styles
Oana Andrei, Muffy Calder, Matthew Chalmers, Alistair Morrison
Unsupervised Algorithm Formal Model Probabilistic Model Checking Interaction Mode Probabilistic Temporal Logic

September 15, 2022

COOL-MC: A Comprehensive Tool for Reinforcement Learning and Model Checking
Dennis Gross, Nils Jansen, Sebastian Junges, Guillermo A. Perez
Reinforcement Learning Markov Decision Process Model Checking State of the Art Reinforcement Probabilistic Model Checking Formal Abstraction

May 11, 2022

Scalable Stochastic Parametric Verification with Stochastic Variational Smoothed Model Checking
Luca Bortolussi, Francesca Cairoli, Ginevra Carbone, Paolo Pulcini
Bayesian Inference Stochastic Variational Inference Probabilistic Model Checking Probabilistic Verification

January 10, 2022

Verified Probabilistic Policies for Deep Reinforcement Learning
Edoardo Bacci, David Parker
Deep Reinforcement Learning Reinforcement Learning Benchmark Probabilistic Guarantee Probabilistic Model Checking