Robust Markov Decision

Robust Markov Decision Processes (RMDPs) aim to design optimal policies for systems with uncertain transition dynamics, ensuring performance even under worst-case scenarios. Current research focuses on developing efficient algorithms, such as Q-learning variants and policy gradient methods, that can handle various types of uncertainty sets, including those defined by Wasserstein distance or divergence measures. These advancements are crucial for improving the reliability and safety of reinforcement learning agents in real-world applications where model misspecification is inevitable, impacting fields like robotics and autonomous systems. The development of finite sample complexity bounds for these algorithms is a key area of progress, providing theoretical guarantees on their performance.

Papers

July 5, 2024

Robust Q-Learning for finite ambiguity sets
Cécile Decker, Julian Sester
Q Learning Probability Measure Ambiguity Set Q$ Learning Algorithm Robust Markov Decision

April 24, 2023

Model-Free Learning and Optimal Policy Design in Multi-Agent MDPs Under Probabilistic Agent Dropout
Carmel Fiscko, Soummya Kar, Bruno Sinopoli
Multi Agent Optimal Policy Structured Dropout Robust Policy Free Learning Dropout Method Robust Markov Decision Process Robust Markov Decision

February 26, 2023

A Finite Sample Complexity Bound for Distributionally Robust Q-learning
Shengbo Wang, Nian Si, Jose Blanchet, Zhengyuan Zhou
Sample Complexity Q Learning Model Free Reinforcement Learning Q$ Learning Robust Markov Decision

September 30, 2022

Robust $Q$-learning Algorithm for Markov Decision Processes under Wasserstein Uncertainty
Ariel Neufeld, Julian Sester
Practical Algorithm Markov Decision Process Robust Version Distributional Robustness Stochastic Optimal Control Wasserstein Uncertainty Q$ Learning Algorithm Robust Markov Decision

September 21, 2022

First-order Policy Optimization for Robust Markov Decision Process
Yan Li, Guanghui Lan, Tuo Zhao
Robust Policy Policy Mirror Descent First Order Policy Robust Markov Decision

December 2, 2021

Sample Complexity of Robust Reinforcement Learning with a Generative Model
Kishan Panaganti, Dileep Kalathil
Generative Modeling Sample Complexity Robust Policy Robust Reinforcement Learning Optimal Robust Policy Robust Markov Decision