Robust Reinforcement Learning

Robust reinforcement learning (RL) focuses on developing RL agents that perform well even when faced with uncertainties in the environment, such as noisy observations, model mismatches, or adversarial attacks. Current research emphasizes techniques like adversarial training, distributionally robust optimization, and the use of pessimistic models to improve robustness, often incorporating actor-critic algorithms, model-based approaches, and Lipschitz-constrained policy networks. This field is crucial for deploying RL agents in real-world settings where perfect knowledge of the environment is unrealistic, impacting areas like robotics, autonomous systems, and finance by enabling safer and more reliable AI.

Papers

May 30, 2023

Robust Reinforcement Learning Objectives for Sequential Recommender Systems
Melissa Mozifian, Tristan Sylvain, Dave Evans, Lili Meng
Reinforcement Learning Recommender System Sequential Recommendation Robust Reinforcement Learning Sequential Recommender System Sequential Model Attention Based Sequential Recommendation

May 18, 2023

Bayesian Risk-Averse Q-Learning with Streaming Observations
Yuhao Wang, Enlu Zhou
Optimal Policy Model Uncertainty Robust Reinforcement Learning Risk Averse Reinforcement Learning

May 11, 2023

On Practical Robust Reinforcement Learning: Practical Uncertainty Set and Double-Agent Algorithm
Ukjo Hwang, Songnam Hong
Markov Decision Process Q Learning Robust Policy Robust Reinforcement Learning Uncertainty Set

April 2, 2023

Risk-Sensitive and Robust Model-Based Reinforcement Learning and Planning
Marc Rigter
Reinforcement Learning Task Planning Sequential Decision Risk Sensitive Robust Reinforcement Learning Sequential Decision Making Problem

March 23, 2023

Connected Superlevel Set in (Deep) Reinforcement Learning and its Application to Minimax Theorems
Sihan Zeng, Thinh T. Doan, Justin Romberg
Reinforcement Learning Application Proficiency Policy Optimization Robust Reinforcement Learning Level Set Minimax Theorem

March 9, 2023

Variance-aware robust reinforcement learning with linear function approximation under heavy-tailed rewards
Xiang Li, Qiang Sun
Linear Bandit Linear Function Approximation Robust Reinforcement Learning Huber Loss Heavy Tailed Reward Variance Aware Variance Dependent Regret Online Sequential Decision Making

March 6, 2023

Reinforcement Learning Based Self-play and State Stacking Techniques for Noisy Air Combat Environment
Ahmet Semih Tasbas, Safa Onur Sahin, Nazim Kemal Ure
Reinforcement Learning Robust Reinforcement Learning State Abstraction Air Combat Unmanned Combat Self Play Reinforcement Learning

March 5, 2023

Improved Sample Complexity Bounds for Distributionally Robust Reinforcement Learning
Zaiyan Xu, Kishan Panaganti, Dileep Kalathil
Robust Reinforcement Learning Distributionally Robust Sample Complexity Bound Uncertainty Set Wasserstein Uncertainty

February 10, 2023

Towards Minimax Optimality of Model-based Robust Reinforcement Learning
Pierre Clavier, Erwan Le Pennec, Matthieu Geist
Markov Decision Process Sample Complexity Optimal Policy Robust Reinforcement Learning Minimax Optimality MDP Model

January 26, 2023

Certifiably Robust Reinforcement Learning through Model-Based Abstract Interpretation
Chenxi Yang, Greg Anderson, Swarat Chaudhuri
Reinforcement Learning Adversarial Robustness Robust Reinforcement Learning Abstract Interpretation

October 27, 2022

Characterising the Robustness of Reinforcement Learning for Continuous Control using Disturbance Injection
Catherine R. Glossop, Jacopo Panerati, Amrit Krishnan, Zhaocong Yuan, Angela P. Schoellig
Reinforcement Learning Native Robustness Action Space Continuous Control Robust Reinforcement Learning Disturbance Injection

October 21, 2022

Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variables
Mengdi Xu, Peide Huang, Yaru Niu, Visak Kumar, Jielin Qiu, Chao Fang, Kuan-Hui Lee, Xuewei Qi, Henry Lam, Bo Li, Ding Zhao
Robust Reinforcement Learning Multi Task Reinforcement Learning Distributional Robustness Hierarchical Latent Hierarchical MDP Distributionally Robust Markov Decision Process

October 12, 2022

Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learning
Yongyuan Liang, Yanchao Sun, Ruijie Zheng, Furong Huang
Reinforcement Learning Deep Reinforcement Learning Robust Reinforcement Learning Efficient Adversarial Training

September 23, 2022

Quantification before Selection: Active Dynamics Preference for Robust Reinforcement Learning
Kang Xu, Yan Ma, Wei Li
View Selection Robust Policy Robust Reinforcement Learning Domain Randomization Dynamic Mismatch Active Preference

September 17, 2022

Robust Reinforcement Learning Algorithm for Vision-based Ship Landing of UAVs
Vishnu Saj, Bochan Lee, Dileep Kalathil, Moble Benedict
Unmanned Aerial Vehicle Robust Reinforcement Learning Monocular Camera Ship Landing

August 29, 2022

Understanding the Limits of Poisoning Attacks in Episodic Reinforcement Learning
Anshuka Rangi, Haifeng Xu, Long Tran-Thanh, Massimo Franceschetti
Reinforcement Learning Continuum Limit Poisoning Attack Robust Reinforcement Learning Episodic Reinforcement Learning Reward Hacking

August 10, 2022

Robust Reinforcement Learning using Offline Data
Kishan Panaganti, Zaiyan Xu, Dileep Kalathil, Mohammad Ghavamzadeh
Robust Reinforcement Learning Offline Data Optimal Robust Policy

July 19, 2022

Feasible Adversarial Robust Reinforcement Learning for Underspecified Environments
JB Lanier, Stephen McAleer, Pierre Baldi, Roy Fox
Robust Policy Robust Reinforcement Learning Adversarial Reinforcement Learning Adversarial Parameter

July 18, 2022

MAD for Robust Reinforcement Learning in Machine Translation
Domenic Donato, Lei Yu, Wang Ling, Chris Dyer
Machine Translation Proximal Policy Optimization Robust Reinforcement Learning Machine Translation Model Policy Gradient Algorithm Reward Model Training Mad Icp

July 5, 2022

Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularization
Yuan Zhang, Jianhong Wang, Joschka Boedecker
Reinforcement Learning Regularization Model Robust Policy Robust Reinforcement Learning Continuous Control Task Real World Reinforcement Learning

Robust Reinforcement Learning

Papers

Robust Reinforcement Learning Objectives for Sequential Recommender Systems

Bayesian Risk-Averse Q-Learning with Streaming Observations

On Practical Robust Reinforcement Learning: Practical Uncertainty Set and Double-Agent Algorithm

Risk-Sensitive and Robust Model-Based Reinforcement Learning and Planning

Connected Superlevel Set in (Deep) Reinforcement Learning and its Application to Minimax Theorems

Variance-aware robust reinforcement learning with linear function approximation under heavy-tailed rewards

Reinforcement Learning Based Self-play and State Stacking Techniques for Noisy Air Combat Environment

Improved Sample Complexity Bounds for Distributionally Robust Reinforcement Learning

Towards Minimax Optimality of Model-based Robust Reinforcement Learning

Certifiably Robust Reinforcement Learning through Model-Based Abstract Interpretation

Characterising the Robustness of Reinforcement Learning for Continuous Control using Disturbance Injection

Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variables

Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learning

Quantification before Selection: Active Dynamics Preference for Robust Reinforcement Learning

Robust Reinforcement Learning Algorithm for Vision-based Ship Landing of UAVs

Understanding the Limits of Poisoning Attacks in Episodic Reinforcement Learning

Robust Reinforcement Learning using Offline Data

Feasible Adversarial Robust Reinforcement Learning for Underspecified Environments

MAD for Robust Reinforcement Learning in Machine Translation

Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularization