Robust Reinforcement Learning

Robust reinforcement learning (RL) focuses on developing RL agents that perform well even when faced with uncertainties in the environment, such as noisy observations, model mismatches, or adversarial attacks. Current research emphasizes techniques like adversarial training, distributionally robust optimization, and the use of pessimistic models to improve robustness, often incorporating actor-critic algorithms, model-based approaches, and Lipschitz-constrained policy networks. This field is crucial for deploying RL agents in real-world settings where perfect knowledge of the environment is unrealistic, impacting areas like robotics, autonomous systems, and finance by enabling safer and more reliable AI.

Papers

April 4, 2024

Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithm
Miao Lu, Han Zhong, Tong Zhang, Jose Blanchet
Robust Policy Robust Reinforcement Learning Hardness Result Distributionally Robust Near Optimal Algorithm

March 27, 2024

Safe and Robust Reinforcement Learning: Principles and Practice
Taku Yamagata, Raul Santos-Rodriguez
Reinforcement Learning Practice Mode General Principle Robust Reinforcement Learning Reinforcement Learning Application

March 21, 2024

Robust Model Based Reinforcement Learning Using $\mathcal{L}_1$ Adaptive Control
Minjun Sung, Sambhu H. Karumanchi, Aditya Gahlawat, Naira Hovakimyan
Model Based Reinforcement Learning Robust Reinforcement Learning Adaptive Control Control Affine

March 6, 2024

Belief-Enriched Pessimistic Q-Learning against Adversarial State Perturbations
Xiaolin Sun, Zizhan Zheng
Reinforcement Learning Q Learning Reinforcement Learning Agent Robust Reinforcement Learning Belief State Diffusion Purification

February 20, 2024

Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies
Xiangyu Liu, Chenghao Deng, Yanchao Sun, Yongyuan Liang, Furong Huang
Reinforcement Learning Sublinear Regret Robust Reinforcement Learning Diverse Policy Adversarial Bandit Adaptive Defense State Adversarial

February 14, 2024

Towards Robust Model-Based Reinforcement Learning Against Adversarial Corruption
Chenlu Ye, Jiafan He, Quanquan Gu, Tong Zhang
Model Based Reinforcement Learning Robust Reinforcement Learning Adversarial Corruption

February 9, 2024

Corruption Robust Offline Reinforcement Learning with Human Feedback
Debmalya Mandal, Andi Nika, Parameswaran Kamalaruban, Adish Singla, Goran Radanović
Reinforcement Learning Human Feedback Reinforcement Learning From Human Feedback Robust Reinforcement Learning

January 9, 2024

Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning
Ding Chen, Peixi Peng, Tiejun Huang, Yonghong Tian
Reinforcement Learning Deep Reinforcement Learning Neuromorphic Hardware Robust Reinforcement Learning Neural Population

December 14, 2023

Improve Robustness of Reinforcement Learning against Observation Perturbations via $l_\infty$ Lipschitz Policy Networks
Buqing Nie, Jingtian Ji, Yangqing Fu, Yue Gao
Reinforcement Learning Deep Reinforcement Learning Native Robustness Robust Reinforcement Learning Lipschitz Network

November 15, 2023

On the Foundation of Distributionally Robust Reinforcement Learning
Shengbo Wang, Nian Si, Jose Blanchet, Zhengyuan Zhou
Optimal Policy Robust Policy Flawed Foundation Robust Reinforcement Learning Distributionally Robust Hybrid Attack

October 23, 2023

Corruption-Robust Offline Reinforcement Learning with General Function Approximation
Chenlu Ye, Rui Yang, Quanquan Gu, Tong Zhang
Offline Reinforcement Learning Robust Reinforcement Learning Corruption Robustness Corruption Robust Algorithm

October 6, 2023

Adjustable Robust Reinforcement Learning for Online 3D Bin Packing
Yuxin Pan, Yize Chen, Fangzhen Lin
Worst Case Robust Reinforcement Learning Online 3D Bin Packing

September 26, 2023

Gray-box Adversarial Attack of Deep Reinforcement Learning-based Trading Agents
Foozhan Ataiefard, Hadi Hemmati
Deep Reinforcement Learning Native Robustness Deep Reinforcement Robust Reinforcement Learning Deep RL Adversary Agent Gray Box

September 13, 2023

Safe Reinforcement Learning with Dual Robustness
Zeyang Li, Chuxiong Hu, Yunan Wang, Yujie Yang, Shengbo Eben Li
Reinforcement Learning Deep Reinforcement Learning Safe Reinforcement Learning Robust Reinforcement Learning Policy Iteration Double Robustness

September 5, 2023

Distributionally Robust Model-based Reinforcement Learning with Large State Spaces
Shyam Sundhar Ramesh, Pier Giuseppe Sessa, Yifan Hu, Andreas Krause, Ilija Bogunovic
Reinforcement Learning State Space Robust Reinforcement Learning Distributionally Robust Optimal Robust Policy

July 22, 2023

Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations
Yongyuan Liang, Yanchao Sun, Ruijie Zheng, Xiangyu Liu, Benjamin Eysenbach, Tuomas Sandholm, Furong Huang, Stephen McAleer
Reinforcement Learning Robust Reinforcement Learning Additive Perturbation

July 17, 2023

Natural Actor-Critic for Robust Reinforcement Learning with Function Approximation
Ruida Zhou, Tao Liu, Min Cheng, Dileep Kalathil, P. R. Kumar, Chao Tian
Robust Reinforcement Learning Function Approximation Optimal Robust Policy Robust Policy Evaluation

July 15, 2023

July 3, 2023

Enhancing the Robustness of QMIX against State-adversarial Attacks
Weiran Guo, Guanjun Liu, Ziyuan Zhou, Ling Wang, Jiacun Wang
Deep Reinforcement Learning Native Robustness Multi Agent Reinforcement Learning Robust Reinforcement Learning Soft QMIX State Adversarial

Robust Reinforcement Learning

Papers

Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithm

Safe and Robust Reinforcement Learning: Principles and Practice

Robust Model Based Reinforcement Learning Using $\mathcal{L}_1$ Adaptive Control

Belief-Enriched Pessimistic Q-Learning against Adversarial State Perturbations

Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies

Towards Robust Model-Based Reinforcement Learning Against Adversarial Corruption

Corruption Robust Offline Reinforcement Learning with Human Feedback

Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning

Improve Robustness of Reinforcement Learning against Observation Perturbations via $l_\infty$ Lipschitz Policy Networks

On the Foundation of Distributionally Robust Reinforcement Learning

Corruption-Robust Offline Reinforcement Learning with General Function Approximation

Adjustable Robust Reinforcement Learning for Online 3D Bin Packing

Gray-box Adversarial Attack of Deep Reinforcement Learning-based Trading Agents

Safe Reinforcement Learning with Dual Robustness

Distributionally Robust Model-based Reinforcement Learning with Large State Spaces

Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations

Natural Actor-Critic for Robust Reinforcement Learning with Function Approximation

Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlation

Efficient Action Robust Reinforcement Learning with Probabilistic Policy Execution Uncertainty

Enhancing the Robustness of QMIX against State-adversarial Attacks