Risk Averse Reinforcement Learning

Risk-averse reinforcement learning (RL) focuses on developing algorithms that optimize decision-making in uncertain environments while explicitly managing risk, rather than solely maximizing expected reward. Current research emphasizes the use of various risk measures, such as Conditional Value at Risk (CVaR), Entropic Value at Risk (EVaR), and Gini deviation, within different RL frameworks including Q-learning, policy gradient methods, and distributional RL, often incorporating techniques like robust optimization and dynamic programming. This field is significant because it enables the development of safer and more reliable AI agents for applications where risk mitigation is crucial, such as autonomous driving, finance, and robotics.

Papers

September 25, 2024

Risk-averse learning with delayed feedback
Siyi Wang, Zifan Wang, Karl Henrik Johansson, Sandra Hirche
Sublinear Regret Delayed Feedback Risk Averse Risk Averse Reinforcement Learning

August 30, 2024

Stationary Policies are Optimal in Risk-averse Total-reward MDPs with EVaR
Xihong Su, Marek Petrik, Julien Grand-Clément
Optimal Policy Dynamic Programming Risk Averse Risk Averse Reinforcement Learning Stationary Policy Entropic Risk

May 2, 2024

Robust Risk-Sensitive Reinforcement Learning with Conditional Value-at-Risk
Xinyi Ni, Lifeng Lai
Markov Decision Process Risk Sensitive Reinforcement Learning Value at Risk Risk Averse Reinforcement Learning Mean CVaR

April 3, 2024

Risk-averse Learning with Non-Stationary Distributions
Siyi Wang, Zifan Wang, Xinlei Yi, Michael M. Zavlanos, Karl H. Johansson, Sandra Hirche
Online Optimization Sub Linear Regret Risk Averse Risk Averse Reinforcement Learning

February 28, 2024

Provable Risk-Sensitive Distributional Reinforcement Learning with General Function Approximation
Yu Chen, Xiangcheng Zhang, Siwei Wang, Longbo Huang
Reinforcement Learning Distributional Reinforcement Learning Risk Sensitive Reinforcement Learning Risk Measure Risk Averse Reinforcement Learning

July 17, 2023

An Alternative to Variance: Gini Deviation for Risk-averse Policy Gradient
Yudong Luo, Guiliang Liu, Pascal Poupart, Yangchen Pan
Policy Learning Variance Information Policy Gradient Algorithm Risk Averse Reinforcement Learning Risk Averse Policy Gini Index

May 18, 2023

Bayesian Risk-Averse Q-Learning with Streaming Observations
Yuhao Wang, Enlu Zhou
Optimal Policy Model Uncertainty Robust Reinforcement Learning Risk Averse Reinforcement Learning

February 27, 2023

Distributional Method for Risk Averse Reinforcement Learning
Ziteng Cheng, Sebastian Jaimungal, Nick Martin
Optimal Policy Distributional Learning Risk Averse Reinforcement Learning Competing Risk Risk Averse Policy

January 14, 2023

Risk-Averse Reinforcement Learning via Dynamic Time-Consistent Risk Measures
Xian Yu, Siqian Shen
Traditional Reinforcement Learning Infinite Horizon Risk Measure Risk Averse Reinforcement Learning

November 4, 2022

Path Planning Using Wassertein Distributionally Robust Deep Q-learning
Cem Alpturk, Venkatraman Renganathan
Deep Reinforcement Learning Q Learning Path Planning Risk Aware Risk Averse Reinforcement Learning

September 9, 2022

RASR: Risk-Averse Soft-Robust MDPs with EVaR and Entropic Risk
Jia Lin Hau, Marek Petrik, Mohammad Ghavamzadeh, Reazul Russel
Safe Reinforcement Learning Infinite Horizon Risk Aversion Risk Averse Reinforcement Learning Entropic Risk

September 6, 2022

A Zeroth-Order Momentum Method for Risk-Averse Online Convex Games
Zifan Wang, Yi Shen, Zachary I. Bell, Scott Nivison, Michael M. Zavlanos, Karl H. Johansson
Bandit Feedback Sub Linear Regret Risk Averse Reinforcement Learning Cournot Game Convex Game

June 15, 2022

Mean-Semivariance Policy Optimization via Risk-Averse Reinforcement Learning
Xiaoteng Ma, Shuai Ma, Li Xia, Qianchuan Zhao
Risk Averse Reinforcement Learning Approximate Transformation Invariance

May 10, 2022

Efficient Risk-Averse Reinforcement Learning
Ido Greenberg, Yinlam Chow, Mohammad Ghavamzadeh, Shie Mannor
Risk Aversion Risk Averse Reinforcement Learning Softmax Cross Entropy Risk Averse Policy

April 20, 2022

SAAC: Safe Reinforcement Learning as an Adversarial Game of Actor-Critics
Yannis Flet-Berliac, Debabrota Basu
Reinforcement Learning Safe Reinforcement Learning Soft Actor Critic Maximum Entropy Reinforcement Learning Adversarial Game Risk Averse Reinforcement Learning

March 16, 2022

How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies
Lukas M. Schmidt, Sebastian Rietsch, Axel Plinge, Bjoern M. Eskofier, Christopher Mutschler
Autonomous Driving Autonomous Vehicle High Efficiency Safe Reinforcement Learning High Quality Risk Description Driving Policy Risk Averse Reinforcement Learning Vanilla Reinforcement Learning