Risk-Sensitive RL

RL: Reinforcement Learning

Risk-sensitive reinforcement learning studies how agents can make sequential decisions that account for rare failures and undesirable outcomes, not just average reward. Common approaches optimize CVaR, quantile, variance, expectile, or exponential-utility objectives, often using distributional critics, policy gradients, Q-learning, or model-based planning. Safety-constrained methods add tools such as shields, reachability checks, and robust uncertainty sets, with applications ranging from robot navigation and autonomous driving to finance and language-model alignment. The field is developing convergence and sample-complexity guarantees alongside practical methods, while a central tension remains how to estimate tail risk reliably from limited data without making policies excessively conservative.

Latest papers

85 papers18 papers in the last four weeks, up 38% on the four weeks before. 0.1% of all new papers.

Deep catastrophic-risk-aware reinforcement learning
Oct 19, 2026Carlos Pérez Octavio Mendoza, Frédéric GodinRisk-Sensitive RL Conditional Value-at-Risk
Concordia University, Department of Mathematics and Statistics, Montréal, Canada · Quantact Laboratory, Centre de Recherches Mathématiques, Montréal, Canada
Reward Objectives for Robust (PO)MDPs: A Law-Preserving Equivalence with (Partially Observable) Games
Oct 19, 2026Durgam Latha, Dion Reji, Dorde Zikelic +1Partially Observable Stochastic Games Robust Partially Observable Markov Decision Processes
Indian Institute of Technology Bombay Mumbai, India · Nanyang Technological University Singapore, Singapore
Provably Efficient Risk-Sensitive Offline Reinforcement Learning via Optimized Certainty Equivalents
Oct 15, 2026Hanzhen Wang, Boxiang Lyu, Zhongxiang Dai +3Reinforcement Learning Risk-Sensitive RL
City University of Hong Kong Hong Kong, China · University of Chicago Chicago, IL, USA · The Chinese University of Hong Kong, Shenzhen Shenzhen, Guangdong, China +1
Stein Variational Distributional Reinforcement Learning: Score-Guided Particles for Multivariate Return Distributions
Oct 13, 2026Manuel Haußmann, Melih KandemirDistributional RL Risk-Sensitive RL
University of Southern Denmark
Bridging Risk Approximation Gaps in Model Predictive Task Sampling via In-Context Modeling
Oct 10, 2026Jiarong Wen, Qi Tao, Kaiyu Zhang +5Sample-Efficient RL RL Post-Training
College of Science, National University of Defense Technology · Department of Automation, Tsinghua University
Equal-information evaluation of credit assignment in modular financial reinforcement learning
Oct 10, 2026Gennadiy Chistiakov, Yury YanovichCredit Assignment in RL Reinforcement Learning
Independent Researcher, Bangkok, 10100, Thailand · Skolkovo Institute of Science and Technology, Moscow, 121205, Russia
Credal Machine Learning for Risk-Averse Decision Making
Oct 8, 2026Timo Löhr, Paul Hofman, Maximilian Muschalik +1Decision-Making under Uncertainty Conditional Value-at-Risk
LMU Munich, MCML · LMU Munich, MCML, DFKI
Safe Meta-Policy Design with Risk Control
Oct 7, 2026Wenbin Zhou, Michael Lingzhi Li, Shixiang ZhuSafe RL Risk-Sensitive RL
Heinz College of Information Systems and Public Policy, Carnegie Mellon University · Technology and Operations Management, Harvard Business School
Safe on Average, Unsafe in the Tail: When Is the Episodic-Cost Tail Controllable?
Oct 7, 2026Samuel Tetteh, Cody FlemingSafe RL Constrained RL
Iowa State University Ames, Iowa, USA
Variance-Averse $n$ -Step Offline Reinforcement Learning for Sparse Long-Horizon Environments
Oct 6, 2026Guhyeon Kang, Minhae KwonDistributional RL Risk-Sensitive RL
Department of Electrical and Computer Engineering Sungkyunkwan University, Republic of Korea
Risk-Sensitive Crowd Navigation with Adaptive Ellipsoidal Conformal Prediction
Oct 5, 2026Ruihan A. Li, Ziyao Guo, Yingying LiDistribution Shift Robustness Robot Navigation
Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL
Near-Optimal Sample Complexity for Recursive Entropic Risk Reinforcement Learning with a Generative Model
Oct 3, 2026Amirparsa Bahrami, Oliver Mortensen, Mohammad Sadegh TalebiRisk-Sensitive RL Sample-Efficient RL
School of Electrical Engineering, Sharif University of Technology · Department of Computer Science, University of Copenhagen
FSPO: Policy-Consistent Risk and Pareto-Feasible Control for Budgeted LLM RL Post-Training
Oct 2, 2026Miaobo Hu, Shuhao Hu, Xiaobo Guo +3Reinforcement Learning RL Post-Training
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
PPO-HRAP: Proximal Policy Optimization with a Hybrid Regime-Aware Policy for Risk-Controlled Trading
Oct 1, 2026Duong Hien Chi Kien, Thanh Trung HuynhReinforcement Learning Risk-Sensitive RL
College of Engineering and Computer Science, VinUniversity Ha Noi, Viet Nam
Exploring More, Reasoning Better: Stepwise Risk-Sensitive GRPO for Diffusion Language Models
Sep 30, 2026Yue Yu, Bowen Zuo, David Crandall +2RL for Language Model Reasoning Group Relative Policy Optimization
Indiana University · University of California, Riverside
Robust Risk-Sensitive Reinforcement Learning from Corrupted Human Feedback
Sep 30, 2026Xinyi Ni, Lifeng LaiRisk-Sensitive RL Robust RL
Department of Electrical and Computer Engineering, University of California, Davis, CA 95616, USA
Tail-Influence Sampling for CVaR Policy Evaluation
Sep 29, 2026Pauline Bourigault, Xiaotong Ji, Matthieu Zimmer +2Policy Evaluation Adaptive Sampling
Imperial College London · Huawei Noah’s Ark Lab · UCL Centre for AI
Character Training for Risk-Averse Agents
Sep 29, 2026Arav Dhoot, Punya Syon Pandey, Jamie Johnson +3Risk-Sensitive RL AI Agent Safety
UK AI Security Institute · Resolution
RSD-Poker: Structure-Adaptive and Shift-Robust Risk-Utility Certification for Residual Policies in Imperfect-Information Games
Sep 27, 2026Miaobo Hu, Shuhao Hu, Xiaobo Guo +4Distribution Shift Robustness Imperfect-Information Games
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
When Should a Human Take Back Control? Optimal Delegation under Turbulent AI Risk
Sep 25, 2026Haoze Yan, Julien Roze, Ved Upadhyay +2Hawkes Processes Human Oversight of AI
Department of Industrial Engineering and Operations Research, UC Berkeley, United States. · Stoa, United States. · Department of Cybersecurity, University at Albany, SUNY, United States.
Robust Adversarial Reinforcement Learning with Risk Sensitivity and Critic Consistency Regularization
Sep 23, 2026Jiaxi Wu, Tiantian Zhang, Yuxing Wang +2Reinforcement Learning Risk-Sensitive RL
Shenzhen International Graduate School, Tsinghua University Shenzhen, China
DUGM-R: Uncertainty-Aware Dynamic Grid Mapping and Risk-Triggered Recovery for Learned Local Navigation
Sep 23, 2026Haoyun Feng, Adrian Rubio-Solis, Zhaodong Guo +1Occupancy Grid Mapping Robot Navigation
The Hamlyn Centre for Robotic Surgery, Imperial College London, London, United Kingdom. · Department of Computing, Imperial College London, London, United Kingdom.
Deep Reinforcement Learning with Buffered Quantile Objectives
Sep 18, 2026Mohammad Alipour-vaezi, Sajad KhodadadianRisk-Sensitive RL Distributional RL
Virginia Tech
Nonparametric Variance-Penalized Actor-Critic: Statistical Inference for Risk-Sensitive Reinforcement Learning
Sep 13, 2026Saunak Kumar Panda, Tong Li, Yisha Xiang +1Risk-Sensitive RL Actor-Critic Methods
Human-Centered AI Institute, C.T. Bauer College of Business, University of Houston, Houston, TX 77204 USA · Department of Industrial and Systems Engineering, University of Houston, Houston, TX 77204 USA · Department of Mathematics and Statistics, Texas Tech University, Lubbock, TX 79409 USA
Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations
Sep 9, 2026Tong Li, Saunak Kumar Panda, Yisha XiangRisk-Sensitive RL Adversarial Perturbations
Department of Industrial and Systems Engineering, University of Houston, 4302 University Dr, Houston, 77004, TX, USA. · Department of Decision and Information Science, University of Houston, 4302 University Dr, Houston, 77004, TX, USA.
A Risk-Sensitive and Uncertainty-Aware Decision-Making and Control Framework for Safe and Robust Autonomous Driving
Sep 9, 2026Zhuoren Li, Ran Yu, Weiqi Zhang +4Control Barrier Functions Autonomous Driving
College of Automotive and Energy Engineering, Tongji University, Shanghai 201804, China · Department of Data and Systems Engineering, The University of Hong Kong, Hong Kong
Risk-Conditioned Fine-Tuning of Large Language Models
Sep 8, 2026Zixuan Liu, Fangzheng Wu, Brian Summa +1Risk-Sensitive RL LLM Safety
Department of Computer Science, Tulane University, New Orleans, LA, 70118, USA
Mini-Batch Risk-Averse Deep Q-Learning: A Robot Navigation Case Study
Sep 7, 2026Aayush Patel, Andrzej RuszczyńskiDeep Q-Learning Robot Navigation
Department of Management Science and Information Systems, Rutgers University
How to Learn from What a Human Would Avoid? Intervention-Aware World Models with Real-World RL for Dexterous Manipulation
Sep 5, 2026Jiaju Yin, Zhenhui Zhang, Lixin Xu +5Robotic RL Human-in-the-Loop AI
HKUST (Guangzhou) · Italian Institute of Technology · Zhejiang University
Risk and Anomaly Identification for Distribution Network Optimal Operation Based on Reinforcement Learning and Uncertainty Quantification
Sep 3, 2026Ziqi ZhangRisk-Sensitive RL OOD Detection
College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211100, China

Oct 19, 2026J/K move · Enter open · S save

Deep catastrophic-risk-aware reinforcement learning

Carlos Pérez Octavio Mendoza, Frédéric Godin

Concordia University, Department of Mathematics and Statistics, Montréal, Canada · Quantact Laboratory, Centre de Recherches Mathématiques, Montréal, Canada

Risk-Sensitive RL Conditional Value-at-Risk Extreme Value Theory Quantitative Finance

Open paper AskPDF arXiv

Estimating extreme risk in reinforcement learning becomes increasingly difficult as the confidence level increases, since only a small number of observations contribute to empirical estimates of Conditional Value-at-Risk (CVaR). In this paper, we extend an Extreme Value Theory (EVT) approach for CVaR optimization to high-dimensional neural-network policies. Using a peaks-over-threshold representation and method-of-moments estimation, we obtain a differentiable tail-risk objective and show that, conditional on the selected threshold, its finite-sample gradient can be computed directly through automatic differentiation. We first validate the approach in a controlled experiment with a known optimal solution and then investigate its performance in a deep-hedging problem with rare extreme market movements. Our results show similar hedging behavior for EVT- and empirical-CVaR-trained policies under normal conditions, while EVT-trained policies exhibit lower hedging losses under the extreme-scenario evaluation across the training sample sizes considered. These results suggest that EVT can provide a practical alternative for training neural policies when the observations relevant to extreme risk are scarce.