Risk-Sensitive RL

RL: Reinforcement Learning

Risk-sensitive reinforcement learning studies how agents can make sequential decisions that account for rare failures and undesirable outcomes, not just average reward. Common approaches optimize CVaR, quantile, variance, expectile, or exponential-utility objectives, often using distributional critics, policy gradients, Q-learning, or model-based planning. Safety-constrained methods add tools such as shields, reachability checks, and robust uncertainty sets, with applications ranging from robot navigation and autonomous driving to finance and language-model alignment. The field is developing convergence and sample-complexity guarantees alongside practical methods, while a central tension remains how to estimate tail risk reliably from limited data without making policies excessively conservative.

Latest papers

85 papers18 papers in the last four weeks, up 38% on the four weeks before. 0.1% of all new papers.

Safety-Constrained Reinforcement Learning with Post-Training Reachability Verification for Robot Navigation
May 13, 2026Qisong He, Xinmiao Huang, Jinwei Hu +4Constrained RL Risk-Sensitive RL
University of Liverpool, Liverpool, UK. · Universit´e Grenoble Alpes, Grenoble, France.
Tight Sample Complexity Bounds for Entropic Best Policy Identification
May 13, 2026Amer Essakine, Claire VernadeReinforcement Learning Risk-Sensitive RL
ENS Paris Saclay · University of Technology Nuremberg
Reinforcement Learning for Exponential Utility: Algorithms and Convergence in Discounted MDPs
May 8, 2026Gugan Thoppe, L. A. Prashanth, Ankur Naskar +1Markov Decision Processes Reinforcement Learning
Computer Science and Automation, Indian Institute of Science, Bengaluru · Computer Science and Engineering, Indian Institute of Technology Madras, Chennai · Tata Consultancy Services Limited, Hyderabad
Actor-Critic Algorithm for Dynamic Expectile and CVaR
May 8, 2026Yudong Luo, Erick DelageReinforcement Learning Risk-Sensitive RL
1GERAD & Department of Decision Sciences, HEC Montr´eal, Canada · 2Mila - Quebec AI Institute, Canada
Learning Material-Aware Hamiltonian Risk Fields for Safe Navigation
May 7, 2026Aditya Sai Ellendula, Yi Wang, Chandrajit BajajRisk-Sensitive RL Dynamical Systems
Department of Computer Science, University of Texas at Austin, Austin, TX 78712, USA · Oden Institute, University of Texas at Austin, Austin, TX 78712, USA
RVPO: Risk-Sensitive Alignment via Variance Regularization
May 7, 2026Ivan Montero, Tomasz Jurczyk, Bhuwan DhingraRL for Language Models Risk-Sensitive RL
Apple
Quantile Geometry Regularization for Distributional Reinforcement Learning
May 5, 2026Zhaofan Zhang, Minghao Yang, Rufeng Chen +2Distributional RL Quantile Regression
Information Hub, AI Thrust Hong Kong University of Science and Technology (Guangzhou) Guangzhou, Guangdong, China
MARS-DA: A Hierarchical Reinforcement Learning Framework for Risk-Aware Multi-Agent Bidding in Power Grids
May 4, 2026Jiayi Chen, Xuan Zhang, Guiling WangHierarchical RL Risk-Sensitive RL
Department of Computer Science, New Jersey Institute of Technology
Reinforcement Learning with Markov Risk Measures and Multipattern Risk Approximation
May 1, 2026Andrzej Ruszczynski, Tiangang ZhangMarkov Decision Processes Reinforcement Learning
Department of Management Science and Information Systems, Rutgers University
Frictive Policy Optimization for LLMs: Epistemic Intervention, Risk-Sensitive Control, and Reflective Alignment
Apr 28, 2026James Pustejovsky, Nikhil KrishnaswamyRL for Language Models Language Model Safety Evaluation
Brandeis University · Waltham, MA, USA · Colorado State University +1
The Price of Paranoia: Robust Risk-Sensitive Cooperation in Non-Stationary Multi-Agent Reinforcement Learning
Apr 17, 2026Deep Kumar Ganguly, Chandradithya S Jonnalagadda, Pratham Chintamani +1Multi-Agent Reinforcement Learning Multi-Agent Coordination
Technical University of Munich (TU Munich) · Brown University · Indian Institute of Technology Tirupati (IIT Tirupati)
Epistemic Robust Offline Reinforcement Learning
Apr 8, 2026Abhilash Reddy Chenreddy, Erick DelageUncertainty Estimation in RL Offline RL
GERAD, Department of Decision Sciences, HEC Montréal, Montréal, Québec H3T 2A7, Canada
Federated Distributional Reinforcement Learning with Distributional Critic Regularization
Mar 18, 2026David Millard, Cecilia Alm, Rashid Ali +2Federated RL Distributional RL
Dept. of Mechanical Engineering, Rochester Institute of Technology, Rochester, NY 14623, USA · Dept. of Psychology and School of Information, Rochester Institute of Technology, Rochester, NY 14623, USA · Department of Engineering Science, University West, Trollhättan 461 30, Sweden +1
Physics-informed offline reinforcement learning eliminates catastrophic fuel waste in maritime routing
Mar 18, 2026Aniruddha Bora, Julie Chalfant, Chryssostomos ChryssostomidisMaritime Transportation Offline RL
Department of Computer Science, Texas State University, San Marcos, TX 78666, USA · MIT Sea Grant College Program, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
Safe RLHF Beyond Expectation: Stochastic Dominance for Universal Spectral Risk Control
Mar 11, 2026Yaswanth Chittepu, Ativ Joshi, Rajarshi Bhattacharjee +1Constrained RL LLM Safety Alignment
University of Massachusetts Amherst
Strategically Robust Multi-Agent Reinforcement Learning with Linear Function Approximation
Mar 10, 2026Jake Gonzales, Max Horwitz, Eric Mazumdar +1Multi-Agent Reinforcement Learning Game-Theoretic RL
Department of Electrical & Computer Engineering, University of Washington · Computing & Mathematical Sciences, California Institute of Technology
Distributional Reinforcement Learning with Information Bottleneck for Uncertainty-Aware DRAM Equalization
Mar 5, 2026Muhammad Usama, Dong Eui ChangRisk-Sensitive RL Distributional RL
Control Laboratory, School of Electrical Engineering, KAIST, Daejeon, 34141, Republic of Korea
Risk-Aware Reinforcement Learning for Mobile Manipulation
Mar 4, 2026Michael Groom, James Wilson, Nick Hawes +1Risk-Sensitive RL Mobile Manipulation
Oxford Robotics Institute, Department of Engineering Science, University of Oxford, Oxford, United Kingdom · Dyson Institute of Engineering & Technology, Malmesbury, United Kingdom · Bristol Robotics Laboratory, School of Engineering, University of West England, Bristol, United Kingdom
Accelerated Online Risk-Averse Policy Evaluation in POMDPs with Theoretical Guarantees and Novel CVaR Bounds
Feb 26, 2026Yaacov Pariente, Vadim IndelmanPartially Observable Markov Decision Processes Policy Evaluation
Technion - Israel Institute of Technology, Israel
Reinforcement learning with an expectile-based objective
Feb 10, 2026Shrey Rakeshkumar Patel, Sumedh Gupte, Soumen Pachal +2Reinforcement Learning Risk-Sensitive RL
Indian Institute of Technology Madras · TCS Research, India
Reward Redistribution for CVaR MDPs using a Bellman Operator on L-infinity
Feb 3, 2026Aneri Muni, Vincent Taboga, Esther Derman +2Markov Decision Processes Q-Learning
Universit´e de Montr´eal, Quebec, Canada · Mila - Quebec AI Institute, Quebec, Canada · GERAD & Department of Decision Sciences, HEC Montr´eal, Quebec, Canada
Cantelli Constrained Policy Optimization
Jan 30, 2026Rohan Tangri, Jan-Peter CalliessPolicy Optimization Constrained RL
Machine Learning Research Group, University of Oxford, Oxford, United Kingdom
Risk-Aware Alignment Must Incorporate Reward Uncertainty
Oct 31, 2024Debangshu Banerjee, Kintan Saha, Aditya GopalanPolicy Optimization LLM Alignment
Department of Electrical and Communication Engineering, Indian Institute of Science, India
ProSpec RL: Plan Ahead, then Execute
Jul 31, 2024Liangliang Liu, Yi Guan, BoRan Wang +5Reinforcement Learning Risk-Sensitive RL
1The Faculty of Computing, Harbin Institute of Technology, Harbin, China · 2The Artificial Intelligence Institute, Harbin Institute of Technology, ShenZhen, China · 3The Artificial Intelligence Institute, Harbin Institute of Technology, Harbin, China
On the Approximation and Convergence of Distributional Policy Gradient Algorithms for Risk-Sensitive Reinforcement Learning
May 23, 2024Xian Yu, Minheng Xiao, Lei YingStochastic Optimization Distributional RL
Department of Integrated Systems Engineering, The Ohio State University, Columbus, OH, USA · Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA

May 13, 2026J/K move · Enter open · S save

Safety-Constrained Reinforcement Learning with Post-Training Reachability Verification for Robot Navigation

Qisong He, Xinmiao Huang, Jinwei Hu, Zhuoyun Li+3

University of Liverpool, Liverpool, UK. · Universit´e Grenoble Alpes, Grenoble, France.

Constrained RL Risk-Sensitive RL Safe Robot Navigation Robot Safety

Open paper AskPDF arXiv

Safe navigation for mobile robots demands policies that remain reliable under the high-consequence perception uncertainty of cluttered environments. Yet most existing safe reinforcement learning (RL) methods assess safety through average cumulative cost. Such metrics can mask dangerous tail-risk behaviors. To address this, we propose a framework that trains risk-sensitive policies through Conditional Value-at-Risk (CVaR) constrained optimization on an off-policy TD3 backbone and evaluates their safety margins post-training through neural network reachability verification. During training, the policy is optimized under CVaR constraints on cumulative costs, promoting sensitivity to high-cost tail outcomes rather than average behavior alone. After training, we compute action reachable sets under bounded observation uncertainty using Taylor Model analysis, yielding a safety rate metric that quantifies the proportion of evaluated states at which the policy's reachable action set remains within prescribed safety margins. A key finding is that policies trained with CVaR constraints maintain larger safety margins from obstacles across evaluated states. This makes them significantly more amenable to formal reachability verification. Experiments across ten navigation scenarios and six baselines show that our method achieves a 98.3% success rate, the highest safety verification rate among all compared methods, while revealing that average cost rankings and reachability-based safety rankings can diverge. This indicates that reachability verification captures risks which are missed by empirical cost metrics alone. We further validate our approach on a physical Clearpath Jackal robot, demonstrating successful sim-to-real transfer.