cs.LGNov 6, 2026

Safety-Focused Reinforcement Learning Using Risk-Sensitive Coupled Control

Authors: Jiabin Lin, Shana Moothedath, Vivek S. Borkar

Organizations: Electrical and Computer Engineering, Iowa State University. · Department of Electrical Engineering, Indian Institute of Technology, Bombay, India.

Abstract

This paper investigates the coupled control problem within safety-focused reinforcement learning, addressing the essential challenge of achieving optimal performance while ensuring safety in systems operating within complex environments. We define safety as the requirement to prevent the system from reaching undesirable or hazardous states. We propose a coupled control approach that prioritizes safety by employing effective methods to prevent the system from entering unsafe states while optimizing the performance metric, defined as the average cost. We propose a novel problem transformation that reformulates it into a risk-sensitive problem combined with an average reward criterion. We present a two-timescale Q-learning algorithm where in the faster timescale, the algorithm solves a risk-sensitive control problem to ensure safety, while on a slower timescale, the algorithm focuses on average cost control. We provide theoretical guarantees for the convergence of our algorithms, establishing their significance in ensuring safe, optimal control in dynamic environments. We validated the performance of our approach using GridWorld and CarGoal task in Safety Gymnasium.

Figures & tables

Explore similar work

CardsList
  1. A Unified Bellman Operator for Safety-Critical Reinforcement Learning

    Oct 8, 2026Nishanth Arun Rao, Royina Karegoudra Jayanth, Benjamin Eysenbach +1Safe RLConstrained RL

  2. Off-Policy Safe Reinforcement Learning with Constrained Optimistic Exploration

    Mar 25, 2026Guopeng Li, Matthijs T. J. Spaan, Julian F. P. KooijSafe RLConstrained RL

  3. Safe Learning Control with Optimality and Stability Guarantees

    Jan 26, 2025Xinyang Wang, Hongwei Zhang, Shimin Wang +2Control Barrier FunctionsRobust Control