Constrained Reinforcement Learning

Constrained Reinforcement Learning (CRL) addresses the challenge of training agents to maximize rewards while simultaneously satisfying safety or resource constraints, crucial for deploying RL in real-world applications. Current research focuses on developing efficient algorithms, such as primal-dual methods, penalty function methods, and those incorporating techniques like log barrier functions or posterior sampling, often within model-based or model-free frameworks. These advancements improve the safety and reliability of RL agents across diverse domains, including robotics, resource allocation, and safe navigation, by ensuring learned policies adhere to critical operational limitations. The resulting improvements in robustness and safety are significant for transitioning RL from simulated to real-world deployments.

Papers

March 21, 2024

Constrained Reinforcement Learning with Smoothed Log Barrier Function
Baohe Zhang, Yuan Zhang, Lilli Frison, Thomas Brox, Joschka Bödecker
Reinforcement Learning Policy Learning Reward Shaping Constrained Reinforcement Learning Soft Actor Critic Control Constraint Logarithmic Barrier

March 12, 2024

Constrained Optimal Fuel Consumption of HEV: A Constrained Reinforcement Learning Approach
Shuchang Yan
Electric Vehicle Constrained Reinforcement Learning Fuel Consumption Hybrid Electric Vehicle Constrained Reinforcement LeArning

March 4, 2024

Enhancing LLM Safety via Constrained Direct Preference Optimization
Zixuan Liu, Xiaolin Sun, Zizhan Zheng
Large Language Model Direct Preference Optimization Constrained Reinforcement Learning LLM Safety

February 16, 2024

Double Duality: Variational Primal-Dual Policy Optimization for Constrained Reinforcement Learning
Zihao Li, Boyi Liu, Zhuoran Yang, Zhaoran Wang, Mengdi Wang
Markov Decision Process Low Rank Primal Dual Constrained Reinforcement Learning Strong Duality Convex Constraint

January 21, 2024

Constrained Reinforcement Learning for Adaptive Controller Synchronization in Distributed SDN
Ioannis Panitsas, Akrit Mudvari, Leandros Tassiulas
Constrained Reinforcement Learning Software Defined Adaptive Controller Value Based Reinforcement Learning

December 29, 2023

Actuator-Constrained Reinforcement Learning for High-Speed Quadrupedal Locomotion
Young-Ha Shin, Tae-Gyu Song, Gwanghyeon Ji, Hae-Won Park
Reinforcement Learning Robot Person Constrained Reinforcement Learning Torque Control Manipulability Maximization

December 28, 2023

Resilient Constrained Reinforcement Learning
Dongsheng Ding, Zhengyan Huan, Alejandro Ribeiro
Constrained Reinforcement Learning

December 16, 2023

Imitate the Good and Avoid the Bad: An Incremental Approach to Safe Reinforcement Learning
Huy Hoang, Tien Mai, Pradeep Varakantham
Reinforcement Learning Safe Reinforcement Learning Constrained Reinforcement Learning Incremental Model Cost Constraint

November 9, 2023

Anytime-Constrained Reinforcement Learning
Jeremy McMahan, Xiaojin Zhu
Markov Decision Process Optimal Policy Constrained Reinforcement Learning Constrained Markov Decision Process Deterministic Policy Markovian Policy

October 14, 2023

Reduced Policy Optimization for Continuous Control with Hard Constraints
Shutong Ding, Jingya Wang, Yali Du, Ye Shi
Reinforcement Learning Policy Gradient Policy OpTimization Continuous Control Constrained Reinforcement Learning Hard Constraint Constrained Reinforcement Learning Algorithm

October 11, 2023

Robust Safe Reinforcement Learning under Adversarial Disturbances
Zeyang Li, Chuxiong Hu, Shengbo Eben Li, Jia Cheng, Yunan Wang
Reinforcement Learning Constrained Reinforcement Learning Constrained Policy Optimization Adversarial Disturbance

October 6, 2023

Confronting Reward Model Overoptimization with Constrained RLHF
Ted Moskovitz, Aaditya K. Singh, DJ Strouse, Tuomas Sandholm, Ruslan Salakhutdinov, Anca D. Dragan, Stephen McAleer
Large Language Model Reward Model Constrained Reinforcement Learning Gradient Free Reward Overoptimization

September 27, 2023

July 13, 2023

Probabilistic Constrained Reinforcement Learning with Formal Interpretability
Yanran Wang, Qiuchen Qian, David Boyle
Reinforcement Learning Policy OpTimization Constrained Reinforcement Learning Sequential Decision Making Problem Wasserstein Gradient Built in Interpretability Decision Logic

July 10, 2023

Continual Learning as Computationally Constrained Reinforcement Learning
Saurabh Kumar, Henrik Marklund, Ashish Rao, Yifan Zhu, Hong Jun Jeon, Yueyang Liu, Benjamin Van Roy
Artificial Intelligence Continual LEArning Constrained Reinforcement Learning AI Capability

June 29, 2023

Probabilistic Constraint for Safety-Critical Reinforcement Learning
Weiqin Chen, Dharmashankar Subramanian, Santiago Paternain
Reinforcement Learning Constrained Reinforcement Learning Primal Dual Algorithm Probabilistic Constraint

June 15, 2023

Generalizable Resource Scaling of 5G Slices using Constrained Reinforcement Learning
Muhammad Sulaiman, Mahdieh Ahmadi, Mohammad A. Salahuddin, Raouf Boutaba, Aladdin Saleh
Constrained Reinforcement Learning Quality of Service Network Slicing Quality of Experience Resource Scaling

June 10, 2023

A Single-Loop Deep Actor-Critic Algorithm for Constrained Reinforcement Learning with Provable Convergence
Kexuan Wang, An Liu, Baishuo Lin
Reinforcement Learning Policy Gradient Actor Critic Actor Critic Algorithm Constrained Reinforcement Learning Single Loop Provable Convergence

May 23, 2023

Constrained Proximal Policy Optimization
Chengbin Xuan, Feng Zhang, Faliang Yin, Hak-Keung Lam
Reinforcement Learning Proximal Policy Optimization First Order Constrained Reinforcement Learning Policy Distribution