Constrained RL

RL: Reinforcement Learning

Constrained reinforcement learning studies how agents can maximize rewards while obeying limits on costs such as collisions, energy use, latency, or harm. Many methods model the problem as a constrained Markov decision process and use Lagrangian or primal-dual updates; others enforce feasibility with action masks, projections, safety shields, control barrier functions, or reachability analysis. These approaches are used in settings from robot control and power systems to language-model alignment, with work spanning online and offline learning, multi-agent tasks, and formal guarantees for regret and constraint satisfaction. A central challenge is balancing useful performance with reliable safety: average-cost limits can miss rare severe failures, while strict safeguards may make learning conservative or difficult to scale.

Latest papers

226 papers38 papers in the last four weeks, up 46% on the four weeks before. 0.2% of all new papers.

Decoupled Guidance Diffusion for Adaptive Offline Safe Reinforcement Learning
May 4, 2026Rufeng Chen, Zhaofan Zhang, Zhejiang Yang +2Diffusion-Based Planning Diffusion Guidance
1The Hong Kong University of Science and Technology (Guangzhou) · 2Jilin University
Experience Constrained Hierarchical Federated Reinforcement Learning for Large-scale UAV Teams in Hazardous Environments
May 4, 2026Qinwei Huang, Rui Zuo, Simon Khan +1Hierarchical Federated Learning Federated RL
EECS Syracuse University Syracuse, USA · AFRL Air Force Research Laboratory Rome, USA
Augmented Lagrangian Multiplier Network for State-wise Safety in Reinforcement Learning
May 1, 2026Jiaming Zhang, Yujie Yang, Yao Lyu +2Constrained RL Augmented Lagrangian Method
School of Vehicle and Mobility & College of AI, Tsinghua University, Beijing, 100084, China
Global Optimality for Constrained Exploration via Penalty Regularization
Apr 30, 2026Florian Wolf, Ilyas Fatkhullin, Niao HeConstrained RL Maximum Entropy RL
The Computing & Mathematical Sciences Department, California Institute of Technology, Pasadena, CA. · Department of Computer Science, ETH Zurich, Switzerland. · 3ETH AI Center, ETH Zurich, Switzerland.
Dynamic-TD3: A Novel Algorithm for UAV Path Planning with Dynamic Obstacle Trajectory Prediction
Apr 30, 2026Wentao Chen, Jingtang Chen, Mingjian Fu +4Reinforcement Learning UAV Navigation
Learning to Route Electric Trucks Under Operational Uncertainty
Apr 29, 2026Stavros Orfanoudakis, Ziyan Li, Ruixiao Yang +4Sequential Decision Making Electric Vehicle Charging
Delft University of Technology, Intelligent Electrical Power Grids, Delft, The Netherlands · Massachusetts Institute of Technology, Center of Transportation and Logistics, Cambridge, MA, United States of America · Massachusetts Institute of Technology, Department of Aeronautics and Astronautics, Cambridge, MA, United States of America
A Distributionally Robust Reinforcement Learning Framework for Constrained Urban EV Dispatch
Apr 28, 2026An Nguyen, Hoang Nguyen, Phuong Le +3Electric Vehicle Charging Constrained RL
College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam · Center for Environmental Intelligence, VinUniversity, Hanoi, Vietnam
GradMAP: Gradient-Based Multi-Agent Proximal Learning for Grid-Edge Flexibility
Apr 27, 2026Yihong Zhou, Hongtai Zeng, Thomas MorstynMulti-Agent Control Constrained RL
Department of Engineering Science, University of Oxford
CAPSULE: Control-Theoretic Action Perturbations for Safe Uncertainty-Aware Reinforcement Learning
Apr 26, 2026Rahul Narava, Siddharth Verma, Ojas Jain +2Control Barrier Functions Constrained RL
Indian Institute of Technology Ropar · CRAN, Université de Lorraine
Bridging Reasoning and Action: Hybrid LLM-RL Framework for Efficient Cross-Domain Task-Oriented Dialogue
Apr 25, 2026Yangyang Zhao, Linfan Dai, Li Cai +2Constrained RL LLM-Guided RL
School of Computer Science and Technology, Changsha University of Science and Technology · School of Computer Science and Engineering, Central South University · Text Computing and Cognitive Intelligence MOE Engineering Research Center, Guizhou University +2
Learning Control Policies to Provably Satisfy Hard Affine Constraints for Black-Box Hybrid Dynamical Systems
Apr 24, 2026Aayushi Shrivastava, Kartik Nagpal, Sairam Jinkala +2Dynamical Systems Safety-Critical Control
Department of Mechanical Engineering, University of California Berkeley, Berkeley, CA 94709, USA
Policy Gradient Primal-Dual Method for Safe Reinforcement Learning from Human Feedback
Apr 21, 2026Qiang Liu, Adrienne Kline, Ermin WeiConstrained RL Policy Gradient Methods
Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL, USA · Center for Artificial Intelligence, Bluhm Cardiovascular Institute, Northwestern Medicine, Chicago, IL, USA · Department of Surgery, Northwestern University, Chicago, IL, USA +3
Physics-Informed Causal MDPs for Sequential Constraint Repair in Engineering Simulation Pipelines
Apr 20, 2026Chuhan QiaoMarkov Decision Processes Reinforcement Learning
Beijing JiaoTong University
Hierarchical Reinforcement Learning with Runtime Safety Shielding for Power Grid Operation
Apr 15, 2026Gitesh MalikSafe RL Power Systems
Department of Electrical Engineering, Delhi Technological University (DTU), India
Optimistic Policy Learning under Pessimistic Adversaries with Regret and Violation Guarantees
Apr 15, 2026Sourav Ganguly, Kartik Pandit, Arnob GhoshAdversarial MARL Reinforcement Learning
Dept. of Electrical and Computer Engineering, NJIT, Newark, NJ, USA
Safe reinforcement learning with online filtering for fatigue-predictive human-robot task planning and allocation in production
Apr 14, 2026Jintao Xue, Xiao Li, Nianmin ZhangHuman-Robot Collaboration Safe RL
CI3 Lab, Department of Civil Engineering, The University of Hong Kong, Hong Kong SAR
Load constrained wind farm flow control through multi-objective multi-agent reinforcement learning
Apr 13, 2026Teodor Åstrand, Marcus Binder Nilsen, Iasonas Tsaklis +3Multi-Agent Reinforcement Learning Constrained RL
Department of Wind and Energy Systems, Technical University of Denmark, Roskilde, Denmark
A Dual-Positive Monotone Parameterization for Multi-Segment Bids and a Validity Assessment Framework for Reinforcement Learning Agent-based Simulation of Electricity Markets
Apr 11, 2026Zunnan Xu, Zhaoxia Jing, Zhanhua PanAutomated Bidding Constrained RL
School of Electric Power Engineering, South China University of Technology, Guangzhou, 510640, Guangdong, China · Department of Engineering, University of Exeter, Exeter EX4 4PY, U.K.
Faithful GRPO: Improving Visual Spatial Reasoning in Multimodal Language Models via Constrained Policy Optimization
Apr 9, 2026Sai Srinivas Kancheti, Aditya Kanade, Rohit Sinha +2Visual Spatial Reasoning Multimodal CoT Reasoning
CSE Dept., IIT Hyderabad, Hyderabad, India · Microsoft Research, Bengaluru, India
Reinforcement Learning with Reward Machines for Sleep Control in Mobile Networks
Apr 8, 2026Kristina Levina, Nikolaos Pappas, Athanasios Karapantelakis +2Wireless Communications Reinforcement Learning
Department of Computer and Information Science, Linköping University, Linköping, Sweden · Ericsson Research
Boosted Distributional Reinforcement Learning: Analysis and Healthcare Applications
Apr 6, 2026Zequn Chen, Wesley J. MarreroReinforcement Learning Distributional RL
Thayer School of Engineering, Dartmouth College
MC-CPO: Mastery-Conditioned Constrained Policy Optimization for Pedagogically Safe Intelligent Tutoring Systems
Apr 5, 2026Oluseyi Olukola, Nick RahimiIntelligent Tutoring Systems Policy Learning
School of Computing Sciences and Computer Engineering University of Southern Mississippi Hattiesburg MS 39406 USA
Improving Feasibility via Fast Autoencoder-Based Projections
Apr 3, 2026Maria Chzhen, Priya L. DontiConstrained Optimization Constrained RL
University of Toronto Toronto, ON, Canada · Massachusetts Institute of Technology Cambridge, MA, USA
Behavior-Constrained Reinforcement Learning with Receding-Horizon Credit Assignment for High-Performance Control
Apr 3, 2026Siwei Ju, Jan Tauberschmidt, Oleg Arenz +2Constrained RL RL for Autonomous Driving
Intelligent Autonomous Systems Lab, Department of Computer Science, TU Darmstadt, Germany. · Porsche AG, Stuttgart, Germany. · German Research Center for Artificial Intelligence (DFKI), Germany.
Tool-Augmented Agent for Closed-loop Optimization,Simulation,and Modeling Orchestration
Apr 1, 2026Liyuan Deng, Shujian Deng, Yongkang Chen +6LLM Agent Orchestration Constrained Optimization
Northwestern Polytechnical University · Shanghai Artificial Intelligence Laboratory
Autonomous Adaptive Solver Selection for Chemistry Integration via Reinforcement Learning
Mar 31, 2026Eloghosa Ikponmwoba, Opeoluwa OwoyeleReinforcement Learning Constrained RL
Department of Mechanical and Industrial Engineering Louisiana State University Baton Rouge, LA 70803
Offline Constrained RLHF with Multiple Preference Oracles
Mar 31, 2026Brenden Latham, Mehrdad MoharramiConstrained RL Reinforcement Learning
Computer Science Department, University of Iowa, Iowa City, IA, USA
Near-Optimal Primal-Dual Algorithm for Learning Linear Mixture CMDPs with Adversarial Rewards
Mar 29, 2026Kihyun Yu, Seoungbin Bae, Dabeen LeeAdversarial MARL Reinforcement Learning
Department of Industrial and Systems Engineering, KAIST · Department of Mathematical Sciences, Seoul National University · Research Institute of Mathematics, Seoul National University +2
Dynamic resource matching in manufacturing using deep reinforcement learning
Mar 28, 2026Saunak Kumar Panda, Yisha Xiang, Ruiqi LiuManufacturing RL for Combinatorial Optimization
Department of Industrial Engineering, University of Houston, Houston, TX 77004, USA · Department of Mathematics and Statistics, Texas Tech University, Lubbock, TX 79409, USA
Off-Policy Safe Reinforcement Learning with Constrained Optimistic Exploration
Mar 25, 2026Guopeng Li, Matthijs T. J. Spaan, Julian F. P. KooijSafe RL Constrained RL
Faculty of Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, Delft, the Netherlands

May 4, 2026J/K move · Enter open · S save

Decoupled Guidance Diffusion for Adaptive Offline Safe Reinforcement Learning

Rufeng Chen, Zhaofan Zhang, Zhejiang Yang, Hechang Chen+1

1The Hong Kong University of Science and Technology (Guangzhou) · 2Jilin University

Diffusion-Based Planning Diffusion Guidance Constrained RL Offline RL Safe RL

Open paper AskPDF arXiv

Offline safe reinforcement learning often requires policies to adapt at deployment time to safety budgets that vary across episodes or change within a single episode. While diffusion-based planners enable flexible trajectory generation, existing guidance schemes often treat reward improvement and constraint satisfaction as competing gradient objectives, which can lead to unreliable safety compliance under cost limits. We reinterpret adaptive safe trajectory generation as sampling from a constrained trajectory distribution, where the budget restricts the trajectory region, and reward shapes preferences within that region. This perspective motivates Safe Decoupled Guidance Diffusion (SDGD), which conditions classifier-free guidance on the cost limit to bias sampling toward trajectories satisfying the specified limit, while using reward-gradient guidance to refine trajectories for higher return. Because direct reward guidance can increase return while also steering samples toward trajectories with higher cumulative cost, we introduce Feasible Trajectory Relabeling (FTR) to reshape reward targets and discourage such directions. We further provide a first-order sampling-time analysis showing that FTR suppresses reward-induced cost drift under a prefix-restorative alignment condition. Extensive evaluations on the DSRL benchmark show that SDGD achieves the strongest safety compliance among baselines, satisfying the constraint on 94.7% of tasks (36/38), while obtaining the highest reward among safe methods on 21 tasks.