Inverse Constrained Reinforcement Learning

Inverse Constrained Reinforcement Learning (ICRL) aims to infer implicit constraints governing expert behavior from observational data, enabling the creation of safe and effective reinforcement learning agents. Current research focuses on developing efficient exploration strategies to improve constraint inference from limited demonstrations, analyzing the identifiability and generalizability of learned constraints, and creating robust benchmarks for evaluating ICRL algorithms, often employing variational methods to model uncertainty in constraint estimation. This field is crucial for deploying RL agents in real-world scenarios where explicitly defining all constraints is impractical, impacting areas like robotics, autonomous driving, and other safety-critical applications.

Papers

October 10, 2024

Offline Inverse Constrained Reinforcement Learning for Safe-Critical Decision Making in Healthcare
Nan Fang, Guiliang Liu, Wei Gong
Reinforcement Learning Markov Decision Process Inverse Reinforcement Learning Healthcare System Constrained Reinforcement Learning Inverse Constrained Reinforcement Learning

September 24, 2024

Provably Efficient Exploration in Inverse Constrained Reinforcement Learning
Bo Yue, Jian Li, Guiliang Liu
Efficient Exploration Constrained Optimization Inverse Constrained Reinforcement Learning Constraint Inference

September 11, 2024

A Survey of Inverse Constrained Reinforcement Learning: Definitions, Progress and Challenges
Guiliang Liu, Sheng Xu, Shicheng Liu, Ashish Gaurav, Sriram Ganapathi Subramanian, Pascal Poupart
Timely Survey Autonomous Driving Technical Challenge Much Progress Robot Control Definition Defining Redefinition Stochastic Environment Knowledge Constraint Inverse Constrained Reinforcement Learning

June 24, 2024

Confidence Aware Inverse Constrained Reinforcement Learning
Sriram Ganapathi Subramanian, Guiliang Liu, Mohammed Elmahgiubi, Kasra Rezaee, Pascal Poupart
Reinforcement Learning Expert Trajectory Inverse Constrained Reinforcement Learning

June 1, 2023

Identifiability and Generalizability in Constrained Inverse Reinforcement Learning
Andreas Schlaginhaufen, Maryam Kamgarpour
Reinforcement Learning Reward Function Inverse Reinforcement Learning Stronger Generalizability Constrained Markov Decision Process Novel Identifiability Test Inverse Reward Design Inverse Constrained Reinforcement Learning

June 20, 2022

Benchmarking Constraint Inference in Inverse Reinforcement Learning
Guiliang Liu, Yudong Luo, Ashish Gaurav, Kasra Rezaee, Pascal Poupart
Reinforcement Learning Inverse Reinforcement Learning Constraint Inference Inverse Constrained Reinforcement Learning