Inverse Reinforcement Learning

Inverse reinforcement learning (IRL) aims to infer an agent's reward function from observations of its behavior, essentially reverse-engineering the decision-making process. Current research emphasizes improving the robustness and efficiency of IRL algorithms, particularly in handling noisy or incomplete data, diverse expert policies, and non-Markovian rewards, often employing techniques like maximum entropy IRL, Bayesian IRL, and various model-predictive control methods. These advancements are crucial for applications such as robotics, autonomous driving, and human-computer interaction, where learning from human demonstrations or preferences is essential for safe and effective system design. Furthermore, research is actively addressing challenges like scalability to large state spaces and the transferability of learned reward functions to new environments.

Papers

October 8, 2024

Demonstration Based Explainable AI for Learning from Demonstration Methods
Morris Gu, Elizabeth Croft, Dana Kulic
Explainable AI Inverse Reinforcement Learning Textual Demonstration Robot Programming Robot Performance

September 26, 2024

Inverse Reinforcement Learning with Multiple Planning Horizons
Jiayu Yao, Weiwei Pan, Finale Doshi-Velez, Barbara E Engelhardt
Reward Function Inverse Reinforcement Learning Planning Horizon Discount Factor

September 25, 2024

Learning Utilities from Demonstrations in Markov Decision Processes
Filippo Lazzati, Alberto Maria Metelli
Markov Decision Process Inverse Reinforcement Learning Noisy Demonstration Sequential Decision Making Problem Utility Learning

September 23, 2024

CANDERE-COACH: Reinforcement Learning from Noisy Feedback
Yuxuan Li, Srijita Das, Matthew E. Taylor
Reinforcement Learning Imitation Learning Inverse Reinforcement Learning Noisy Feedback Human COACH AI Coach

September 22, 2024

Distributionally Robust Inverse Reinforcement Learning for Identifying Multi-Agent Coordinated Sensing
Luke Snow, Vikram Krishnamurthy
Inverse Reinforcement Learning Robust Estimation Multi Agent Coordination Distributionally Robust

September 12, 2024

Learning Causally Invariant Reward Functions from Diverse Demonstrations
Ivan Ovinnikov, Eugene Bykovets, Joachim M. Buhmann
Reward Function Regularization Model Inverse Reinforcement Learning Noisy Demonstration Model Overfitting

September 2, 2024

Imitating Language via Scalable Inverse Reinforcement Learning
Markus Wulfmeier, Michael Bloesch, Nino Vieillard, Arun Ahuja, Jorg Bornschein, Sandy Huang, Artem Sokolov, Matt Barnes, Guillaume Desjardins, Alex Bewley, Sarah Maria Elisabeth Bechtle, Jost Tobias Springenberg, Nikola Momchev, Olivier Bachem, Matthieu Geist, Martin Riedmiller
Language Model Imitation Learning Inverse Reinforcement Learning

August 22, 2024

Pareto Inverse Reinforcement Learning for Diverse Expert Policy Generation
Woo Kyung Kim, Minjong Yoo, Honguk Woo
Offline Reinforcement Learning Inverse Reinforcement Learning Diversity Reinforcement Learning Pareto Optimal Policy

July 29, 2024

A Differential Dynamic Programming Framework for Inverse Reinforcement Learning
Kun Cao, Xinhang Xu, Wanxin Jin, Karl H. Johansson, Lihua Xie
Inverse Reinforcement Learning Closed Loop Differential Dynamic Programming

July 26, 2024

PP-TIL: Personalized Planning for Autonomous Driving with Instance-based Transfer Imitation Learning
Fangze Lin, Ying He, Fei Yu
Autonomous Driving Pre Trained Model Imitation Learning Inverse Reinforcement Learning Urban Autonomous Driving Hierarchical Imitation Personalized Motion

July 17, 2024

Robotic Arm Manipulation with Inverse Reinforcement Learning & TD-MPC
Md Shoyib Hassan, Sabir Md Sanaullah
Inverse Reinforcement Learning Manipulation Task Td MPC Cost Function

July 15, 2024

Walking the Values in Bayesian Inverse Reinforcement Learning
Ondrej Bajgar, Alessandro Abate, Konstantinos Gatsis, Michael A. Osborne
Inverse Reinforcement Learning Markov Chain Monte Carlo Policy Value Hamiltonian Monte Carlo

July 13, 2024

Preserving the Privacy of Reward Functions in MDPs through Deception
Shashank Reddy Chirra, Pradeep Varakantham, Praveen Paruchuri
Differential Privacy Reward Function Privacy Policy Inverse Reinforcement Learning Multi Agent Sequential Decision

June 30, 2024

Maximum Entropy Inverse Reinforcement Learning of Diffusion Models with Energy-Based Models
Sangwoong Yoon, Himchan Hwang, Dohyun Kwon, Yung-Kyun Noh, Frank C. Park
Diffusion Model Inverse Reinforcement Learning Energy Based Model Novel Reinforcement Learning Entropy Maximization

June 29, 2024

A Bayesian Solution To The Imitation Gap
Risto Vuorio, Mattie Fellows, Cong Lu, Clémence Grislain, Shimon Whiteson
Imitation Learning Inverse Reinforcement Learning Expert Demonstration Bayes Optimal Policy Probabilistic Solution

June 24, 2024

MEReQ: Max-Ent Residual-Q Inverse RL for Sample-Efficient Alignment from Intervention
Yuxin Chen, Chen Tang, Chenran Li, Ran Tian, Wei Zhan, Peter Stone, Masayoshi Tomizuka
Inverse Reinforcement Learning Residual Reinforcement Learning Early Intervention Sample Efficient Policy

June 20, 2024

Bayesian Inverse Reinforcement Learning for Non-Markovian Rewards
Noah Topper, Alvaro Velasquez, George Atia
Inverse Reinforcement Learning Reward Machine Non Markovian Reward Markovian Reward

June 18, 2024

Autonomous navigation of catheters and guidewires in mechanical thrombectomy using inverse reinforcement learning
Harry Robertshaw, Lennart Karstensen, Benjamin Jackson, Alejandro Granados, Thomas C. Booth
Autonomous Navigation Inverse Reinforcement Learning Reward Shaping Central Venous Catheter Endovascular Intervention Guidewire Segmentation Endovascular Navigation

June 15, 2024

EvIL: Evolution Strategies for Generalisable Imitation Learning
Silvia Sapora, Gokul Swamy, Chris Lu, Yee Whye Teh, Jakob Nicolaus Foerster
Imitation Learning Inverse Reinforcement Learning Reward Model Feature Imitation Behavior Cloning Evolution Strategy

June 12, 2024

RILe: Reinforced Imitation Learning
Mert Albaba, Sammy Christen, Thomas Langarek, Christoph Gebhardt, Otmar Hilliges, Michael J. Black
Reinforcement Learning Imitation Learning Inverse Reinforcement Learning Expert Model Reward Engineering