Inverse Reinforcement Learning

Inverse reinforcement learning (IRL) aims to infer an agent's reward function from observations of its behavior, essentially reverse-engineering the decision-making process. Current research emphasizes improving the robustness and efficiency of IRL algorithms, particularly in handling noisy or incomplete data, diverse expert policies, and non-Markovian rewards, often employing techniques like maximum entropy IRL, Bayesian IRL, and various model-predictive control methods. These advancements are crucial for applications such as robotics, autonomous driving, and human-computer interaction, where learning from human demonstrations or preferences is essential for safe and effective system design. Furthermore, research is actively addressing challenges like scalability to large state spaces and the transferability of learned reward functions to new environments.

Papers

November 15, 2024

Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review
Hossein Hassani, Roozbeh Razavi-Far, Mehrdad Saif, Liang Lin
Reinforcement Learning Strong Generalization Reward Function Formality Transfer Inverse Reinforcement Learning Sample Efficiency Sequential Decision Making Problem Inverse Task

November 14, 2024

Approximated Variational Bayesian Inverse Reinforcement Learning for Large Language Model Alignment
Yuang Cai, Yuyu Yuan, Jinsheng Shi, Qinhong Lin
Large Language Model Imitation Learning Inverse Reinforcement Learning Large Language Model Alignment

November 12, 2024

Learning Autonomous Docking Operation of Fully Actuated Autonomous Surface Vessel from Expert data
Akash Vijayakumar, Atmanand M A, Abhilash Somayajula
Inverse Reinforcement Learning Autonomous Surface Vessel Expert Annotated Docking Process Autonomous Docking Freight Carrying Vessel

November 11, 2024

Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching
Arnav Kumar Jain, Harley Wiltzer, Jesse Farebrother, Irina Rish, Glen Berseth, Sanjiban Choudhury
Inverse Reinforcement Learning Reward Model Matching Accuracy Adversarial Game Successor Feature Adversarial Inverse Reinforcement Learning Game Solving

November 8, 2024

Real-World Offline Reinforcement Learning from Vision Language Model Feedback
Sreyas Venkataraman, Yufei Wang, Ziyu Wang, Zackory Erickson, David Held
Offline Reinforcement Learning Human Feedback Inverse Reinforcement Learning Policy Learning Real Robot

November 7, 2024

November 6, 2024

From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning
Zhirui Deng, Zhicheng Dou, Yutao Zhu, Ji-Rong Wen, Ruibin Xiong, Mang Wang, Weipeng Chen
Reinforcement Learning Expert Knowledge Inverse Reinforcement Learning Sample STEP Implicit Reward Novice Classification

November 5, 2024

Out-of-Distribution Recovery with Object-Centric Keypoint Inverse Policy For Visuomotor Imitation Learning
George Jiayuan Gao, Tianyu Li, Nadia Figueroa
Inverse Reinforcement Learning Behavior Cloning Visuomotor Policy Recovery Guarantee Visual Imitation Learning Class Out of Distribution Keypoint Based

November 4, 2024

Learning to Assist Humans without Inferring Rewards
Vivek Myers, Evan Ellis, Sergey Levine, Benjamin Eysenbach, Anca Dragan
Reinforcement Learning LeArning Abstract Real Human Inverse Reinforcement Learning Novel Predecessor and Successor Reward Inference

October 31, 2024

Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment
Weichao Zhou, Wenchao Li
Reinforcement Learning Adversarial Attack Alignment Problem Related Task Inverse Reinforcement Learning Task Alignment Database Alignment

October 24, 2024

Learning Transparent Reward Models via Unsupervised Feature Selection
Daulet Baimukashev, Gokhan Alcan, Kevin Sebastian Luck, Ville Kyrki
Inverse Reinforcement Learning Reward Model Behavior Cloning Dimensional State Space Standard Reinforcement Learning Unsupervised Feature Selection

October 21, 2024

October 18, 2024

Inverse Reinforcement Learning from Non-Stationary Learning Agents
Kavinayan P. Sivakumar, Yi Shen, Zachary Bell, Scott Nivison, Boyuan Chen, Michael M. Zavlanos
Inverse Reinforcement Learning Behavior Cloning Non Stationary Reinforcement Learning Problem

October 16, 2024

Insights from the Inverse: Reconstructing LLM Training Goals Through Inverse RL
Jared Joselowitz, Arjun Jagota, Satyapriya Krishna, Sonali Parbhoo
Large Language Model DCU Insight AQ Inverse Reinforcement Learning Reward Model Toxicity Detection LLM Training Inverse Task Implicit Reward

October 10, 2024

October 8, 2024

Quality Diversity Imitation Learning
Zhenglin Wan, Xingrui Yu, David Mark Bossens, Yueming Lyu, Qing Guo, Flint Xiaofeng Fan, Ivor Tsang
Imitation Learning Inverse Reinforcement Learning Adversarial Imitation Learning Obstacle Scenario

Inverse Reinforcement Learning

Papers

Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review

Approximated Variational Bayesian Inverse Reinforcement Learning for Large Language Model Alignment

Learning Autonomous Docking Operation of Fully Actuated Autonomous Surface Vessel from Expert data

Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching

Real-World Offline Reinforcement Learning from Vision Language Model Feedback

Pruning the Path to Optimal Care: Identifying Systematically Suboptimal Medical Decision-Making with Inverse Reinforcement Learning

Inverse Transition Learning: Learning Dynamics from Demonstrations

From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning

Out-of-Distribution Recovery with Object-Centric Keypoint Inverse Policy For Visuomotor Imitation Learning

Learning to Assist Humans without Inferring Rewards

Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment

Learning Transparent Reward Models via Unsupervised Feature Selection

In-Trajectory Inverse Reinforcement Learning: Learn Incrementally Before An Ongoing Trajectory Terminates

Reinforced Imitative Trajectory Planning for Urban Automated Driving

Inverse Reinforcement Learning from Non-Stationary Learning Agents

Insights from the Inverse: Reconstructing LLM Training Goals Through Inverse RL

UNIQ: Offline Inverse Q-learning for Avoiding Undesirable Demonstrations

Rethinking Adversarial Inverse Reinforcement Learning: From the Angles of Policy Imitation and Transferable Reward Recovery

Offline Inverse Constrained Reinforcement Learning for Safe-Critical Decision Making in Healthcare

Quality Diversity Imitation Learning