Interpretable Reward

Interpretable reward learning in reinforcement learning (RL) aims to create reward functions that are both effective in training agents and easily understood by humans, addressing the challenge of designing complex reward functions manually. Current research focuses on methods that learn rewards from human feedback, leveraging large language models or simpler models like differentiable decision trees and prototypical networks to improve data efficiency and interpretability. This work is significant because it enables more reliable and explainable RL agents, particularly beneficial in high-stakes applications like healthcare and robotics where understanding the agent's decision-making process is crucial. Improved interpretability also facilitates debugging and validation of RL systems, enhancing the reproducibility and trustworthiness of research findings.

Papers

June 24, 2024

OCALM: Object-Centric Assessment with Language Models
Timo Kaufmann, Jannis Blüml, Antonia Wüst, Quentin Delfosse, Kristian Kersting, Eyke Hüllermeier
Language Model Reinforcement Learning Reward Function Interpretable Reward

June 6, 2024

Prototypical Reward Network for Data-Efficient RLHF
Jinghan Zhang, Xiting Wang, Yiqiao Jin, Changyu Chen, Xinhao Zhang, Kunpeng Liu
Language Model Reinforcement Learning Reward Model Reinforcement Learning From Human Feedback Prototypical Network Interpretable Reward

March 26, 2024

TractOracle: towards an anatomically-informed reward function for RL-based tractography
Antoine Théberge, Maxime Descoteaux, Pierre-Marc Jodoin
Reinforcement Learning Reward Function Tractography Method Interpretable Reward

March 20, 2024

Dynamic Reward Adjustment in Multi-Reward Reinforcement Learning for Counselor Reflection Generation
Do June Min, Veronica Perez-Rosas, Kenneth Resnicow, Rada Mihalcea
Language Generation Interpretable Reward Thresholding Linear Bandit Counselor Reflection Generation

February 6, 2024

Transductive Reward Inference on Graph
Bohao Qu, Xiaofeng Cao, Qing Guo, Yi Chang, Ivor W. Tsang, Chengqi Zhang
Graph Drawing Interpretable Reward Reward Inference

November 23, 2023

Multi-intention Inverse Q-learning for Interpretable Behavior Representation
Hao Zhu, Brice De La Crompe, Gabriel Kalweit, Artur Schneider, Maria Kalweit, Ilka Diester, Joschka Boedecker
Inverse Reinforcement Learning Human Decision Behavior Representation Interpretable Reward Inverse Q Learning

June 22, 2023

Can Differentiable Decision Trees Enable Interpretable Reward Learning from Human Feedback?
Akansha Kalra, Daniel S. Brown
Human Feedback Reward Function Semi Supervised Reward Interpretable Reward Differentiable Decision Tree

October 3, 2022

Reward Learning with Trees: Methods and Evaluation
Tom Bewley, Jonathan Lawry, Arthur Richards, Rachel Craddock, Ian Henderson
Global Evaluation NCD Method Reward Function Tree Specie Reward Learning Interpretable Tree Interpretable Reward Reward Tree

March 28, 2022

Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions
Alejandro Escontrela, Xue Bin Peng, Wenhao Yu, Tingnan Zhang, Atil Iscen, Ken Goldberg, Pieter Abbeel
Interpretable Reward Substitute Model Complex Reward Function

March 2, 2022

Reliable validation of Reinforcement Learning Benchmarks
Matthias Müller-Brockhausen, Aske Plaat, Mike Preuss
Reinforcement Learning Offline Reinforcement Learning Experimental Validation Reinforcement Learning Benchmark Game AI Reinforcement Learning Environment Interpretable Reward

February 6, 2022

Learning Synthetic Environments and Reward Networks for Reinforcement Learning
Fabio Ferreira, Thomas Nierhoff, Andreas Saelinger, Frank Hutter
Reinforcement Learning Reinforcement Learning Algorithm Synthetic Environment Proxy Model Interpretable Reward