Reward Function

Reward functions, crucial for guiding reinforcement learning agents towards desired behaviors, are the focus of intense research. Current efforts center on automatically learning reward functions from diverse sources like human preferences, demonstrations (including imperfect ones), and natural language descriptions, often employing techniques like inverse reinforcement learning, large language models, and Bayesian optimization within various architectures including transformers and generative models. This research is vital for improving the efficiency and robustness of reinforcement learning, enabling its application to complex real-world problems where manually designing reward functions is impractical or impossible. The ultimate goal is to create more adaptable and human-aligned AI systems.

Papers

February 1, 2024

January 25, 2024

January 23, 2024

Towards Socially and Morally Aware RL agent: Reward Design With LLM
Zhaoyue Wang
Large Language Model Language Model Reinforcement Learning Reward Function Reward Signal Social Interaction Reward Design

December 30, 2023

December 27, 2023

Inverse Reinforcement Learning with Unknown Reward Model based on Structural Risk Minimization
Chendi Qu, Jianping He, Xiaoming Duan, Jiming Chen
Policy Gradient Reward Function Inverse Reinforcement Learning Reward Model Risk Minimization

December 22, 2023

December 18, 2023

Counting Reward Automata: Sample Efficient Reinforcement Learning Through the Exploitation of Reward Function Structure
Tristan Bester, Benjamin Rosman, Steven James, Geraud Nangue Tasse
Reward Function Sample Efficient Reinforcement Learning Real World eXploitation Finite State Machine Adaptive Machine Automaton Model

December 16, 2023

Deriving Rewards for Reinforcement Learning from Symbolic Behaviour Descriptions of Bipedal Walking
Daniel Harnack, Christoph Lüth, Lukas Gross, Shivesh Kumar, Frank Kirchner
Reinforcement Learning Reward Function Efficient Reinforcement Learning Reward Structure Symbolic AI Hybrid Automaton

December 15, 2023

Automating reward function configuration for drug design
Marius Urbonas, Temitope Ajileye, Paul Gainer, Douglas Pires
Reward Function Drug Discovery Drug Design Reward Function Design Generative Molecular Design

December 14, 2023

Vision-Language Models as a Source of Rewards
Kate Baumli, Satinder Baveja, Feryal Behbahani, Harris Chan, Gheorghe Comanici, Sebastian Flennerhag, Maxime Gazeau, Kristian Holsheimer, Dan Horgan, Michael Laskin, Clare Lyle, Hussain Masoom, Kay McKinney, Volodymyr Mnih, Alexander Neitz, Fabio Pardo, Jack Parker-Holder, John Quan, Tim Rocktäschel, Himanshu Sahni, Tom Schaul, Yannick Schroecker, Stephen Spencer, Richie Steigerwald, Luyu Wang, Lei Zhang
Reinforcement Learning Vision Language Model Reward Function Reward Report Source Table Generalist Agent Visual Goal

December 6, 2023

FoMo Rewards: Can we cast foundation models as reward functions?
Ekdeep Singh Lubana, Johann Brehmer, Pim de Haan, Taco Cohen
Reinforcement Learning Foundation Model Reward Function Open Ended Shelf Convolutional Neural Network

November 30, 2023

Handling Cost and Constraints with Off-Policy Deep Reinforcement Learning
Jared Markowitz, Jesse Silverberg, Gary Collins
Reward Function Action Space Participation Constraint Hidden CoST Policy Actor Critic Policy Deep Reinforcement Learning

November 29, 2023

Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
Lei Zhao, Mengdi Wang, Yu Bai
Reward Function Inverse Reinforcement Learning Standard Reinforcement Learning Intermediate Target

November 27, 2023

Reward Function

Papers

Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learning

RadDQN: a Deep Q Learning-based Architecture for Finding Time-efficient Minimum Radiation Exposure Pathway

Sample Efficient Reinforcement Learning by Automatically Learning to Compose Subtasks

Reinforcement Learning with Hidden Markov Models for Discovering Decision-Making Dynamics

Towards Socially and Morally Aware RL agent: Reward Design With LLM

Principal-Agent Reward Shaping in MDPs

Policy Optimization with Smooth Guidance Learned from State-Only Demonstrations

Causal State Distillation for Explainable Reinforcement Learning

Inverse Reinforcement Learning with Unknown Reward Model based on Structural Risk Minimization

A Survey of Reinforcement Learning from Human Feedback

REBEL: A Regularization-Based Solution for Reward Overoptimization in Robotic Reinforcement Learning from Human Feedback

Counting Reward Automata: Sample Efficient Reinforcement Learning Through the Exploitation of Reward Function Structure

Deriving Rewards for Reinforcement Learning from Symbolic Behaviour Descriptions of Bipedal Walking

Automating reward function configuration for drug design

Vision-Language Models as a Source of Rewards

FoMo Rewards: Can we cast foundation models as reward functions?

Handling Cost and Constraints with Off-Policy Deep Reinforcement Learning

Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective

A Graph Neural Network-Based QUBO-Formulated Hamiltonian-Inspired Loss Function for Combinatorial Optimization using Reinforcement Learning

Evaluating the Impact of Personalized Value Alignment in Human-Robot Interaction: Insights into Trust and Team Performance Outcomes