Complex Reward Function

Complex reward functions are a critical challenge in reinforcement learning (RL), hindering the development of effective agents for real-world tasks. Current research focuses on methods to learn reward functions from limited data (e.g., using inverse reinforcement learning and human feedback), structure reward functions to improve sample efficiency and safety (e.g., through Reward Machines and geometric fabrics), and address the iterative and uncertain nature of reward design. These advancements are crucial for deploying RL in high-stakes applications like robotics and personalized medicine, where accurately specifying a reward function is often difficult or impossible.

Papers

October 4, 2024

ShieldDiff: Suppressing Sexual Content Generation from Diffusion Models through Reinforcement Learning
Dong Han, Salaheldin Mohamed, Yong Li
Reinforcement Learning Diffusion Model Unsafe Image Unsafe Content Complex Reward Function

September 20, 2024

OMG-RL:Offline Model-based Guided Reward Learning for Heparin Treatment
Yooseok Lim, Sujee Lee
Reinforcement Learning Model Based Offline Reinforcement Learning Optimal Treatment Complex Reward Function Reward Function Parameter Warfarin Dosing

May 31, 2024

Reward Machines for Deep RL in Noisy and Uncertain Environments
Andrew C. Li, Zizhao Chen, Toryn Q. Klassen, Pashootan Vaezipoor, Rodrigo Toro Icarte, Sheila A. McIlraith
Deep Reinforcement Learning Industrial Disturbing Noise Deep RL Uncertain Environment Reward Machine Complex Reward Function

May 3, 2024

Geometric Fabrics: a Safe Guiding Medium for Policy Learning
Karl Van Wyk, Ankur Handa, Viktor Makoviychuk, Yijie Guo, Arthur Allshire, Nathan D. Ratliff
Reinforcement Learning Nonlinear Model Geometric Fabric Complex Reward Function

January 10, 2024

Experiment Planning with Function Approximation
Aldo Pacchiano, Jonathan N. Lee, Emma Brunskill
Experimental Design Adaptive Learning Function Approximation Contextual Bandit Problem Complex Reward Function

August 30, 2023

Iterative Reward Shaping using Human Feedback for Correcting Reward Misspecification
Jasmina Gajcin, James McCarthy, Rahul Nair, Radu Marinescu, Elizabeth Daly, Ivana Dusparic
Reinforcement Learning Human Feedback Reward Shaping Complex Reward Function Reward Misspecification

March 28, 2022

Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions
Alejandro Escontrela, Xue Bin Peng, Wenhao Yu, Tingnan Zhang, Atil Iscen, Ken Goldberg, Pieter Abbeel
Interpretable Reward Substitute Model Complex Reward Function

February 25, 2022

Context-Hierarchy Inverse Reinforcement Learning
Wei Gao, David Hsu, Wee Sun Lee
Reward Function Inverse Reinforcement Learning Complex Reward Function

November 18, 2021

Assisted Robust Reward Design
Jerry Zhi-Yang He, Anca D. Dragan
Reward Design Reward Function Design Robust Reward Complex Reward Function Design Flexibility