Reward Function Parameter

Reward function parameterization is crucial in reinforcement learning, aiming to accurately represent the desired behavior of an agent, often learned from human feedback or expert demonstrations. Current research focuses on improving the efficiency and robustness of learning these parameters, employing methods like inverse reinforcement learning, Bayesian approaches, and policy optimization techniques, often incorporating gradient descent or genetic algorithms for optimization. These advancements are significant for improving the data efficiency and reliability of reinforcement learning systems across diverse applications, from personalized medicine to robotics and natural language processing. The ultimate goal is to develop methods that reliably learn accurate reward functions from limited, potentially noisy, human input.

Papers

September 20, 2024

OMG-RL:Offline Model-based Guided Reward Learning for Heparin Treatment
Yooseok Lim, Sujee Lee
Reinforcement Learning Model Based Offline Reinforcement Learning Optimal Treatment Reward Function Parameter Complex Reward Function Warfarin Dosing

March 9, 2024

A Generalized Acquisition Function for Preference-based Reward Learning
Evan Ellis, Gaurav R. Ghosal, Stuart J. Russell, Anca Dragan, Erdem Bıyık
Reward Function Reward Learning Information Gain Equivalence Class Reward Function Parameter

February 15, 2024

Exploration-Driven Policy Optimization in RLHF: Theoretical Insights on Efficient Data Utilization
Yihan Du, Anna Winnicki, Gal Dalal, Shie Mannor, R. Srikant
Reinforcement Learning Policy Gradient Policy OpTimization Theoretical Analysis Reinforcement Learning From Human Feedback Reward Function Parameter

August 19, 2023

StROL: Stabilized and Robust Online Learning from Humans
Shaunak A. Mehta, Forrest Meng, Andrea Bajcsy, Dylan P. Losey
Real Human Robust Learning Learning Dynamic Stable Learning Reward Function Parameter

May 16, 2023

Reward Learning with Intractable Normalizing Functions
Joshua Hoegerman, Dylan P. Losey
Non Humanoid Robot Reward Function Reward Learning Unified Normalization Reward Function Parameter Bayesian Reward Model

March 13, 2023

Kernel Density Bayesian Inverse Reinforcement Learning
Aishwarya Mandyam, Didong Li, Diana Cai, Andrew Jones, Barbara E. Engelhardt
Inverse Reinforcement Learning Reward Function Parameter

January 3, 2023

Genetic Imitation Learning by Reward Extrapolation
Boyuan Zheng, Jianlong Zhou, Fang Chen
Imitation Learning Genetic Algorithm Stochastic Policy Suboptimal Demonstration Evolutionary Learning Reward Function Parameter Reward Misspecification

October 19, 2022

Scaling Laws for Reward Model Overoptimization
Leo Gao, John Schulman, Jacob Hilton
Reinforcement Learning Scaling Law Proxy Reward Reward Overoptimization Reward Function Parameter

May 12, 2022

Collaborative Multi-agent Stochastic Linear Bandits
Ahmadreza Moradipari, Mohammad Ghavamzadeh, Mahnoosh Alizadeh
Linear Bandit Reward Function Parameter Efficient Consensus