Reward Estimator

Reward estimation focuses on learning accurate reward functions from often limited and noisy data, crucial for optimizing agent behavior in reinforcement learning. Current research emphasizes improving reward estimators by leveraging diverse feedback sources (e.g., human preferences, auxiliary data), incorporating second-order preference information, and employing advanced model architectures like neural networks and energy-based models to handle complex, non-linear relationships. These advancements aim to enhance the efficiency and robustness of reinforcement learning algorithms, particularly in applications like personalized recommendations and human-AI alignment, where precise reward modeling is paramount.

Papers

December 11, 2024

CANDOR: Counterfactual ANnotated DOubly Robust Off-Policy Evaluation
Aishwarya Mandyam, Shengpu Tang, Jiayu Yao, Jenna Wiens, Barbara E. Engelhardt
Policy Evaluation Truthful Space Policy Estimation Counterfactual Data Reward Estimator

October 3, 2024

Dual Active Learning for Reinforcement Learning from Human Feedback
Pangpang Liu, Chengchun Shi, Will Wei Sun
Large Language Model Reinforcement Learning Offline Reinforcement Learning Human Feedback Reward Report Dual Learning Reward Estimator

August 8, 2024

Listwise Reward Estimation for Offline Preference-based Reinforcement Learning
Heewoong Choi, Sangwon Jung, Hongjoon Ahn, Taesup Moon
Reinforcement Learning Reward Function Reward Model Second Order Offline Preference Based Reinforcement Learning Reward Estimator

May 31, 2024

Provably Efficient Interactive-Grounded Learning with Personalized Reward
Mengxiao Zhang, Yuheng Zhang, Haipeng Luo, Paul Mineiro
LeArning Abstract Grounded Language Page Wise Feedback Personalized Incentive Reward Estimator

November 5, 2023

Exploiting Correlated Auxiliary Feedback in Parameterized Bandits
Arun Verma, Zhongxiang Dai, Yao Shu, Bryan Kian Hsiang Low
Low Regret Reward Estimator

October 12, 2022

Maximum entropy exploration in contextual bandits with neural networks and energy based models
Adam Elwood, Marco Leonardi, Ashraf Mohamed, Alessandro Rozza
Neural Network Full Model Contextual Bandit Energy Policy Research Exploration Exploitation Trade Entropy Exploration Reward Estimator