Informative Reward

Informative reward design in reinforcement learning (RL) focuses on creating reward functions that effectively guide agents towards desired behaviors, particularly in complex or sparsely rewarded environments. Current research emphasizes methods for automatically learning reward functions from various data sources, including passive video demonstrations and human feedback, often employing techniques like adversarial imitation learning, contrastive rewards, and preference learning. These advancements aim to reduce the reliance on hand-crafted rewards, improving the efficiency and robustness of RL algorithms across diverse applications, such as robotics and large language model training. The ultimate goal is to create more efficient and reliable RL systems by improving the informativeness and adaptability of the reward signal.

Papers

April 23, 2024

Rank2Reward: Learning Shaped Reward Functions from Passive Video
Daniel Yang, Davin Tjia, Jacob Berg, Dima Damen, Pulkit Agrawal, Abhishek Gupta
Reinforcement Learning Reward Function Robot Learning Reward Shaping Informative Reward

March 12, 2024

Improving Reinforcement Learning from Human Feedback Using Contrastive Rewards
Wei Shen, Xiaoying Zhang, Yuanshun Yao, Rui Zheng, Hongyi Guo, Yang Liu
Reinforcement Learning Human Feedback Informative Reward Contrastive Reward

February 10, 2024

Informativeness of Reward Functions in Reinforcement Learning
Rati Devidze, Parameswaran Kamalaruban, Adish Singla
Reinforcement Learning Reward Function Informative Reward

December 6, 2022

Few-Shot Preference Learning for Human-in-the-Loop RL
Joey Hejna, Dorsa Sadigh
Reinforcement Learning Multi Task Learning Reward Function Preference Learning Human in the Loop Reinforcement Informative Reward

November 3, 2022

Sensor Control for Information Gain in Dynamic, Sparse and Partially Observed Environments
J. Brian Burns, Aravind Sundaresan, Pedro Sequeira, Vidyasagar Sadhu
Reinforcement Learning Many Sparse Predictive Coding Information Gain Observable Environment Non Stationary Environment Autonomous Control Informative Reward

October 20, 2022

Task Phasing: Automated Curriculum Learning from Demonstrations
Vaibhav Bajaj, Guni Sharon, Peter Stone
Reinforcement Learning Reinforcement Learning Algorithm Inverse Reinforcement Learning Curriculum Learning Sparse Reward Noisy Demonstration Informative Reward

September 6, 2022

Multi-Armed Bandits with Self-Information Rewards
Nir Weinberger, Michal Yemini
Multi Armed Bandit Shannon Entropy UCB Algorithm Entropy Estimation Informative Reward

November 23, 2021

Object Recognition by a Minimally Pre-Trained System in the Process of Environment Exploration
Dmitry Maximov, Sekou A. K. Diane
Environment Exploration Object Recognition Complex Process Informative Reward Cognition Mechanism