Reward Consistency

Reward consistency in reinforcement learning (RL) focuses on ensuring that the reward signal accurately and reliably guides an agent's learning, avoiding inconsistencies that hinder performance and interpretability. Current research emphasizes developing methods to improve reward model consistency, particularly in RL from human feedback (RLHF), through techniques like curriculum learning and intrinsic reward shaping that encourage exploration and avoid redundant actions. Addressing reward inconsistencies is crucial for improving the reliability and explainability of RL agents, ultimately leading to more robust and trustworthy AI systems across various applications.

Papers

October 30, 2023

Diversify & Conquer: Outcome-directed Curriculum RL via Out-of-Distribution Disagreement
Daesol Cho, Seungjae Lee, H. Jin Kim
Reinforcement Learning Curriculum Reinforcement Learning Expert Diversification Reward Consistency

September 28, 2023

The Trickle-down Impact of Reward (In-)consistency on RLHF
Lingfeng Shen, Sihao Chen, Linfeng Song, Lifeng Jin, Baolin Peng, Haitao Mi, Daniel Khashabi, Dong Yu
Reinforcement Learning Strong Consistency Reward Model Reinforcement Learning From Human Feedback Reward Report Spillover Effect Reward Consistency

September 4, 2023

Leveraging Reward Consistency for Interpretable Feature Discovery in Reinforcement Learning
Qisen Yang, Huanqian Wang, Mukun Tong, Wenjie Shi, Gao Huang, Shiji Song
Reinforcement Learning Deep Reinforcement Learning Vision Based Reinforcement Learning Action Principle Reward Consistency

August 30, 2023

Cyclophobic Reinforcement Learning
Stefan Sylvius Wagner, Peter Arndt, Jan Robine, Stefan Harmeling
Reinforcement Learning Intrinsic Reward Active Exploration Curiosity Driven Exploration Reward Consistency

June 1, 2023

Active Vision Reinforcement Learning under Limited Visual Observability
Jinghuan Shang, Michael S. Ryoo
Full Observability Active Vision Active Reinforcement Learning Reward Consistency

Reward Consistency

Papers

Diversify & Conquer: Outcome-directed Curriculum RL via Out-of-Distribution Disagreement

The Trickle-down Impact of Reward (In-)consistency on RLHF

Leveraging Reward Consistency for Interpretable Feature Discovery in Reinforcement Learning

Cyclophobic Reinforcement Learning

Active Vision Reinforcement Learning under Limited Visual Observability