Semi Supervised Reward

Semi-supervised reward learning aims to reduce the reliance on expensive, human-labeled data in training reward models for reinforcement learning. Current research focuses on leveraging unlabeled data through techniques like iterative self-training, pseudo-labeling, and data augmentation, often integrated with various model architectures including deep neural networks and differentiable decision trees. These methods improve the efficiency and scalability of reward learning, impacting diverse applications such as robotics, automated driving, and large language model alignment by enabling the development of more effective and human-aligned AI systems with less human intervention. Active learning strategies are also being explored to minimize the number of human queries needed for effective reward model training.

Papers

September 10, 2024

Semi-Supervised Reward Modeling via Iterative Self-Training
Yifei He, Haoxiang Wang, Ziyan Jiang, Alexandros Papangelis, Han Zhao
Large Language Model Self Training Reward Model Semi Supervised Reward

May 7, 2024

In-context Learning for Automated Driving Scenarios
Ziqi Zhou, Jingyue Zhang, Jingyuan Zhang, Boyue Wang, Tianyu Shi, Alaa Khamis
Context Learning Automated Driving Reinforcement Learning Agent Semi Supervised Reward Current Reinforcement Learning

May 1, 2024

Deep Reward Supervisions for Tuning Text-to-Image Diffusion Models
Xiaoshi Wu, Yiming Hao, Manyuan Zhang, Keqiang Sun, Zhaoyang Huang, Guanglu Song, Yu Liu, Hongsheng Li
Text to Image Diffusion Model Stable Diffusion Deep Supervision Reward Fine Tuning Semi Supervised Reward

October 4, 2023

SemiReward: A General Reward Model for Semi-supervised Learning
Siyuan Li, Weiyang Jin, Zedong Wang, Fang Wu, Zicheng Liu, Cheng Tan, Stan Z. Li
Semi Supervised Learning Pseudo Label Pseudo Label Selection Semi Supervised Reward Monetary Reward

June 22, 2023

Can Differentiable Decision Trees Enable Interpretable Reward Learning from Human Feedback?
Akansha Kalra, Daniel S. Brown
Human Feedback Reward Function Semi Supervised Reward Interpretable Reward Differentiable Decision Tree

June 2, 2023

PAGAR: Taming Reward Misalignment in Inverse Reinforcement Learning-Based Imitation Learning with Protagonist Antagonist Guided Adversarial Reward
Weichao Zhou, Wenchao Li
Imitation Learning Inverse Reinforcement Learning Adversarial Reward Hybrid Reward Semi Supervised Reward

April 18, 2023

Provably Feedback-Efficient Reinforcement Learning via Active Reward Learning
Dingwen Kong, Lin F. Yang
Reinforcement Learning Reward Function Reinforcement Learning From Human Feedback Semi Supervised Reward

March 2, 2023

Active Reward Learning from Multiple Teachers
Peter Barnett, Rachel Freedman, Justin Svegliato, Stuart Russell
Reward Function Artificial Intelligence System Feedback System Semi Supervised Reward Multiple Teacher

February 27, 2023

Active Reward Learning from Online Preferences
Vivek Myers, Erdem Bıyık, Dorsa Sadigh
Human Feedback Domain Knowledge Robot Policy Pairwise Preference User Preference Semi Supervised Reward

November 24, 2022

Discovering Generalizable Spatial Goal Representations via Graph-based Active Reward Learning
Aviv Netanyahu, Tianmin Shu, Joshua Tenenbaum, Pulkit Agrawal
Inverse Reinforcement Learning Goal Representation Semi Supervised Reward

November 7, 2022

Reward-Predictive Clustering
Lucas Lehnert, Michael J. Frank, Michael L. Littman
Reinforcement Learning Reward Model Reinforcement Learning System Semi Supervised Reward Predictive Clustering

March 18, 2022

SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learning
Jongjin Park, Younggyo Seo, Jinwoo Shin, Honglak Lee, Pieter Abbeel, Kimin Lee
Reinforcement Learning Data Augmentation Reward Function Reward Learning Preference Based Reinforcement Learning Semi Supervised Reward