Human Feedback

Human feedback is crucial for aligning artificial intelligence models, particularly large language models, with human preferences and values. Current research focuses on improving the efficiency and reliability of incorporating human feedback into reinforcement learning frameworks, exploring techniques like macro actions, active learning, and reward model optimization to address challenges such as the cost and subjectivity of human judgments. This work is significant because it directly impacts the safety, trustworthiness, and overall effectiveness of AI systems across diverse applications, from autonomous driving to educational assessment. The development of more robust and efficient methods for integrating human feedback is a key area of ongoing investigation.

Papers

November 13, 2023

Coffee: Boost Your Code LLMs by Fixing Bugs with Feedback
Seungjun Moon, Hyungjoo Chae, Yongho Song, Taeyoon Kwon, Dongjin Kang, Kai Tzu-iunn Ong, Seung-won Hwang, Jinyoung Yeo
Human Feedback Open Source LLM Bug Report Code LLM Self Service CAFE Code Editing

November 4, 2023

Accelerating Reinforcement Learning of Robotic Manipulations via Feedback from Large Language Models
Kun Chu, Xufeng Zhao, Cornelius Weber, Mengdi Li, Stefan Wermter
Language Model Reinforcement Learning Human Feedback Manipulation Task Robot Manipulation Interactive Reinforcement Learning

November 2, 2023

The Impact of Preference Agreement in Reinforcement Learning from Human Feedback: A Case Study in Summarization
Sian Gooding, Hassan Mansoor
Reinforcement Learning Case Study Language Generation Human Feedback Structured Summary Text Summarization Preference Feedback Reward Estimation Accuracy

October 31, 2023

The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedback
Nathan Lambert, Roberto Calandra
Reinforcement Learning Human Feedback Model Based Reinforcement Learning Reward Model Human Alignment Model Mismatch Reward Model Training

October 29, 2023

Reward Finetuning for Faster and More Accurate Unsupervised Object Discovery
Katie Z Luo, Zhenzhen Liu, Xiangyu Chen, Yurong You, Sagie Benaim, Cheng Perng Phoo, Mark Campbell, Wen Sun, Bharath Hariharan, Kilian Q. Weinberger
Reinforcement Learning Autonomous Vehicle Human Feedback Reward Function Reward Report Object Discovery

October 25, 2023

October 24, 2023

October 23, 2023

Active teacher selection for reinforcement learning from human feedback
Rachel Freedman, Justin Svegliato, Kyle Wray, Stuart Russell
Reinforcement Learning Human Feedback Reward Model Robust Reward Teacher Selection

October 20, 2023

October 19, 2023

Safe RLHF: Safe Reinforcement Learning from Human Feedback
Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, Yaodong Yang
Large Language Model Human Feedback Safe Reinforcement Learning RLHF Training Value Aligned

October 18, 2023

Quality Diversity through Human Feedback: Towards Open-Ended Diversity-Driven Optimization
Li Ding, Jenny Zhang, Jeff Clune, Lee Spector, Joel Lehman
Reinforcement Learning Human Feedback Open Ended Quality Diversity Diversity Metric Diversity Maximization

October 16, 2023

Verbosity Bias in Preference Labeling by Large Language Models
Keita Saito, Akifumi Wachi, Koki Wataoka, Youhei Akimoto
Natural Language Processing Human Feedback AI Feedback Simplicity Bias Preference Label

October 11, 2023

October 10, 2023

Diversity from Human Feedback
Ren-Jian Wang, Ke Xue, Yutong Wang, Peng Yang, Haobo Fu, Qiang Fu, Chao Qian
Human Feedback Diversity Awareness Behavior Space Behavior Descriptor Diversity Score