Reward Design

Reward design in reinforcement learning focuses on crafting effective reward functions that guide agents towards desired behaviors, a crucial yet challenging aspect impacting learning efficiency and performance. Current research emphasizes robust reward design methods that account for uncertainties in agent behavior and environment models, often leveraging large language models (LLMs) to translate natural language descriptions or human feedback into reward functions, and exploring hierarchical or debate-based reward structures for improved interpretability and alignment with human values. These advancements are significant for improving the reliability and applicability of reinforcement learning across diverse domains, from autonomous driving and game AI to healthcare and robotics.

Papers

October 29, 2024

Democratizing Reward Design for Personal and Representative Value-Alignment
Carter Blair, Kate Larson, Edith Law
Value Alignment Reward Design Life Event Agent Alignment

October 17, 2024

ORSO: Accelerating Reward Design via Online Reward Selection and Policy Optimization
Chen Bo Calvin Zhang, Zhang-Wei Hong, Aldo Pacchiano, Pulkit Agrawal
Optimization Purpose Reward Function Reward Shaping Health Policy Recommendation Reward Design Reward Maximizing Policy

October 11, 2024

Language-Model-Assisted Bi-Level Programming for Reward Learning from Internet Videos
Harsh Mahesheka, Zhixian Xie, Zhaoran Wang, Wanxin Jin
Vision Language Model Bilevel Optimization Task Specific Model Reward Learning Complex Behavior Reward Design Online Video

June 7, 2024

June 3, 2024

REvolve: Reward Evolution with Large Language Models using Human Feedback
Rishi Hazra, Alkis Sygkounas, Andreas Persson, Amy Loutfi, Pedro Zuidberg Dos Martires
Autonomous Driving Reward Function Reward Structure Reward Design

February 24, 2024

Reward Design for Justifiable Sequential Decision-Making
Aleksa Sukovic, Goran Radanovic
Zero Sum Game Argumentation Theory Multi Agent Debate Reward Design

February 6, 2024

Informed Reinforcement Learning for Situation-Aware Traffic Rule Exceptions
Daniel Bogdoll, Jing Qin, Moritz Nekolla, Ahmed Abouelazm, Tim Joseph, J. Marius Zöllner
Reinforcement Learning Autonomous Driving Traffic Rule Reward Design Reward Function Design

January 23, 2024

Towards Socially and Morally Aware RL agent: Reward Design With LLM
Zhaoyue Wang
Large Language Model Language Model Reinforcement Learning Reward Function Reward Signal Social Interaction Reward Design

October 19, 2023

Eureka: Human-Level Reward Design via Coding Large Language Models
Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, Anima Anandkumar
Reinforcement Learning Reward Function Reward Design

September 6, 2023

Deep Reinforcement Learning from Hierarchical Preference Design
Alexander Bukharin, Yixiao Li, Pengcheng He, Tuo Zhao
Reinforcement Learning Deep Reinforcement Learning Reward Design

June 11, 2023

Multi-Agent Reinforcement Learning Guided by Signal Temporal Logic Specifications
Jiangwei Wang, Shuo Yang, Ziyan An, Songyang Han, Zhili Zhang, Rahul Mangharam, Meiyi Ma, Fei Miao
Deep Reinforcement Learning Multi Agent Reinforcement Learning Signal Temporal Logic Reward Design

February 27, 2023

Reward Design with Language Models
Minae Kwon, Sang Michael Xie, Kalesha Bullard, Dorsa Sadigh
Language Model Reinforcement Learning Reward Function Reward Design Proxy Reward Multi Issue Negotiation

February 5, 2023

Open Problems and Modern Solutions for Deep Reinforcement Learning
Weiqin Chen
Deep Reinforcement Learning Obstacle Avoidance Open Problem Reward Design Reward Function Design Deterministic Policy Gradient

January 24, 2023

Story Shaping: Teaching Agents Human-like Behavior with Stories
Xiangyu Peng, Christopher Cui, Wei Zhou, Renee Jia, Mark Riedl
Reinforcement Learning Agent BEHAVIOR Explanation Intrinsic Reward Written Story Reward Design Pedagogical Agent Narrative Designer Tacit Knowledge

September 9, 2022

Extending Open Bandit Pipeline to Simulate Industry Challenges
Bram van den Akker, Niklas Weber, Felipe Moraes, Dmitri Goldenberg
Concept Drift Bandit Algorithm Policy Optimization Bandit Model Reward Design

August 15, 2022

Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A. Murphy
Online Reinforcement Learning Reward Design Oral Self Care

July 29, 2022

Automatic Reward Design via Learning Motivation-Consistent Intrinsic Rewards
Yixiang Wang, Yujing Hu, Feng Wu, Yingfeng Chen
Intrinsic Reward Reward Design

May 30, 2022

Designing Rewards for Fast Learning
Henry Sowerby, Zhiyuan Zhou, Michael L. Littman
Reinforcement Learning Q Learning Reward Function Fast Learning Reward Design

April 11, 2022

Linguistic communication as (inverse) reward design
Theodore R. Sumers, Robert D. Hawkins, Mark K. Ho, Thomas L. Griffiths, Dylan Hadfield-Menell
Natural Language Language Supervision Inverse Task Reward Design Natural Language Communication Latent Speech Inverse Reward Design