Reward Shaping

Reward shaping in reinforcement learning aims to accelerate agent training and improve sample efficiency by modifying the reward function to provide more informative feedback. Current research focuses on developing methods that automatically generate or adapt reward functions, leveraging large language models and incorporating domain knowledge to guide the learning process, often within frameworks like potential-based shaping or by directly shaping Q-values. These advancements are significant because they address the challenge of designing effective reward functions, a critical bottleneck in applying reinforcement learning to complex real-world problems, leading to more efficient and robust agent training across diverse applications.

Papers

September 21, 2023

State2Explanation: Concept-Based Explanations to Benefit Agent Learning and User Understanding
Devleena Das, Sonia Chernova, Been Kim
Line by Line Explanation Learning Rate Reward Shaping Sequential Decision Making Concept Based Explanation Human AI Decision Making User Understanding Agent Learning

September 20, 2023

Text2Reward: Reward Shaping with Language Models for Reinforcement Learning
Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, Tao Yu
Language Model Reinforcement Learning Sparse Reward Reward Shaping Reward Function Design

August 30, 2023

Iterative Reward Shaping using Human Feedback for Correcting Reward Misspecification
Jasmina Gajcin, James McCarthy, Rahul Nair, Radu Marinescu, Elizabeth Daly, Ivana Dusparic
Reinforcement Learning Human Feedback Reward Shaping Reward Misspecification Complex Reward Function

August 2, 2023

Reward Shaping for Building Trustworthy Robots in Sequential Human-Robot Interaction
Yaohui Guo, X. Jessie Yang, Cong Shi
Human Robot Interaction Reward Shaping Human Robot Team Human Trust Trust Model Human Robot Trust Trustworthy Robot

July 19, 2023

Benchmarking Potential Based Rewards for Learning Humanoid Locomotion
Se Hwan Jeon, Steve Heim, Charles Khazoom, Sangbae Kim
Robotics Domain Reward Function Humanoid Robot Reward Shaping Humanoid Locomotion Effective Reinforcement Learning

July 16, 2023

Magnetic Field-Based Reward Shaping for Goal-Conditioned Reinforcement Learning
Hongyu Ding, Yuanze Tang, Qing Wu, Bo Wang, Chunlin Chen, Zhi Wang
Reward Shaping Goal Conditioned Reinforcement Learning Permanent Magnet

July 10, 2023

Loss Dynamics of Temporal Difference Reinforcement Learning
Blake Bordelon, Paul Masset, Henry Kuo, Cengiz Pehlevan
Reinforcement Learning Reward Shaping Temporal Difference Learning Temporal Difference Reinforcement Learning Model Gradient Descent Dynamic

June 20, 2023

Reward Shaping via Diffusion Process in Reinforcement Learning
Peeyush Kumar
Reinforcement Learning Markov Decision Process Reward Shaping Stochastic Dynamical System Maximum Entropy Diffusion Process Exploration Exploitation Trade

May 28, 2023

Potential-based Credit Assignment for Cooperative RL-based Testing of Autonomous Vehicles
Utku Ayvaz, Chih-Hong Cheng, Hao Shen
Reinforcement Learning Autonomous Vehicle Long Term Cooperation Reward Shaping Credit Assignment Collaborative Reinforcement Learning

March 25, 2023

Embedding Contextual Information through Reward Shaping in Multi-Agent Learning: A Case Study from Google Football
Chaoyi Gu, Varuna De Silva, Corentin Artaud, Rafael Pina
Multi Agent Reinforcement Learning Case Study Imitation Learning Sparse Reward Contextual Information Multi Agent Learning Reward Shaping Google Research Football

February 28, 2023

Exploiting Multiple Abstractions in Episodic RL via Reward Shaping
Roberto Cipollone, Giuseppe De Giacomo, Marco Favorito, Luca Iocchi, Fabio Patrizi
Reinforcement Learning Markov Decision Process Reward Shaping Active Exploration Episodic Reinforcement Learning Level Abstraction

February 8, 2023

Temporal Video-Language Alignment Network for Reward Shaping in Reinforcement Learning
Ziyuan Cao, Reshma Anugundanahalli Ramachandra, Kelin Yu
Reinforcement Learning Reward Shaping Language Reward Video Language Alignment

January 30, 2023

Transferring Multiple Policies to Hotstart Reinforcement Learning in an Air Compressor Management Problem
Hélène Plisnier, Denis Steckelmacher, Jeroen Willems, Bruno Depraetere, Ann Nowé
Reinforcement Learning Reinforcement Learning Agent Reward Shaping Diverse Policy Industrial Machine Policy Alignment Compressor Based Machine

January 26, 2023

Automatic Intrinsic Reward Shaping for Exploration in Deep Reinforcement Learning
Mingqi Yuan, Bo Li, Xin Jin, Wenjun Zeng
Reinforcement Learning Deep Reinforcement Learning Environment Exploration Intrinsic Reward Reward Shaping

January 5, 2023

Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping
Lina Mezghani, Sainbayar Sukhbaatar, Piotr Bojanowski, Alessandro Lazaric, Karteek Alahari
Reward Function Reward Shaping Long Horizon Task Goal Conditioned Policy Continuous Control Task Offline Goal Conditioned Reinforcement Learning

December 29, 2022

Backward Curriculum Reinforcement Learning
KyungMin Ko
Reinforcement Learning Reward Shaping Curriculum Reinforcement Learning

December 6, 2022

Reinforcement Learning for UAV control with Policy and Reward Shaping
Cristian Millán-Arias, Ruben Contreras, Francisco Cruz, Bruno Fernandes
Reinforcement Learning Unmanned Aerial Vehicle Autonomous Agent Reinforcement Learning Agent Reward Shaping Health Policy Recommendation Drone Control Interactive Reinforcement Learning

December 2, 2022

Utilizing Prior Solutions for Reward Shaping and Composition in Entropy-Regularized Reinforcement Learning
Jacob Adamczyk, Argenis Arriojas, Stas Tiomkin, Rahul V. Kulkarni
Reinforcement Learning Reward Function Prior Knowledge Reward Shaping Compositional Ability Entropy Regularized

November 30, 2022

Funnel-based Reward Shaping for Signal Temporal Logic Tasks in Reinforcement Learning
Naman Saxena, Gorantla Sandeep, Pushpak Jagtap
Reinforcement Learning State Space Reward Shaping Signal Temporal Logic

November 29, 2022

Configurable Agent With Reward As Input: A Play-Style Continuum Generation
Pierre Le Pelletier de Woillemont, Rémi Labory, Vincent Corruble
Reinforcement Learning Reward Report Reward Shaping Rich Input Play Style Agent Capability

Reward Shaping

Papers

State2Explanation: Concept-Based Explanations to Benefit Agent Learning and User Understanding

Text2Reward: Reward Shaping with Language Models for Reinforcement Learning

Iterative Reward Shaping using Human Feedback for Correcting Reward Misspecification

Reward Shaping for Building Trustworthy Robots in Sequential Human-Robot Interaction

Benchmarking Potential Based Rewards for Learning Humanoid Locomotion

Magnetic Field-Based Reward Shaping for Goal-Conditioned Reinforcement Learning

Loss Dynamics of Temporal Difference Reinforcement Learning

Reward Shaping via Diffusion Process in Reinforcement Learning

Potential-based Credit Assignment for Cooperative RL-based Testing of Autonomous Vehicles

Embedding Contextual Information through Reward Shaping in Multi-Agent Learning: A Case Study from Google Football

Exploiting Multiple Abstractions in Episodic RL via Reward Shaping

Temporal Video-Language Alignment Network for Reward Shaping in Reinforcement Learning

Transferring Multiple Policies to Hotstart Reinforcement Learning in an Air Compressor Management Problem

Automatic Intrinsic Reward Shaping for Exploration in Deep Reinforcement Learning

Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping

Backward Curriculum Reinforcement Learning

Reinforcement Learning for UAV control with Policy and Reward Shaping

Utilizing Prior Solutions for Reward Shaping and Composition in Entropy-Regularized Reinforcement Learning

Funnel-based Reward Shaping for Signal Temporal Logic Tasks in Reinforcement Learning

Configurable Agent With Reward As Input: A Play-Style Continuum Generation