Task Specific Reward

Task-specific reward design in reinforcement learning (RL) aims to create reward functions that effectively guide agents towards desired behaviors in complex environments, overcoming challenges like sparse or ambiguous rewards. Current research focuses on leveraging large language models (LLMs) and vision-language models (VLMs) to automatically generate or refine reward functions, often incorporating techniques like reward shaping, inverse reinforcement learning, and power regularization to improve efficiency and robustness. These advancements are significant because they reduce the reliance on manual reward engineering, enabling more efficient and adaptable RL agents for diverse applications, including robotics and natural language processing.

Papers

October 14, 2024

FairMindSim: Alignment of Behavior, Emotion, and Belief in Humans and LLM Agents Amid Ethical Dilemmas
Yu Lei, Hao Liu, Chengxing Xie, Songjia Liu, Zhiyu Yin, Canyu chen, Guohao Li, Philip Torr, Zhen Wu
Alignment Problem Underlying Emotion BEHAVIOR Explanation AI Alignment Task Specific Reward Altruistic Behavior

October 11, 2024

Automated Rewards via LLM-Generated Progress Functions
Vishnu Sarukkai, Brennan Shacklett, Zander Majercik, Kush Bhatia, Christopher Ré, Kayvon Fatahalian
Reward Function Intrinsic Reward Task Specific Reward Progress Monitoring

September 4, 2024

Large Language Models as Efficient Reward Function Searchers for Custom-Environment Multi-Objective Reinforcement Learning
Guanwen Xie, Jingzehua Xu, Yiyuan Yang, Yimian Ding, Shuai Zhang
Reinforcement Learning Reward Function Multi Objective Reinforcement Learning Reward Learning Task Specific Reward

July 14, 2024

Affordance-Guided Reinforcement Learning via Visual Prompting
Olivia Y. Lee, Annie Xie, Kuan Fang, Karl Pertsch, Chelsea Finn
Reinforcement Learning Robot Learning Visual Prompting Task Specific Reward Real World Manipulation Task

July 5, 2024

Efficient Controlled Language Generation with Low-Rank Autoregressive Reward Models
Sergey Troshin, Vlad Niculae, Antske Fokkens
Language Model High Efficiency Accurate Decoding Best Arm Task Specific Reward

June 17, 2024

The Benefits of Power Regularization in Cooperative Reinforcement Learning
Michelle Li, Michael Dennis
Reinforcement Learning Multi Agent Reinforcement Learning Complementary Benefit Cooperative Multi Agent Reinforcement Learning Power Allocation Task Specific Reward Power Regularization

May 16, 2024

Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
Yuexiang Zhai, Hao Bai, Zipeng Lin, Jiayi Pan, Shengbang Tong, Yifei Zhou, Alane Suhr, Saining Xie, Yann LeCun, Yi Ma, Sergey Levine
Reinforcement Learning Large Vision Language Model Decision Making Agent Task Specific Reward Language Reasoning

March 11, 2024

Generalising Multi-Agent Cooperation through Task-Agnostic Communication
Dulhan Jayalath, Steven Morad, Amanda Prorok
Multi Agent Reinforcement Learning Latent State Multi Agent Cooperation Task Specific Reward Task Oriented Communication Order Markov

February 22, 2024

Q-Probe: A Lightweight Approach to Reward Maximization for Language Models
Kenneth Li, Samy Jelassi, Hugh Zhang, Sham Kakade, Martin Wattenberg, David Brandfonbrener
Language Model Policy Gradient Linear Probing Reward Maximization Task Specific Reward Direct Policy Probe Motion

February 17, 2024

Multi Task Inverse Reinforcement Learning for Common Sense Reward
Neta Glazer, Aviv Navon, Aviv Shamsian, Ethan Fetaya
Reinforcement Learning Reward Function Inverse Reinforcement Learning Task Specific Reward

October 18, 2023

Understanding Reward Ambiguity Through Optimal Transport Theory in Inverse Reinforcement Learning
Ali Baheri
Optimal Transport Reward Function Inverse Reinforcement Learning Task Specific Reward Reward Ambiguity

October 9, 2023

DSAC-T: Distributional Soft Actor-Critic with Three Refinements
Jingliang Duan, Wenxuan Wang, Liming Xiao, Jiaxin Gao, Shengbo Eben Li
Reinforcement Learning Policy Reinforcement Learning Model Free Reinforcement Learning Task Specific Reward

May 26, 2023

Physical Deep Reinforcement Learning: Safety and Unknown Unknowns
Hongpeng Cao, Yanbing Mao, Lui Sha, Marco Caccamo
Reinforcement Learning Human SAFETY Physic Based Task Specific Reward Unknown Unknown Data Driven Control Critical Autonomous System

April 25, 2023

Fulfilling Formal Specifications ASAP by Model-free Reinforcement Learning
Mengyu Liu, Pengyuan Lu, Xin Chen, Fanxin Kong, Oleg Sokolsky, Insup Lee
Model Free Reinforcement Learning Actor Critic Algorithm Deep Deterministic Policy Gradient Task Specific Reward Formal Specification Phi 3

March 30, 2023

Language Models can Solve Computer Tasks
Geunwoo Kim, Pierre Baldi, Stephen McAleer
Language Model Reinforcement Learning Natural Language Command Task Specific Reward Language Reasoning Task

March 19, 2023

CLIP4MC: An RL-Friendly Vision-Language Model for Minecraft
Ziluo Ding, Hao Luo, Ke Li, Junpeng Yue, Tiejun Huang, Zongqing Lu
Reinforcement Learning Vision Language Model Reward Signal Task Specific Reward Motion Embeddings

February 23, 2023

To the Noise and Back: Diffusion for Shared Autonomy
Takuma Yoneda, Luzhe Sun, and Ge Yang, Bradly Stadie, Matthew Walter
Diffusion Explainer Industrial Disturbing Noise Autonomous Agent Legal Autonomy Model Free Deep Reinforcement Learning Task Specific Reward Shared Autonomy Direct Teleoperation

February 16, 2023

Tuning computer vision models with task rewards
André Susano Pinto, Alexander Kolesnikov, Yuge Shi, Lucas Beyer, Xiaohua Zhai
Reinforcement Learning Vision Task Computer Vision Task Computer Vision Model Model Alignment Task Specific Reward

December 21, 2022

Reward Bonuses with Gain Scheduling Inspired by Iterative Deepening Search
Taisuke Kobayashi
Sparse Reward Reward Report Task Specific Reward Iterative Neural Network Learning Search Automatic Gain Control

December 7, 2022

Tiered Reward Functions: Specifying and Fast Learning of Desired Behavior
Zhiyuan Zhou, Shreyas Sundara Raman, Henry Sowerby, Michael L. Littman
Reward Function Reward Report Fast Learning Task Specific Reward Reward Function Design