Reinforcement Learning Objective

Reinforcement learning (RL) objective design focuses on defining how agents learn optimal behaviors, balancing reward maximization with constraints like safety and robustness. Current research emphasizes developing objectives that handle multiple, potentially conflicting goals (multi-objective RL), incorporate risk awareness, and improve sample efficiency through techniques like transition occupancy matching and fine-grained reward modeling. These advancements are crucial for deploying RL in complex, real-world scenarios, particularly in safety-critical applications and for improving the efficiency and reliability of training large language models.

Papers

May 26, 2024

Safe and Balanced: A Framework for Constrained Multi-Objective Reinforcement Learning
Shangding Gu, Bilgehan Sel, Yuhao Ding, Lu Wang, Qingwei Lin, Alois Knoll, Ming Jin
New Framework Policy OpTimization Participation Constraint Multi Objective Reinforcement Learning Natural Policy Gradient Multi Objective Learning Reinforcement Learning Objective

May 20, 2024

Reward-Punishment Reinforcement Learning with Maximum Entropy
Jiexin Wang, Eiji Uchibe
Reinforcement Learning Softmax Function Maximum Entropy Reinforcement Learning Objective Behavior Policy

February 11, 2024

Future Prediction Can be a Strong Evidence of Good History Representation in Partially Observable Environments
Jeongyeol Kwon, Liu Yang, Robert Nowak, Josiah Hanna
Reinforcement Learning Representation Learning Evidence Piece Long Term Memory Observable Environment Future Prediction Reinforcement Learning Objective History Representation

January 11, 2024

Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint
Zhipeng Chen, Kun Zhou, Wayne Xin Zhao, Junchen Wan, Fuzheng Zhang, Di Zhang, Ji-Rong Wen
Reinforcement Learning Fine Grained Adversarial Reward Editing Model Reinforcement Learning Objective

October 25, 2023

Controlled Decoding from Language Models
Sidharth Mudgal, Jong Lee, Harish Ganapathy, YaGuang Li, Tao Wang, Yanping Huang, Zhifeng Chen, Heng-Tze Cheng, Michael Collins, Trevor Strohman, Jilin Chen, Alex Beutel, Ahmad Beirami
Language Model Reinforcement Learning Regularized Reinforcement Learning Compound Token Reinforcement Learning Objective

October 18, 2023

On The Expressivity of Objective-Specification Formalisms in Reinforcement Learning
Rohan Subramani, Marcus Williams, Max Heitmann, Halfdan Holm, Charlie Griffin, Joar Skalse
Reinforcement Learning Multi Objective Reinforcement Learning Behavior Expressivity Style Reinforcement Learning Objective Markovian Reward

September 12, 2023

Risk-Aware Reinforcement Learning through Optimal Transport Theory
Ali Baheri
Reinforcement Learning Optimal Transport Risk Aware Reward Maximization Reinforcement Learning Objective Risk Aware Reinforcement Learning

May 22, 2023

TOM: Learning Policy-Aware Models for Model-Based Reinforcement Learning via Transition Occupancy Matching
Yecheng Jason Ma, Kausik Sivakumar, Jason Yan, Osbert Bastani, Dinesh Jayaraman
LeArning Abstract Model Based Reinforcement Learning Continuous Control Task State Occupancy Matching Reinforcement Learning Objective

March 9, 2023

Computably Continuous Reinforcement-Learning Objectives are PAC-learnable
Cambridge Yang, Michael Littman, Michael Carbin
PAC Learnability Reinforcement Learning Objective Cumulative Reward

September 18, 2022

Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective
Raj Ghugare, Homanga Bharadhwaj, Benjamin Eysenbach, Sergey Levine, Ruslan Salakhutdinov
Reinforcement Learning Representation Learning Latent Space Model Free Reinforcement Learning Prior Policy Reinforcement Learning Objective

July 17, 2022

Reinforcement Learning For Survival, A Clinically Motivated Method For Critically Ill Patients
Thesath Nanayakkara
Reinforcement Learning Deep RL Intensive Care Deep Survival Optimal Treatment Reinforcement Learning Objective

June 17, 2022

AnyMorph: Learning Transferable Polices By Inferring Agent Morphology
Brandon Trabucco, Mariano Phielipp, Glen Berseth
Reinforcement Learning Mathematical Morphology Policy Transfer Reinforcement Learning Objective

June 7, 2022

Imitating Past Successes can be Very Suboptimal
Benjamin Eysenbach, Soumith Udatha, Sergey Levine, Ruslan Salakhutdinov
Reinforcement Learning Financial Success Reward Maximization Balancing Efficiency Reinforcement Learning Objective Conditioned Imitation

Reinforcement Learning Objective

Papers

Safe and Balanced: A Framework for Constrained Multi-Objective Reinforcement Learning

Reward-Punishment Reinforcement Learning with Maximum Entropy

Future Prediction Can be a Strong Evidence of Good History Representation in Partially Observable Environments

Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint

Controlled Decoding from Language Models

On The Expressivity of Objective-Specification Formalisms in Reinforcement Learning

Risk-Aware Reinforcement Learning through Optimal Transport Theory

TOM: Learning Policy-Aware Models for Model-Based Reinforcement Learning via Transition Occupancy Matching

Computably Continuous Reinforcement-Learning Objectives are PAC-learnable

Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective

Reinforcement Learning For Survival, A Clinically Motivated Method For Critically Ill Patients

AnyMorph: Learning Transferable Polices By Inferring Agent Morphology

Imitating Past Successes can be Very Suboptimal