Value Learning

Value learning, a core component of reinforcement learning, aims to estimate the long-term value of actions and states to guide optimal decision-making. Current research focuses on improving the accuracy and efficiency of value estimation, particularly addressing challenges in offline reinforcement learning where data quality is limited and generalization to unseen situations is crucial. This involves exploring novel algorithms for policy extraction and test-time policy improvement, as well as developing methods to mitigate issues like value overestimation and the impact of model errors in model-based approaches. Advances in value learning have significant implications for various applications, including multi-agent systems, robotics, and natural language processing, enabling more robust and efficient learning in complex environments.

Papers

June 13, 2024

Is Value Learning Really the Main Bottleneck in Offline RL?
Seohong Park, Kevin Frans, Sergey Levine, Aviral Kumar
Reinforcement Learning Imitation Learning Offline Reinforcement Learning Policy Learning Offline Reinforcement Learning Algorithm Major Challenge Bottleneck Value Learning

April 5, 2024

ROMA-iQSS: An Objective Alignment Approach via State-Based Value Learning and ROund-Robin Multi-Agent Scheduling
Chi-Hui Lin, Joewie J. Koh, Alessandro Roncone, Lijun Chen
Multi Agent Multi Agent Collaboration Multi Agent Interaction Alignment Objective Large Scale Multi Agent Decentralized Agent Value Learning Round Robin Tournament

February 19, 2024

The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning
Anya Sims, Cong Lu, Yee Whye Teh
Offline Reinforcement Learning Model Based Reinforcement Learning Online Reinforcement Learning Action Free Offline Learned Dynamic Value Learning

November 16, 2023

OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoning
Fei Yu, Anningzhe Gao, Benyou Wang
Task Planning Mathematical Reasoning Multi Step Reasoning Reasoning Path Value Learning Outcome Supervision

July 17, 2023

Meta-Value Learning: a General Framework for Learning with Learning Awareness
Tim Cooijmans, Milad Aghajohari, Aaron Courville
Multi Agent System Q Learning Action Space General Framework Matrix Game Value Learning Object Awareness

June 6, 2023

Mildly Constrained Evaluation Policy for Offline Reinforcement Learning
Linjie Xu, Zhengyao Jiang, Jinyu Wang, Lei Song, Jiang Bian
Offline Reinforcement Learning Value Learning

April 4, 2022

Value Gradient weighted Model-Based Reinforcement Learning
Claas Voelcker, Victor Liao, Animesh Garg, Amir-massoud Farahmand
Model Based Reinforcement Learning Model Learning Valuation Model Value Learning

February 20, 2022

Selective Credit Assignment
Veronica Chelu, Diana Borsa, Doina Precup, Hado van Hasselt
Credit Assignment Value Learning

December 12, 2021

ValueNet: A New Dataset for Human Value Driven Dialogue System
Liang Qiu, Yizhou Zhao, Jinchao Li, Pan Lu, Baolin Peng, Jianfeng Gao, Song-Chun Zhu
Empathetic Dialogue Human Value Value Learning