Value Function

Value functions, central to reinforcement learning and optimal control, estimate the expected cumulative reward from a given state or state-action pair, guiding agents towards optimal behavior. Current research focuses on improving value function approximation accuracy and stability, particularly using neural networks (including shallow ReLU networks and transformers), and developing algorithms that address challenges like offline learning, multi-task optimization, and robustness to noise and uncertainty. These advancements are crucial for enhancing the efficiency and reliability of reinforcement learning agents in diverse applications, from robotics and autonomous systems to personalized recommendations and safe AI.

Papers

June 4, 2022

Hybrid Value Estimation for Off-policy Evaluation and Offline Reinforcement Learning
Xue-Kun Jin, Xu-Hui Liu, Shengyi Jiang, Yang Yu
Offline Reinforcement Learning Policy Evaluation Value Function

May 25, 2022

QGNN: Value Function Factorisation with Graph Neural Networks
Ryan Kortvelesy, Amanda Prorok
Graph Neural Network Multi Agent Reinforcement Learning Value Function Value Function Factorization Quantum Graph Neural Network

May 18, 2022

World Value Functions: Knowledge Representation for Multitask Reinforcement Learning
Geraud Nangue Tasse, Steven James, Benjamin Rosman
Value Function Knowledge Representation Task Learning Goal Oriented Task

May 17, 2022

Robust Losses for Learning Value Functions
Andrew Patterson, Victor Liao, Martha White
Reinforcement Learning Value Function Saddle Point Gradient Update Robust Loss Policy Prediction

April 26, 2022

Learning Value Functions from Undirected State-only Experience
Matthew Chang, Arjun Gupta, Saurabh Gupta
Markov Decision Process Q Learning Value Function eXperience Report Latent Action Unknown Out of Domain Goal Directed Behavior

April 22, 2022

Making Parameterization and Constrains of Object Landmark Globally Consistent via SPD(3) Manifold and Improved Cost Functions
Yutong Hu, Wei Wang
Value Function Dimensional Manifold Object Landmark Artificial Landmark Object SLAM Landmark Matching

March 28, 2022

Revisiting Model-based Value Expansion
Daniel Palenicek, Michael Lutter, Jan Peters
Model Based Reinforcement Learning Value Function Value Expansion

March 7, 2022

Fast and Data Efficient Reinforcement Learning from Pixels via Non-Parametric Value Approximation
Alexander Long, Alan Blair, Herke van Hoof
Reinforcement Learning Algorithm Value Function Tetromino Pixel Data Efficient Episodic Reinforcement Learning Polygonal Environment Lazy Learning

March 3, 2022

Optimized cost function for demand response coordination of multiple EV charging stations using reinforcement learning
Manu Lahariya, Nasrin Sadeghianpourhamami, Chris Develder
Reinforcement Learning Markov Decision Process Value Function Electric Vehicle Demand Response Reinforcement Learning Policy Optimal Service Station Design

March 2, 2022

Follow your Nose: Using General Value Functions for Directed Exploration in Reinforcement Learning
Durgesh Kalwar, Omkar Shelke, Somjit Nath, Hardik Meisheri, Harshad Khadilkar
Reinforcement Learning Value Function Sparse Reward Exploration Method Exploration Strategy Directed Exploration

February 24, 2022

Threading the Needle of On and Off-Manifold Value Functions for Shapley Explanations
Chih-Kuan Yeh, Kuan-Yun Lee, Frederick Liu, Pradeep Ravikumar
Shapley Value Value Function Needle Insertion Manifold Value Function

February 23, 2022

Training Characteristic Functions with Reinforcement Learning: XAI-methods play Connect Four
Stephan Wäldchen, Felix Huber, Sebastian Pokutta
Reinforcement Learning Explainable AI XAI Method Value Function Attribution Method Counterfactual Reasoning New Connection Cooperative Game Theory Saliency Explanation

February 21, 2022

MCMARL: Parameterizing Value Function via Mixture of Categorical Distributions for Multi-Agent Reinforcement Learning
Jian Zhao, Mingyu Yang, Youpeng Zhao, Xunhan Hu, Wengang Zhou, Jiangcheng Zhu, Houqiang Li
Multi Agent Reinforcement Learning Multi Agent Mixture Component Value Function Stochastic Game Categorical Distribution

February 10, 2022

Augmenting Neural Networks with Priors on Function Values
Hunter Nisonoff, Yixin Wang, Jennifer Listgarten
Neural Network Bayesian Neural Network Epistemic Uncertainty Value Function Prior Knowledge SAM Prior

January 29, 2022

On Polynomial Approximation of Activation Function
John Chiang
Natural Gradient Activation Function Value Function Least Square Polynomial System Polynomial Approximation

January 28, 2022

Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error
Scott Fujimoto, David Meger, Doina Precup, Ofir Nachum, Shixiang Shane Gu
Value Function State Action Pair Bellman Equation Bellman Operator Bellman Error Value Prediction

January 25, 2022

Differentially Private Temporal Difference Learning with Stochastic Nonconvex-Strongly-Concave Optimization
Canzhe Zhao, Yanjie Ze, Jing Dong, Baoxiang Wang, Shuai Li
Reinforcement Learning Differential Privacy Value Function Temporal Difference Learning

January 21, 2022

Tensor and Matrix Low-Rank Value-Function Approximation in Reinforcement Learning
Sergio Rozada, Santiago Paternain, Antonio G. Marques
Reinforcement Learning Value Function Tensor Type Calculus Terminator Economy Parsimonious Machine Learning Online Tensor

January 17, 2022

Chaining Value Functions for Off-Policy Learning
Simon Schmitt, John Shawe-Taylor, Hado van Hasselt
Reinforcement Learning Policy Learning Value Function Policy Prediction

January 11, 2022

A Negotiating Strategy for a Hybrid Goal Function in Multilateral Negotiation
Alon Stern, Sarit Kraus, David Sarne
Value Function Strategy Logic Iron Ore Negotiation Agent Negotiation Multi Issue Negotiation

Value Function

Papers

Hybrid Value Estimation for Off-policy Evaluation and Offline Reinforcement Learning

QGNN: Value Function Factorisation with Graph Neural Networks

World Value Functions: Knowledge Representation for Multitask Reinforcement Learning

Robust Losses for Learning Value Functions

Learning Value Functions from Undirected State-only Experience

Making Parameterization and Constrains of Object Landmark Globally Consistent via SPD(3) Manifold and Improved Cost Functions

Revisiting Model-based Value Expansion

Fast and Data Efficient Reinforcement Learning from Pixels via Non-Parametric Value Approximation

Optimized cost function for demand response coordination of multiple EV charging stations using reinforcement learning

Follow your Nose: Using General Value Functions for Directed Exploration in Reinforcement Learning

Threading the Needle of On and Off-Manifold Value Functions for Shapley Explanations

Training Characteristic Functions with Reinforcement Learning: XAI-methods play Connect Four

MCMARL: Parameterizing Value Function via Mixture of Categorical Distributions for Multi-Agent Reinforcement Learning

Augmenting Neural Networks with Priors on Function Values

On Polynomial Approximation of Activation Function

Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error

Differentially Private Temporal Difference Learning with Stochastic Nonconvex-Strongly-Concave Optimization

Tensor and Matrix Low-Rank Value-Function Approximation in Reinforcement Learning

Chaining Value Functions for Off-Policy Learning

A Negotiating Strategy for a Hybrid Goal Function in Multilateral Negotiation