Value Function

Value functions, central to reinforcement learning and optimal control, estimate the expected cumulative reward from a given state or state-action pair, guiding agents towards optimal behavior. Current research focuses on improving value function approximation accuracy and stability, particularly using neural networks (including shallow ReLU networks and transformers), and developing algorithms that address challenges like offline learning, multi-task optimization, and robustness to noise and uncertainty. These advancements are crucial for enhancing the efficiency and reliability of reinforcement learning agents in diverse applications, from robotics and autonomous systems to personalized recommendations and safe AI.

Papers

April 22, 2022

Making Parameterization and Constrains of Object Landmark Globally Consistent via SPD(3) Manifold and Improved Cost Functions
Yutong Hu, Wei Wang
Value Function Dimensional Manifold Object Landmark Artificial Landmark Object SLAM Landmark Matching

March 28, 2022

Revisiting Model-based Value Expansion
Daniel Palenicek, Michael Lutter, Jan Peters
Model Based Reinforcement Learning Value Function Value Expansion

March 7, 2022

Fast and Data Efficient Reinforcement Learning from Pixels via Non-Parametric Value Approximation
Alexander Long, Alan Blair, Herke van Hoof
Reinforcement Learning Algorithm Value Function Tetromino Pixel Data Efficient Episodic Reinforcement Learning Polygonal Environment Lazy Learning

March 3, 2022

Optimized cost function for demand response coordination of multiple EV charging stations using reinforcement learning
Manu Lahariya, Nasrin Sadeghianpourhamami, Chris Develder
Reinforcement Learning Markov Decision Process Value Function Electric Vehicle Demand Response Reinforcement Learning Policy Optimal Service Station Design

March 2, 2022

Follow your Nose: Using General Value Functions for Directed Exploration in Reinforcement Learning
Durgesh Kalwar, Omkar Shelke, Somjit Nath, Hardik Meisheri, Harshad Khadilkar
Reinforcement Learning Value Function Sparse Reward Exploration Method Exploration Strategy Directed Exploration

February 24, 2022

Threading the Needle of On and Off-Manifold Value Functions for Shapley Explanations
Chih-Kuan Yeh, Kuan-Yun Lee, Frederick Liu, Pradeep Ravikumar
Shapley Value Value Function Needle Insertion Manifold Value Function

February 23, 2022

Training Characteristic Functions with Reinforcement Learning: XAI-methods play Connect Four
Stephan Wäldchen, Felix Huber, Sebastian Pokutta
Reinforcement Learning Explainable AI XAI Method Value Function Attribution Method Counterfactual Reasoning New Connection Saliency Explanation Cooperative Game Theory

February 21, 2022

MCMARL: Parameterizing Value Function via Mixture of Categorical Distributions for Multi-Agent Reinforcement Learning
Jian Zhao, Mingyu Yang, Youpeng Zhao, Xunhan Hu, Wengang Zhou, Jiangcheng Zhu, Houqiang Li
Multi Agent Reinforcement Learning Multi Agent Mixture Component Value Function Stochastic Game Categorical Distribution

February 10, 2022

Augmenting Neural Networks with Priors on Function Values
Hunter Nisonoff, Yixin Wang, Jennifer Listgarten
Neural Network Bayesian Neural Network Epistemic Uncertainty Value Function Prior Knowledge SAM Prior

January 29, 2022

On Polynomial Approximation of Activation Function
John Chiang
Natural Gradient Activation Function Value Function Least Square Polynomial System Polynomial Approximation

January 28, 2022

Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error
Scott Fujimoto, David Meger, Doina Precup, Ofir Nachum, Shixiang Shane Gu
Value Function State Action Pair Bellman Equation Bellman Operator Bellman Error Value Prediction

January 25, 2022

Differentially Private Temporal Difference Learning with Stochastic Nonconvex-Strongly-Concave Optimization
Canzhe Zhao, Yanjie Ze, Jing Dong, Baoxiang Wang, Shuai Li
Reinforcement Learning Differential Privacy Value Function Temporal Difference Learning

January 21, 2022

Tensor and Matrix Low-Rank Value-Function Approximation in Reinforcement Learning
Sergio Rozada, Santiago Paternain, Antonio G. Marques
Reinforcement Learning Value Function Tensor Type Calculus Terminator Economy Parsimonious Machine Learning Online Tensor

January 17, 2022

Chaining Value Functions for Off-Policy Learning
Simon Schmitt, John Shawe-Taylor, Hado van Hasselt
Reinforcement Learning Policy Learning Value Function Policy Prediction

January 11, 2022

January 5, 2022

A Generalized Bootstrap Target for Value-Learning, Efficiently Combining Value and Feature Predictions
Anthony GX-Chen, Veronica Chelu, Blake A. Richards, Joelle Pineau
Value Function Net Present Value Parametric Bootstrap Predictive Feature Value Prediction

January 4, 2022

Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy Gradients
Hanhan Zhou, Tian Lan, Vaneet Aggarwal
Value Function Latent State Soft Actor Critic Parameter Sharing Value Function Factorization Decentralized Policy Gradient

December 21, 2021

Point spread function estimation for blind image deblurring problems based on framelet transform
Reza Parvaz
Regularization Model Value Function Exceptional Point Related Problem Point Spread Function Framelet Transform Blind Image

December 8, 2021

Model-Value Inconsistency as a Signal for Epistemic Uncertainty
Angelos Filos, Eszter Vértes, Zita Marinho, Gregory Farquhar, Diana Borsa, Abram Friesen, Feryal Behbahani, Tom Schaul, André Barreto, Simon Osindero
Model Based Reinforcement Learning Epistemic Uncertainty Value Function Abnormal Signal Value Based Consistent Model

Value Function

Papers

Making Parameterization and Constrains of Object Landmark Globally Consistent via SPD(3) Manifold and Improved Cost Functions

Revisiting Model-based Value Expansion

Fast and Data Efficient Reinforcement Learning from Pixels via Non-Parametric Value Approximation

Optimized cost function for demand response coordination of multiple EV charging stations using reinforcement learning

Follow your Nose: Using General Value Functions for Directed Exploration in Reinforcement Learning

Threading the Needle of On and Off-Manifold Value Functions for Shapley Explanations

Training Characteristic Functions with Reinforcement Learning: XAI-methods play Connect Four

MCMARL: Parameterizing Value Function via Mixture of Categorical Distributions for Multi-Agent Reinforcement Learning

Augmenting Neural Networks with Priors on Function Values

On Polynomial Approximation of Activation Function

Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error

Differentially Private Temporal Difference Learning with Stochastic Nonconvex-Strongly-Concave Optimization

Tensor and Matrix Low-Rank Value-Function Approximation in Reinforcement Learning

Chaining Value Functions for Off-Policy Learning

A Negotiating Strategy for a Hybrid Goal Function in Multilateral Negotiation

Pavlovian Signalling with General Value Functions in Agent-Agent Temporal Decision Making

A Generalized Bootstrap Target for Value-Learning, Efficiently Combining Value and Feature Predictions

Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy Gradients

Point spread function estimation for blind image deblurring problems based on framelet transform

Model-Value Inconsistency as a Signal for Epistemic Uncertainty