Value Function

Value functions, central to reinforcement learning and optimal control, estimate the expected cumulative reward from a given state or state-action pair, guiding agents towards optimal behavior. Current research focuses on improving value function approximation accuracy and stability, particularly using neural networks (including shallow ReLU networks and transformers), and developing algorithms that address challenges like offline learning, multi-task optimization, and robustness to noise and uncertainty. These advancements are crucial for enhancing the efficiency and reliability of reinforcement learning agents in diverse applications, from robotics and autonomous systems to personalized recommendations and safe AI.

Papers

December 1, 2021

Robust and Adaptive Temporal-Difference Learning Using An Ensemble of Gaussian Processes
Qin Lu, Georgios B. Giannakis
Reinforcement Learning Gaussian Process Robust Version Diverse Ensemble Value Function Temporal Difference Learning Trajectory Wise Reward State Reward

November 21, 2021

Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation
Dylan J. Foster, Akshay Krishnamurthy, David Simchi-Levi, Yunzong Xu
Reinforcement Learning Offline Reinforcement Learning Value Function Knowledge Barrier Sample Efficient Offline Reinforcement Learning

November 18, 2021

Finding Useful Predictions by Meta-gradient Descent to Improve Decision-making
Alex Kearney, Anna Koop, Johannes Günther, Patrick M. Pilarski
Human Prediction Decision Making Autonomous System Value Function Full Observability Meta Gradient

November 12, 2021

A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes
Chengchun Shi, Masatoshi Uehara, Jiawei Huang, Nan Jiang
Markov Decision Process Policy Evaluation Value Function Minimax Optimal

November 10, 2021

Search in Imperfect Information Games
Martin Schmid
Search Query Value Function Imperfect Information Game Chess Engine Game Research

May 19, 2021

Multiply Robust Causal Mediation Analysis with Continuous Treatments
Yizhen Xu, Numair Sani, AmirEmad Ghassami, Ilya Shpitser
Causal Effect Value Function Influence Function Causal Mediation Analysis Valued Treatment