State Action Value

State-action value (Q-value) estimation is central to reinforcement learning, aiming to accurately predict the expected cumulative reward for taking a specific action in a given state. Current research focuses on improving Q-value estimation efficiency and robustness, particularly in high-dimensional spaces, by exploring techniques like pessimism/optimism control in actor-critic methods, developing alternative approaches that avoid explicit state-action value function representation, and employing methods to mitigate estimation bias and uncertainty. These advancements are crucial for enabling reliable and efficient reinforcement learning in complex real-world applications, such as robotics and personalized recommendations, where accurate value estimation is paramount for optimal decision-making.

Papers

June 6, 2024

Exploring Pessimism and Optimism Dynamics in Deep Reinforcement Learning
Bahareh Tasdighi, Nicklas Werge, Yi-Shan Wu, Melih Kandemir
Deep Reinforcement Learning Policy Actor Critic State Action Value Optimism Bias

March 7, 2024

Vlearn: Off-Policy Learning with Efficient State-Value Function Estimation
Fabian Otto, Philipp Becker, Ngo Anh Vien, Gerhard Neumann
Policy Learning Policy Reinforcement Learning Policy Deep Reinforcement Learning State Action Value Deep Function

August 24, 2023

Extreme Risk Mitigation in Reinforcement Learning using Extreme Value Theory
Karthik Somayaji NS, Yu Wang, Malachi Schram, Jan Drgona, Mahantesh Halappanavar, Frank Liu, Peng Li
Reinforcement Learning Risk Sensitive Reinforcement Learning Risk Management Extreme Value Theory State Action Distribution Risk Aware Reinforcement Learning State Action Value

July 29, 2023

First-order Policy Optimization for Robust Policy Evaluation
Yan Li, Guanghui Lan
State Action Value First Order Policy Robust Policy Evaluation Frank Wolfe Policy Optimization

February 24, 2023

VIPeR: Provably Efficient Algorithm for Offline RL with Neural Function Approximation
Thanh Nguyen-Tang, Raman Arora
Gradient Descent Markov Decision Process Offline Reinforcement Learning Efficient Algorithm Value Iteration State Action Value

October 6, 2022

What Should the System Do Next?: Operative Action Captioning for Estimating System Actions
Taiki Nakamura, Seiya Kawano, Akishige Yuguchi, Yasutomo Kawanishi, Koichiro Yoshino
System Description Auxiliary Task Domain Specific Task Scene Graph Prediction State Action Value

June 1, 2022

ResAct: Reinforcing Long-term Engagement in Sequential Recommendation with Residual Actor
Wanqi Xue, Qingpeng Cai, Ruohan Zhan, Dong Zheng, Peng Jiang, Kun Gai, Bo An
Reinforcement Learning Long Term Sequential Recommendation Actor Loss State Action Value Long Term User Engagement

November 19, 2021

Uncertainty-aware Low-Rank Q-Matrix Estimation for Deep Reinforcement Learning
Tong Sang, Hongyao Tang, Jianye Hao, Yan Zheng, Zhaopeng Meng
Reinforcement Learning Deep Reinforcement Learning Uncertainty Aware Low Rank Matrix Estimation State Action Value

November 12, 2021

AWD3: Dynamic Reduction of the Estimation Bias
Dogan C. Cicek, Enes Duran, Baturay Saglam, Kagan Kaya, Furkan B. Mutlu, Suleyman S. Kozat
Constructive Reduction Estimation Bias Policy Algorithm Value Based Deep Reinforcement Learning Delayed Deep Deterministic Policy Gradient State Action Value Policy Policy Gradient