Value Based Deep Reinforcement Learning

Value-based deep reinforcement learning (RL) focuses on learning optimal action-value functions to guide decision-making in complex environments. Current research emphasizes improving the efficiency and robustness of these methods, exploring techniques like network pruning, optimized hyperparameter selection, and novel regularization strategies to mitigate issues such as estimation bias and the loss of plasticity in continual learning. These advancements are crucial for enhancing the performance and reliability of RL agents across diverse applications, from program synthesis and robotics to healthcare and personalized medicine, where accurate and efficient decision-making is paramount.

Papers

October 8, 2024

Reinforcement Learning From Imperfect Corrective Actions And Proxy Rewards
Zhaohui Jiang, Xuening Feng, Paul Weng, Yifei Zhu, Yan Song, Tianze Zhou, Yujing Hu, Tangjie Lv, Changjie Fan
Reinforcement Learning Proxy Reward Sparse Semantic Value Based Deep Reinforcement Learning Agent Alignment Action Correction

July 10, 2024

CM-DQN: A Value-Based Deep Reinforcement Learning Model to Simulate Confirmation Bias
Jiacheng Shen, Lihan Feng
Deep Reinforcement Learning DQN Agent Confirmation Bias Value Based Deep Reinforcement Learning

June 25, 2024

On the consistency of hyper-parameter selection in value-based deep reinforcement learning
Johan Obando-Ceron, João G. M. Araújo, Aaron Courville, Pablo Samuel Castro
Deep Reinforcement Learning Strong Consistency Hyper Parameter Deep RL Value Based Deep Reinforcement Learning

February 19, 2024

In value-based deep reinforcement learning, a pruned network is a good network
Johan Obando-Ceron, Aaron Courville, Pablo Samuel Castro
Network Programming Performance Improvement Deep Reinforcement Magnitude Pruning Value Based Deep Reinforcement Learning Network Characteristic Effective Parameter

October 5, 2023

Small batch deep reinforcement learning
Johan Obando-Ceron, Marc G. Bellemare, Pablo Samuel Castro
Deep Reinforcement Learning Mini Batch Memory Replay Batch Size Value Based Deep Reinforcement Learning

October 4, 2023

$\mathcal{B}$-Coder: Value-Based Deep Reinforcement Learning for Program Synthesis
Zishun Yu, Yunzhe Tao, Liyu Chen, Tao Sun, Hongxia Yang
Reinforcement Learning Program Synthesis Value Based Reinforcement Learning Value Based Deep Reinforcement Learning Value Based Policy Based Reinforcement Learning

May 22, 2023

Regularization and Variance-Weighted Regression Achieves Minimax Optimality in Linear MDPs: Theory and Practice
Toshinori Kitamura, Tadashi Kozuno, Yunhao Tang, Nino Vieillard, Michal Valko, Wenhao Yang, Jincheng Mei, Pierre Ménard, Mohammad Gheshlaghi Azar, Rémi Munos, Olivier Pietquin, Matthieu Geist, Csaba Szepesvári, Wataru Kumagai, Yutaka Matsuo
Theoretical Understanding Practice Mode Mirror Descent Infinite Horizon Minimax Optimal Linear Markov Decision Process Value Based Deep Reinforcement Learning

March 13, 2023

Loss of Plasticity in Continual Deep Reinforcement Learning
Zaheer Abbas, Rosie Zhao, Joseph Modayil, Adam White, Marlos C. Machado
Continual LEArning Interest Loss Plasticity Rule Continual Reinforcement Learning Deep Reinforcement Learning Agent Value Based Deep Reinforcement Learning

September 15, 2022

Optimistic Curiosity Exploration and Conservative Exploitation with Linear Reward Shaping
Hao Sun, Lei Han, Rui Yang, Xiaoteng Ma, Jian Guo, Bolei Zhou
Reward Shaping Exploration Exploitation Curiosity Driven Exploration Value Based Deep Reinforcement Learning Conservative Prediction

July 12, 2022

December 9, 2021

DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization
Aviral Kumar, Rishabh Agarwal, Tengyu Ma, Aaron Courville, George Tucker, Sergey Levine
Deep Reinforcement Learning Implicit Regularization Explicit Regularization Effective Regularization Implicit Regularization Effect Value Based Deep Reinforcement Learning

November 12, 2021

AWD3: Dynamic Reduction of the Estimation Bias
Dogan C. Cicek, Enes Duran, Baturay Saglam, Kagan Kaya, Furkan B. Mutlu, Suleyman S. Kozat
Constructive Reduction Estimation Bias Policy Algorithm Value Based Deep Reinforcement Learning Delayed Deep Deterministic Policy Gradient State Action Value Policy Policy Gradient

Value Based Deep Reinforcement Learning

Papers

Reinforcement Learning From Imperfect Corrective Actions And Proxy Rewards

CM-DQN: A Value-Based Deep Reinforcement Learning Model to Simulate Confirmation Bias

On the consistency of hyper-parameter selection in value-based deep reinforcement learning

In value-based deep reinforcement learning, a pruned network is a good network

Small batch deep reinforcement learning

$\mathcal{B}$-Coder: Value-Based Deep Reinforcement Learning for Program Synthesis

Regularization and Variance-Weighted Regression Achieves Minimax Optimality in Linear MDPs: Theory and Practice

Loss of Plasticity in Continual Deep Reinforcement Learning

Optimistic Curiosity Exploration and Conservative Exploitation with Linear Reward Shaping

Pseudo value-based Deep Neural Networks for Multi-state Survival Analysis

FedPseudo: Pseudo value-based Deep Learning Models for Federated Survival Analysis

DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization

AWD3: Dynamic Reduction of the Estimation Bias