Double Q Learning

Double Q-learning is a reinforcement learning technique designed to mitigate the overestimation bias inherent in standard Q-learning, improving the accuracy of value estimations and ultimately leading to better policy learning. Current research focuses on enhancing its efficiency and applicability, exploring variations like simultaneous updates, incorporating expectile losses for pessimism, and integrating it within other architectures such as actor-critic methods and ensembles. These advancements are significant because they improve the sample efficiency and robustness of reinforcement learning algorithms, leading to better performance in various applications, from robotics and game playing to more complex domains like financial modeling and cybersecurity.

Papers

June 14, 2024

Finite-Time Analysis of Simultaneous Double Q-learning
Hyunjun Na, Donghwan Lee
Reinforcement Learning Finite Time Q$ Learning Double Q Learning Function Estimation

June 6, 2024

Bootstrapping Expectiles in Reinforcement Learning
Pierre Clavier, Emmanuel Rachelson, Erwan Le Pennec, Matthieu Geist
Reinforcement Learning Robust Reinforcement Learning Double Q Learning Critic Regularization

December 10, 2023

Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularization
Takuya Hiraoka
Reinforcement Learning Sparse Reward Environment Double Q Learning Dense Reward

September 25, 2023

Adapting Double Q-Learning for Continuous Reinforcement Learning
Arsenii Kuznetsov
Reinforcement Learning Policy Reinforcement Learning Bias Correction Double Q Learning Overestimation Bias

June 27, 2023

Optimizing Credit Limit Adjustments Under Adversarial Goals Using Reinforcement Learning
Sherly Alfonso-Sánchez, Jesús Solano, Alejandro Correa-Bahnsen, Kristina P. Sendova, Cristián Bravo
Reinforcement Learning Adversarial Objective Double Q Learning

March 15, 2023

Smoothed Q-learning
David Barber
Reinforcement Learning Q Learning Double Q Learning

March 4, 2023

Double A3C: Deep Reinforcement Learning on OpenAI Gym Games
Yangxin Zhong, Jiajie He, Lingjie Kong
Reinforcement Learning Deep Reinforcement Learning Double Q Learning OpenAI Gym

February 7, 2023

Catch Me If You Can: Improving Adversaries in Cyber-Security With Q-Learning Algorithms
Arti Bandhana, Ondřej Lukáš, Sebastian Garcia, Tomáš Kroupa
Q Learning Q Function Adversary Agent Pseudo Victim AttackER Cyber Security Double Q Learning

January 13, 2023

Hierarchical Deep Q-Learning Based Handover in Wireless Networks with Dual Connectivity
Pedro Enrique Iturria Rivera, Medhat Elsayed, Majid Bavand, Raimundas Gaigalas, Steve Furr, Melike Erol-Kantarci
Reinforcement Learning Wireless Network Robot Handover Dual Network REinforcement Learning Double Q Learning Wireless Environment Radio Access Technology Deep Hierarchical

September 21, 2022

Revisiting Discrete Soft Actor-Critic
Haibin Zhou, Zichuan Lin, Junyou Li, Qiang Fu, Wei Yang, Deheng Ye
Action Space Discrete Environment Soft Actor Critic Double Q Learning

May 30, 2022

GraMeR: Graph Meta Reinforcement Learning for Multi-Objective Influence Maximization
Sai Munikoti, Balasubramaniam Natarajan, Mahantesh Halappanavar
Influence Maximization Double Q Learning First Choice Maximality Step Gram Influence Campaign

May 8, 2022

Simultaneous Double Q-learning with Conservative Advantage Learning for Actor-Critic Methods
Qing Li, Wengang Zhou, Zhenbo Lu, Houqiang Li
Actor Critic Policy Actor Critic Double Q Learning Advantage Learning

March 22, 2022

Action Candidate Driven Clipped Double Q-learning for Discrete and Continuous Action Tasks
Haobo Jiang, Jin Xie, Jian Yang
Markov Decision Process Discrete Environment Multiple Candidate Double Q Learning

November 30, 2021

Continuous Control With Ensemble Deep Deterministic Policy Gradients
Piotr Januszewski, Mateusz Olko, Michał Królikowski, Jakub Świątkowski, Marcin Andrychowicz, Łukasz Kuciński, Piotr Miłoś
Deep Reinforcement Learning Continuous Control Deep Deterministic Policy Gradient Continuous Control Task Ensemble Based Double Q Learning

November 17, 2021

Aggressive Q-Learning with Ensembles: Achieving Both High Sample Efficiency and High Asymptotic Performance
Yanqiu Wu, Xinyue Chen, Che Wang, Yiming Zhang, Keith W. Ross
Q Learning Diverse Ensemble Model Free Model Free Deep Reinforcement Learning Single Sample Model Free Algorithm Double Q Learning Quantile Temporal Difference Asymptotic Performance

Double Q Learning

Papers

Finite-Time Analysis of Simultaneous Double Q-learning

Bootstrapping Expectiles in Reinforcement Learning

Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularization

Adapting Double Q-Learning for Continuous Reinforcement Learning

Optimizing Credit Limit Adjustments Under Adversarial Goals Using Reinforcement Learning

Smoothed Q-learning

Double A3C: Deep Reinforcement Learning on OpenAI Gym Games

Catch Me If You Can: Improving Adversaries in Cyber-Security With Q-Learning Algorithms

Hierarchical Deep Q-Learning Based Handover in Wireless Networks with Dual Connectivity

Revisiting Discrete Soft Actor-Critic

GraMeR: Graph Meta Reinforcement Learning for Multi-Objective Influence Maximization

Simultaneous Double Q-learning with Conservative Advantage Learning for Actor-Critic Methods

Action Candidate Driven Clipped Double Q-learning for Discrete and Continuous Action Tasks

Continuous Control With Ensemble Deep Deterministic Policy Gradients

Aggressive Q-Learning with Ensembles: Achieving Both High Sample Efficiency and High Asymptotic Performance