Q$ Learning Algorithm

Q-learning is a reinforcement learning algorithm aiming to find optimal action-selection policies by iteratively estimating the expected cumulative reward (Q-value) for each state-action pair. Current research focuses on improving Q-learning's robustness and efficiency, particularly through techniques like distributionally robust optimization, knowledge transfer from related tasks, and the use of function approximation (e.g., linear architectures and nearest neighbors) to handle large state spaces. These advancements enhance the algorithm's applicability to complex real-world problems in diverse fields such as marketing, healthcare, and robotics, while also addressing challenges like instability and sample inefficiency. The development of provably convergent and efficient Q-learning variants is a significant area of ongoing investigation.

Papers

July 5, 2024

Robust Q-Learning for finite ambiguity sets
Cécile Decker, Julian Sester
Q Learning Probability Measure Ambiguity Set Q$ Learning Algorithm Robust Markov Decision

April 1, 2024

Data-Driven Knowledge Transfer in Batch $Q^*$ Learning
Elynn Chen, Xi Chen, Wenbo Jing
Markov Decision Process Knowledge Transfer Data Driven Decision Action Value Function Dynamic Decision Batch Greenkhorn Q$ Learning Algorithm

December 8, 2023

Two-Timescale Q-Learning with Function Approximation in Zero-Sum Stochastic Games
Zaiwei Chen, Kaiqing Zhang, Eric Mazumdar, Asuman Ozdaglar, Adam Wierman
Stochastic Approximation Function Approximation Stochastic Game Q$ Learning Zero Sum Markov Game Q$ Learning Algorithm Two Timescale

November 26, 2023

FRAC-Q-Learning: A Reinforcement Learning with Boredom Avoidance Processes for Social Robots
Akinari Onishi
Reinforcement Learning Q Learning Social Robot Q$ Learning Q$ Learning Algorithm

September 28, 2023

Multi-Bellman operator for convergence of $Q$-learning with linear function approximation
Diogo S. Carvalho, Pedro A. Santos, Francisco S. Melo
Early Stage Convergence Linear Function Approximation Q$ Learning Bellman Operator Q$ Learning Algorithm

August 3, 2023

Minimax Optimal Q Learning with Nearest Neighbors
Puning Zhao, Lifeng Lai
Markov Decision Process Nearest Neighbor State Space Minimax Q Learning Function Estimation Q$ Learning Algorithm

March 17, 2023

Comparing NARS and Reinforcement Learning: An Analysis of ONA and $Q$-Learning Algorithms
Ali Beikmohammadi, Sindri Magnússon
Reinforcement Learning General Analysis Reasoning System Q$ Learning Non Axiomatic Reasoning Sequential Task Q$ Learning Algorithm

September 30, 2022

Robust $Q$-learning Algorithm for Markov Decision Processes under Wasserstein Uncertainty
Ariel Neufeld, Julian Sester
Practical Algorithm Markov Decision Process Robust Version Distributional Robustness Stochastic Optimal Control Q$ Learning Algorithm Wasserstein Uncertainty Robust Markov Decision

June 1, 2022

Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning
Andrea Zanette, Martin J. Wainwright
Reinforcement Learning Linear Function Approximation Linear MDPs Q$ Learning Algorithm Provably Efficient Learning

February 9, 2022

Transferred Q-learning
Elynn Y. Chen, Michael I. Jordan, Sai Li
Transfer Learning Q$ Learning Transfer Reinforcement Learning Function Estimation Q$ Learning Algorithm Target Policy

Q$ Learning Algorithm

Papers

Robust Q-Learning for finite ambiguity sets

Data-Driven Knowledge Transfer in Batch $Q^*$ Learning

Two-Timescale Q-Learning with Function Approximation in Zero-Sum Stochastic Games

FRAC-Q-Learning: A Reinforcement Learning with Boredom Avoidance Processes for Social Robots

Multi-Bellman operator for convergence of $Q$-learning with linear function approximation

Minimax Optimal Q Learning with Nearest Neighbors

Comparing NARS and Reinforcement Learning: An Analysis of ONA and $Q$-Learning Algorithms

Robust $Q$-learning Algorithm for Markov Decision Processes under Wasserstein Uncertainty

Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning

Transferred Q-learning