Ensemble Q Learning

Ensemble Q-learning improves the robustness and efficiency of reinforcement learning by using multiple Q-function approximators to estimate action values, mitigating the overestimation bias common in standard Q-learning. Current research focuses on enhancing sample efficiency through techniques like self-attention and bootstrapping, adaptively adjusting the ensemble size based on error feedback, and integrating prior knowledge or using synthetic environments ("digital cousins") to accelerate learning and improve performance in complex domains like wireless network optimization. These advancements are significant for tackling challenging control problems in various fields, offering improved data efficiency, reduced computational complexity, and more reliable policy optimization.

Papers

June 28, 2024

Contextualized Hybrid Ensemble Q-learning: Learning Fast with Control Priors
Emma Cramer, Bernd Frauenknecht, Ramil Sabirov, Sebastian Trimpe
Reinforcement Learning Prior Model Hybrid Reinforcement Learning Ensemble Q Learning

May 14, 2024

Smart Sampling: Self-Attention and Bootstrapping for Improved Ensembled Q-Learning
Muhammad Junaid Khan, Syed Hammad Ahmed, Gita Sukthankar
Self Attention Q Learning Large Relevance Improvement Multi Head Self Attention Bootstrapping End to End State Action Pair Q Ensemble Ensemble Q Learning

February 12, 2024

Leveraging Digital Cousins for Ensemble Q-Learning in Large-Scale Wireless Networks
Talha Bozkus, Urbashi Mitra
Large Scale Q Learning Diverse Ensemble Wireless Network Ensemble Q Learning

February 8, 2024

Multi-Timescale Ensemble Q-learning for Markov Decision Process Policy Optimization
Talha Bozkus, Urbashi Mitra
Reinforcement Learning Q Learning Model Free Reinforcement Learning Ensemble Q Learning Frank Wolfe Policy Optimization

June 20, 2023

Adaptive Ensemble Q-learning: Minimizing Estimation Bias via Error Feedback
Hang Wang, Sen Lin, Junshan Zhang
Q Learning Error Feedback Estimation Bias Adaptive Ensemble Function Estimation Ensemble Q Learning

Ensemble Q Learning

Papers

Contextualized Hybrid Ensemble Q-learning: Learning Fast with Control Priors

Smart Sampling: Self-Attention and Bootstrapping for Improved Ensembled Q-Learning

Leveraging Digital Cousins for Ensemble Q-Learning in Large-Scale Wireless Networks

Multi-Timescale Ensemble Q-learning for Markov Decision Process Policy Optimization

Adaptive Ensemble Q-learning: Minimizing Estimation Bias via Error Feedback