Q-Learning
Overview
Q-learning studies how an agent can estimate the long-term value of each action from rewards and transitions, then choose actions with high estimated value without first learning a full environment model. Methods range from tabular Bellman updates and fitted Q-iteration to deep critics, multi-step and entropy-regularized updates, and offline methods that constrain actions to those supported by data. Theory addresses convergence, finite-sample error, regret, and extensions to robust, risk-sensitive, continuous-time, and multi-agent settings; experiments use benchmarks including Atari, D4RL, and robotic manipulation. A central tension is that bootstrapping and greedy action selection can make learning efficient but amplify estimation errors and instability, particularly with function approximation, off-policy data, sparse coverage, or changing dynamics.
Latest papers
114 papers18 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.
Steady-State Convergence of Stochastic Approximation
HQARRF: Hierarchical Q-learning and Force-aware Routing for Multi-Charger Scheduling in Wireless Rechargeable Sensor Networks
Adaptive Agent Design
An Evolutionary Computation Framework for Multi-Agent Q-Learning with Mean-Field Environmental Feedback
Trajectory-Initialized Neural Double Q-Routing for Large-Scale Overhead Hoist Transport Systems
Robust Federated Q-Learning with Almost No Communication
Q-Learning with Stable Infinite-Dimensional Linear Function Approximation
Reinforcement Learning for Continuous-Time Jump Markov Decision Processes with Applications to Network Dynamic Pricing
Q-Learning With World Models
Revisiting Overestimation Bias Problem of Q-learning: Settling Large Discrete Action Space via Action Intersection
Decentralized Multi-Player Q-Learning in Episodic Markov Decision Processes with Information Asymmetry
mQ-learning and mQ-learning-intervals, achieving regret, where is the horizon, the state count, the total steps, and the joint action space across players. For Problem C we give mEXC and mEXC-Bellman, two-phase explore-then-commit algorithms with regret . Against the centralized joint-action benchmark, decentralized learning under information asymmetry matches the single-agent Q-learning rate of \cite{jin2018q} up to logarithmic factors. Because grows exponentially in , the bounds are most meaningful for small or small per-player action sets.