Overview

Q-learning studies how an agent can estimate the long-term value of each action from rewards and transitions, then choose actions with high estimated value without first learning a full environment model. Methods range from tabular Bellman updates and fitted Q-iteration to deep critics, multi-step and entropy-regularized updates, and offline methods that constrain actions to those supported by data. Theory addresses convergence, finite-sample error, regret, and extensions to robust, risk-sensitive, continuous-time, and multi-agent settings; experiments use benchmarks including Atari, D4RL, and robotic manipulation. A central tension is that bootstrapping and greedy action selection can make learning efficient but amplify estimation errors and instability, particularly with function approximation, off-policy data, sparse coverage, or changing dynamics.

Latest papers

CardsList

114 papers18 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

  1. Towards Surrogate Based Dequantization of Quantum Reinforcement Learning

    Sep 14, 2026Pablo Rodriguez-Grasa, Sofiene Jerbi, Mikel Sanz +1Reinforcement Learningfitted q-iteration

  2. Steady-State Convergence of Stochastic Approximation

    Sep 14, 2026Yixuan Zhang, Qiaomin XieStochastic ApproximationQ-Learning

  3. HQARRF: Hierarchical Q-learning and Force-aware Routing for Multi-Charger Scheduling in Wireless Rechargeable Sensor Networks

    Sep 12, 2026Liang-Ching Tao, Pi-Chung WangWireless Sensor NetworksInternet of Things

  4. Adaptive Agent Design

    Sep 11, 2026Raj Kiriti Velicheti, Subhonmesh Bose, Tamer BaşarReinforcement LearningQ-Learning

  5. An Evolutionary Computation Framework for Multi-Agent Q-Learning with Mean-Field Environmental Feedback

    Sep 5, 2026Lichen Wang, Shijia Hua, Linjie LiuMean-Field MARLMulti-Agent Reinforcement Learning

  6. Trajectory-Initialized Neural Double Q-Routing for Large-Scale Overhead Hoist Transport Systems

    Aug 31, 2026Cheng Gu, Qiusheng Zhao, Anbang Liu +2Q-LearningVehicle Routing

  7. Robust Federated Q-Learning with Almost No Communication

    Aug 24, 2026Sreejeet Maity, Aritra MitraFederated RLQ-Learning

  8. Q-Learning with Stable Infinite-Dimensional Linear Function Approximation

    Aug 23, 2026Shengbo WangQ-LearningFunction Approximation in RL

  9. Reinforcement Learning for Continuous-Time Jump Markov Decision Processes with Applications to Network Dynamic Pricing

    Aug 21, 2026Huiling Meng, Ningyuan Chen, Xuefeng GaoMarkov Decision ProcessesContinuous-Time RL

  10. Q-Learning With World Models

    Aug 17, 2026Perry Dong, Yueru Jia, Chelsea Finn +1RL for RoboticsModel-Based RL

  11. Revisiting Overestimation Bias Problem of Q-learning: Settling Large Discrete Action Space via Action Intersection

    Aug 13, 2026Pu Li, Tao Tan, Hong Xie +2Reinforcement LearningDeep Q-Learning

  12. Decentralized Multi-Player Q-Learning in Episodic Markov Decision Processes with Information Asymmetry

    Aug 13, 2026Larissa Xu, King Bi, William ChangEpisodic RLRegret Minimization in RL

  13. Aftab: A Progressive Design Study of Visual Encoders and Value Estimation for Replay-Free Parallelized Q-Learning

    Aug 7, 2026Taha Shieenavaz, Shabnam Zareshahraki, Loris NanniDeep Q-LearningQ-Learning

  14. Adaptive Finite-Budget Training for CVaR Risk-Aware Q-Learning

    Aug 5, 2026Yifan Wu, Junjie Lei, Wenjie HuangReinforcement LearningQ-Learning

  15. Revisiting TD Target Aggregation under Uncertainty in Q-Learning

    Aug 4, 2026Lipeng Zu, Xiaonan ZhangDeep Q-LearningValue Function Estimation

  16. Upper-Expectile Multi-Step Q-Learning for Off-Policy Reinforcement Learning

    Aug 3, 2026Abdelghani Ghanem, Mounir GhoghoQ-LearningTemporal-Difference Learning

  17. Variance-Reduced Q-Learning over Static and Time-Varying Networks

    Jul 24, 2026Sreejeet Maity, Feng Zhu, Aritra Mitra +1Reinforcement LearningQ-Learning

  18. Robust Asynchronous Q-Learning under Reward and State Corruption via Batching

    Jul 23, 2026Sreejeet Maity, Aritra MitraAsynchronous RLQ-Learning

  19. Equilibrium stability as a driver of cooperation among Q-learners

    Jul 15, 2026Janusz M. Meylahn, Maximilian SchäferGame-Playing AgentsRepeated Games

  20. Q-Learning Lab: Teaching Reinforcement Learning Through Learner-Generated Trace Analysis

    Jul 12, 2026Ekkachai JuengReinforcement LearningQ-Learning

  21. Spectral Analysis of Dueling Q-Learning

    Jul 9, 2026Donghwan LeeQ-Learning

  22. Optimal Transport Q-Learning for Flow Policy Steering and Acceleration

    Jul 7, 2026Andreas Sochopoulos, Esmeralda S. Whitammer, Nikolaos Tsagkas +3Robotic RLPolicy Learning

  23. Mean Field Reinforcement Learning: Theory and Algorithms for Mean Field Control

    Jul 1, 2026René Carmona, Mathieu LaurièreReinforcement LearningQ-Learning

  24. Accelerating Q-learning through Efficient Value-Sharing across Actions

    Jun 29, 2026Prabhat Nagarajan, Brett Daley, Martha White +1Deep Q-LearningValue Function Estimation

  25. Heavy-Ball Q-Learning with Residual Weighting Correction

    Jun 25, 2026Donghwan LeeMomentum MethodsQ-Learning

  26. NASDAQ: Normalized Observation Space Dynamics-Augmented Q-Learning

    Jun 19, 2026Xinwei Liu, Junyuan Liang, Zicong Hong +2Reinforcement LearningQ-Learning

  27. Robust QQ-learning for mean-field control under Wasserstein uncertainty in common noise

    Jun 18, 2026Mathieu Laurière, Ariel Neufeld, Kyunghyun ParkQ-LearningWasserstein Distributionally Robust Optimization

  28. Pareto Q-Learning with Reward Machines

    Jun 17, 2026Arnaud Lequen, Clément Legrand-Lixon, Léo SaulièresMulti-Objective Reinforcement LearningReward Machines

  29. Quantum Annealing Enhanced Reinforcement Learning for Accurate Remaining Useful Lifetime Prediction

    Jun 16, 2026Manoranjan Gandhudi, Arunkumar V., G. R. Anil +1RUL EstimationQuantum Annealing