Overview

Q-learning studies how an agent can estimate the long-term value of each action from rewards and transitions, then choose actions with high estimated value without first learning a full environment model. Methods range from tabular Bellman updates and fitted Q-iteration to deep critics, multi-step and entropy-regularized updates, and offline methods that constrain actions to those supported by data. Theory addresses convergence, finite-sample error, regret, and extensions to robust, risk-sensitive, continuous-time, and multi-agent settings; experiments use benchmarks including Atari, D4RL, and robotic manipulation. A central tension is that bootstrapping and greedy action selection can make learning efficient but amplify estimation errors and instability, particularly with function approximation, off-policy data, sparse coverage, or changing dynamics.

Latest papers

CardsList

114 papers18 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

  1. Support-Constrained Offline Policy Improvement for Pediatric ECMO from Limited Clinical Data

    Nov 12, 2026Fateme Golivand Darvishvand, Saurabh Mathur, Nikhilesh Prabhakar +6fitted q-iterationOffline RL

  2. Understanding the Asymptotic Transience of Q-Learning via Distributional Expansion

    Nov 11, 2026Shengbo WangQ-LearningTemporal-Difference Learning

  3. Horizon-Scalable Q-Learning

    Nov 7, 2026Haoran Xu, Pragya Srivastava, Xinjie Liu +2Q-LearningTemporal-Difference Learning

  4. An Analysis of Reinforcement Learning with Function Approximation

    Nov 7, 2026Francisco S. Melo, Sean P. Meyn, M. Isabel RibeiroQ-LearningFunction Approximation in RL

  5. Bayesian transfer for offline Q-learning with multi-task Gaussian processes

    Nov 7, 2026Wenxing GuoOffline RLTransfer Learning in RL

  6. O(1/n)O(1/n) Convergence Rate for RVI QQ-Learning

    Nov 5, 2026Avik Kar, Siddharth Chandak, Rahul Singh +1Reinforcement LearningQ-Learning

  7. Direct Knowledge Transfer for Offline Q-Learning

    Nov 1, 2026Zhanye Luo, Guangyan Gan, Lin William Congfitted q-iterationTransfer Learning in RL

  8. Entropy-Regularized Reinforcement Learning for the Regime-Switching Consumption-Investment Problem

    Oct 31, 2026Congde Hu, Danping Li, Xun Li +1Financial Decision-MakingContinuous-Time RL

  9. Cooperative EV Charging Assignment Using Multi-Agent Reinforcement Learning: A Comparison of QMIX and Independent Q-Learning Under Urban Traffic Uncertainty

    Oct 27, 2026Nour-Eddine Moumni, Rachid Alaoui, Driss KiouachElectric Vehicle ChargingCooperative MARL

  10. Frequency-Uniform Log-Regret Reinforcement Learning for Diffusions

    Oct 27, 2026Jose Blanchet, Lutong Hao, Renyuan Xufitted q-iterationRegret Minimization in RL

  11. Provably Efficient Q-learning for Robust Reinforcement Learning

    Oct 26, 2026Yuchen Jiao, Gen LiRobust RLDistributionally Robust RL

  12. Generative Value Classifier

    Oct 15, 2026Alexander Swerdlow, Perry Dong, Chelsea FinnReinforcement LearningQ-Learning

  13. Q(γHγ_H): Entropy-Guided Horizon Mixing in Multi-Discount Q-Learning

    Oct 15, 2026Arwa Albelaihi, Abdulrahman AltahhanReinforcement LearningQ-Learning

  14. Selective Growing Q-Networks (SGQN): An Analysis of Adaptive Action Grids in Decoupled Q-Learning

    Oct 14, 2026Parham Nooranbakht, Seyed Mahdi Basiri Azad, Joschka BödeckerReinforcement LearningQ-Learning

  15. Non-asymptotic Convergence of Average-reward Q-learning with Options

    Oct 14, 2026Kintan Saha, Ahana Deb, Anders Jonsson +1Q-LearningAverage-Reward RL

  16. Value-Guided Repulsion for Offline Reinforcement Learning

    Oct 14, 2026Yixue Hao, Jiahao Zhang, Jiesong Lian +5Offline RLPolicy Learning

  17. On the Convergence of Semi-Gradient TD: A Mean Field Perspective

    Oct 14, 2026Haruki Settai, Tadashi Kozuno, Shinji ItoTemporal-Difference LearningRL Stability

  18. Neural Supervised Clustering for Learning Individualized Treatment Rules with Many Treatments

    Oct 13, 2026Daixi Huang, Yixin Yuan, Yifan Dai +1individualized treatment rule learningIndividualized Treatment Effect Estimation

  19. Fast Rates for Offline Q-Learning with KL Regularization

    Oct 13, 2026Yize Hao, Guangxin Jiang, Shuang QiuOffline RLQ-Learning

  20. From Policy Evaluation to Off-Policy Control: Provable Emergence of In-Context Q-Learning

    Oct 13, 2026Yingjun Mo, Yajie Zhao, Fang KongIn-Context RLTransformer-Based RL

  21. Entropy-Regularized Mean-Field Games with Average-Reward Criterion: Span-Seminorm Contractions and Model-Free Learning

    Oct 11, 2026Gunes Akbas, Naci Saldi, Berkay Anahtarci +1Mean Field GamesModel-Free RL

  22. Mixing Regularization for Average-Reward QQ-Learning and QQ-Planning

    Oct 10, 2026Julien Roger, Sam Bodmer, Jian Qian +1Regularized RLReinforcement Learning

  23. Hitting-Time Bounds for Asynchronous Q-Learning

    Oct 9, 2026Yang PengReinforcement LearningQ-Learning

  24. Q-Learning with Scalar Adjoint Matching

    Oct 7, 2026Yonghoon Dong, Minsung Yoon, Jaehyuk Kim +3Q-LearningOff-Policy RL

  25. QQ Can Play That Game: Online Fitted QQ-Iteration for Continuous-Action Zero-Sum Markov Games with Convex-Concave Function Approximation

    Oct 2, 2026Kushagra Gupta, Jingqi Li, Cade Armstrong +4fitted q-iterationZero-Sum Games

  26. Q-Learning for Reachability in MEC-Free MDPs

    Oct 1, 2026Lu-Chin Chang, Suguman BansalMarkov Decision ProcessesReinforcement Learning

  27. Finite-Sample Theory for Fitted Q-Iteration When Actions Are Functions

    Sep 28, 2026Gefei Lin, Rui Miao, Xiaoke Zhangfitted q-iterationReinforcement Learning

  28. All On-Board: Fully On-Chip Neuromorphic Q-Learning with Embedded CartPole Simulation

    Sep 26, 2026Steven C. Nesbit, Giovanni T. Michel, Gerd J. Kunde +2Neuromorphic ComputingQ-Learning

  29. Provably Efficient Reinforcement Learning in Continuous-Time Episodic MDPs with Poisson Decision Epochs

    Sep 19, 2026Kenny Guo, Valentio Iverson, Sahan Wijetunga +1Markov Decision ProcessesContinuous-Time RL

  30. Mitigating Retaliatory Algorithmic Collusion in Repeated Games

    Sep 17, 2026Karthik Sivachandran, Rohan PalejaAlgorithmic CollusionRepeated Games