Deep Q-Learning

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 34

All topics

Common-Mode Errors Limit Low-Timestep Deep Spiking Q-Networks
Oct 6, 2026Zijie Xu, Bingrui Guo, Yiding Sun +3Deep Q-Learning Temporal-Difference Learning
Peking University Beijing, 100871, China
Ramp Metering Control via Hybrid State Deep Reinforcement Learning in Partially Observable Connected Vehicle Environments
Oct 5, 2026Youcef Mehamlia, Nadir Farhi, Meriem BoualiRL Control Deep Q-Learning
Cosys-Grettia, Univ Gustave Eiffel, F-77454 Marne-la-Vallee, France. · Laboratoire LITAN, ESTIN, Amizour, Algeria.
Adaptive Chemotherapy Control under Tumor Heterogeneity via Reinforcement Learning
Sep 14, 2026Bereket Sitotaw Kidane, Md Samiul Haque Motayed, Shuo WangRL Control Reinforcement Learning
Department of Mechanical and Aerospace Engineering, University of Texas at Arlington, Arlington, TX 76019, USA
Generative Replay Mitigates Sample Starvation in Quantum Architecture Search
Sep 12, 2026Akash Kundu, Amit Kumar Jaiswal, Sebastian Feld +1Generative Replay Deep Q-Learning
Delft University of Technology, Delft, The Netherlands. · Quantum Computing Division, QuTech, Delft, The Netherlands. · Indian Institute of Technology (BHU), Varanasi, India. +1
Mini-Batch Risk-Averse Deep Q-Learning: A Robot Navigation Case Study
Sep 7, 2026Aayush Patel, Andrzej RuszczyńskiDeep Q-Learning Robot Navigation
Department of Management Science and Information Systems, Rutgers University
Revisiting Overestimation Bias Problem of Q-learning: Settling Large Discrete Action Space via Action Intersection
Aug 13, 2026Pu Li, Tao Tan, Hong Xie +2Reinforcement Learning Deep Q-Learning
Chongqing Institute of Green and Intelligent Technology, Chongqing, China · University of Science and Technology of China, Hefei, Anhui, China
Machine Learning-Based Cyber Defense for Cloud Infrastructure: An Adaptive Deep Q-Network Architecture for Intelligent Intrusion Detection and Automated Threat Mitigation
Aug 12, 2026Md Yassir Mottalib, Md Yousuf, Eklachur Rahman Bhuiyan +5Autonomous Cyber Defense Deep Q-Learning
Master of Science in Information System Technology, Wilmington University, USA · Master's of Science in Data Analytics, College of Sciences and Technology, University of Houston- Downtown, USA · Master of Science in Information Technology, Washington University of Science and Technology, USA +4
Dueling Deep Q-Learning for Intrusion Detection
Aug 11, 2026Logan Luna, Matthew P. Berkowitz, Laxima Niure Kandel +1Deep Q-Learning Network Intrusion Detection
Georgia Institute of Technology · Department of Electrical Engineering and Computer Science Embry-Riddle Aeronautical University Daytona Beach, USA · School of Computer Science, College of Computing, Georgia Institute of Technology, Atlanta, GA 30332 USA +1
Aftab: A Progressive Design Study of Visual Encoders and Value Estimation for Replay-Free Parallelized Q-Learning
Aug 7, 2026Taha Shieenavaz, Shabnam Zareshahraki, Loris NanniDeep Q-Learning Q-Learning
Department of Information Engineering, University of Padua, Italy
Revisiting TD Target Aggregation under Uncertainty in Q-Learning
Aug 4, 2026Lipeng Zu, Xiaonan ZhangDeep Q-Learning Value Function Estimation
Computer Science Department, Florida State University, Tallahassee, Florida, USA
Endpoint Replay: Compressing the Recency Buffer in Deep Reinforcement Learning
Jul 27, 2026Parham Mohammad Panahi, Armin Ashrafi, Haoyu Du +3Deep Q-Learning Experience Replay
Department of Computing Science, University of Alberta, Canada · Alberta Machine Intelligence Institute (Amii) · Canada CIFAR AI Chair
Optimal Reward Shaping: Autonomous Car Parking Case Study
Jul 26, 2026Emre Özkaya, Nicolas R. GaugerDeep Q-Learning Autonomous Parking
Chair for Scientific Computing, RPTU University of Kaiserslautern-Landau Paul-Ehrlich-Strasse 34, 67663 Kaiserslautern
When Every Simulation Counts: Value-Based Reinforcement Learning for Accelerated Photonics Inverse Design
Jul 26, 2026Longying Wen, Feiyang Wu, Jinglin Yu +3Deep Q-Learning Sample-Efficient RL
School of Science and Engineering The Chinese University of Hong Kong, Shenzhen Shenzhen, China · Nick Holonyak Micro and Nanotechnology Laboratory University of Illinois Urbana-Champaign Urbana, Illinois, USA
Learning-Driven Adaptive Audit Scheduling: A Sequential Decision Approach to Off-Chain Data Integrity
Jul 19, 2026Changting Lin, Fan Li, Weihang Yu +4Markov Decision Processes Deep Q-Learning
Hangzhou Yunphant Network Technology Co., Ltd., Hangzhou, China · Zhejiang University, Hangzhou, China · Central University of Finance and Economics, Beijing, China +2
CLaC@FinMMEval 2026 Task 3: Sentiment-Augmented Deep Reinforcement Learning for Active Trading -- An Alpha-Reward Approach
Jul 17, 2026Andrei Neagu, Eeham Khan, Leila KosseimReinforcement Learning Deep Q-Learning
Department of Computer Science and Software Engineering, Concordia University, Montréal, Canada
Accelerating Q-learning through Efficient Value-Sharing across Actions
Jun 29, 2026Prabhat Nagarajan, Brett Daley, Martha White +1Deep Q-Learning Value Function Estimation
Department of Computing Science, University of Alberta, Edmonton, AB, Canada. · Alberta Machine Intelligence Institute · Meta (independent work, not on Meta’s behalf). +1
Deep Q-Learning on Hölder Spaces
Jun 15, 2026Qian QiRL Control Deep Q-Learning
Peking University
Space-sampled Value Decay: Forgetting Mechanisms for Non-stationary Deep Reinforcement Learning
Jun 10, 2026Felix Störck, Fabian Hinder, Barbara HammerNon-Stationary RL Reinforcement Learning
CITEC, Faculty of Technology, Bielefeld University, Germany.
Discovering Interpretable Multi-Parameter Control Policies for Evolutionary Algorithms Using Deep Reinforcement Learning
Jun 8, 2026Tai Nguyen, Phong Le, Carola Doerr +1Evolutionary Optimization Deep Q-Learning
University of St Andrews, St Andrews, United Kingdom · Sorbonne Universit´e, CNRS, LIP6, Paris, France
COP-Q: Safety-First Reinforcement Learning for Robot Control via Cholesky-Ordered Projection
Jun 3, 2026Guopeng Li, Moritz A. Zanger, Matthijs T. J. Spaan +1Robotic Control Multi-Objective Reinforcement Learning
Department of Cognitive Robotics, Delft University of Technology · School of Transportation, Southeast University, China · Department of Intelligent Systems, Delft University of Technology, the Netherlands
Task-Induced Representational Invariances Depend on Learning Objective in Deep RL
Jun 1, 2026Manu Srinath Halvagal, Sebastian Lee, SueYeon ChungMarkov Decision Processes Deep Q-Learning
Department of Physics, Harvard University · 2Kempner Institute, Harvard University · Center for Computational Neuroscience, Flatiron Institute
QSplitFL: Capability Aware Deep Q-Learning for Optimal Split Point Selection in Split Federated Learning
Jun 1, 2026Nazmus Shakib Shadin, Xinyue Zhang, Jingyi Wang +1Reinforcement Learning Deep Q-Learning
Department of Computer Science, Kennesaw State University, Marietta, GA, 30060 USA · Department of Computer Science, San Francisco State University, San Francisco, CA, 94132 USA · Department of Electrical and Computer Engineering, University of Houston, Houston, TX, 77204 USA
SDM-Q: Cost-Aware Staged Decision-Making for Multi-Omics Classification with Deep Q-Learning
May 29, 2026Nan Mu, Yangfan Xiao, Ling Wang +3Cost-Sensitive Learning Deep Q-Learning
College of Computer Science, Sichuan Normal University, Chengdu, Sichuan 610101, China. · Department of Mathematics, College of Science and Mathematics, Kennesaw State University, Kennesaw, GA 30144, USA. · Department of Computer Science, College of Computing and Software Engineering, Kennesaw State University, Marietta, GA 30060, USA.
Don't Forget the Critic: Value-Based Data Rehearsal for Multi-Cyclic Continual Reinforcement Learning
May 21, 2026Benjamin Poole, Andrew Quinn, Li Yang +1Reinforcement Learning Deep Q-Learning
Department of Computer Science University of North Carolina at Charlotte
TabQL: In-Context Q-Learning with Tabular Foundation Models
May 18, 2026Qisai Liu, Zhanhong Jiang, Timilehin Ayanlade +4Deep Q-Learning Q-Learning
Department of Mechanical Engineering Iowa State University Ames, IA 50011 · Translational AI Center Iowa State University Ames, IA 50011 · Department of Computer Science Iowa State University Ames, IA 50011 +1
Rainbow Deep Q-Learning with Kinematics-Aware Design for Cooperative Delta and 3-RRS Parallel Robot Insertion
May 12, 2026Hassen Nigatu, Gaokun Shi, Jituo Li +2Dexterous Manipulation Deep Q-Learning
Revisiting Adam for Streaming Reinforcement Learning
May 7, 2026Florin Gogianu, Adrian Catalin Lutu, Razvan PascanuDistributional RL Reinforcement Learning
Bitdefender, Bucharest, Romania · Mila, Montreal, Canada
Beyond the Independence Assumption: Finite-Sample Guarantees for Deep Q-Learning under $τ$ -Mixing
May 7, 2026Leon Halgryn, Sophie Langer, Janusz M. Meylahn +1Reinforcement Learning Deep Q-Learning
University of Twente · Department of Applied Mathematics University of Twente · Ruhr-Universität Bochum +1
Dueling DDQN-Based Adaptive Multi-Objective Handover Optimization for LEO Satellite Networks
May 4, 2026Po-Heng Chou, Chiapin Wang, Chung-Chi Huang +1Multi-Objective Reinforcement Learning Deep Q-Learning
Research Center for Information Technology Innovation (CITI), Academia Sinica (AS), Taipei 11529, Taiwan · Department of Electrical Engineering, National Taiwan Normal University (NTNU), Taipei 106308 Taiwan
Spatial-Temporal Learning-Based Distributed Routing for Dynamic LEO Satellite Networks
May 4, 2026Po-Heng Chou, Chiapin Wang, Shou-Yu Chen +1Deep Q-Learning Partially Observable RL
Research Center for Information Technology Innovation (CITI), Academia Sinica (AS), Taipei 11529, Taiwan · Department of Electrical Engineering, National Taiwan Normal University (NTNU), Taipei 106308 Taiwan

Oct 6, 2026, cs.NEJ/K move · Enter open · S save

Common-Mode Errors Limit Low-Timestep Deep Spiking Q-Networks

Zijie Xu, Bingrui Guo, Yiding Sun, Yiting Dong+2

Peking University Beijing, 100871, China

Deep Q-Learning Temporal-Difference Learning Spiking Neural Networks

Open paper AskPDF arXiv

Spiking neural networks (SNNs) offer sparse and event-driven computation, making them attractive for energy-constrained reinforcement learning (RL) on edge devices. In value-based RL, deep spiking Q-networks (DSQNs) combine such efficiency with action-value estimation for decision making. However, existing DSQNs often require multiple simulation timesteps for competitive performance, increasing computational and energy costs, whereas reducing the timesteps can cause substantial performance degradation. We investigate this degradation from the perspective of Q-value estimation errors. By decomposing errors across actions into common-mode and differential-mode components, we find that low-timestep DSQNs suffer disproportionately from common-mode errors shared across action values, which are particularly detrimental to temporal-difference learning through bootstrapped targets. Based on this finding, we propose Common-Mode Compensation Deep Spiking Q-Network (CMC-DSQN), which uses an auxiliary ANN to compensate for common-mode errors in the SNN outputs. At inference, greedy action selection can be performed directly from the SNN outputs, allowing the auxiliary ANN to be completely removed and preserving the energy efficiency of SNNs. Extensive experiments on Atari and MiniAtar environments demonstrate substantial performance improvements under low-timestep settings. CMC-DSQN outperforms state-of-the-art DSQN baselines by nearly

20\%

T=2

and further surpasses the ANN baseline at

T=4

Momentum

Related topics

Latest papers 34