Q Value

Q-values, representing the expected cumulative reward from a given state-action pair in reinforcement learning, are central to value-based decision-making in agents, particularly those interacting with complex environments. Current research focuses on improving Q-value estimation accuracy and efficiency, exploring techniques like Q-shaping for faster learning, step-level Q-value models for multi-step decision problems, and addressing issues like Q-value divergence in offline reinforcement learning through methods such as improved model architectures and novel sampling strategies. These advancements enhance the performance and sample efficiency of reinforcement learning agents, impacting diverse applications from robotics and portfolio management to natural language processing and optical systems.

Papers

October 27, 2024

Q-Distribution guided Q-learning for offline reinforcement learning: Uncertainty penalized Q-value via consistency model
Jing Zhang, Linjiajie Fang, Kexin Shi, Wenjia Wang, Bing-Yi Jing
Reinforcement Learning Offline Reinforcement Learning Q Learning Consistency Model Q Value

October 15, 2024

Process Reward Model with Q-Value Rankings
Wendi Li, Yixuan Li
Markov Decision Process Reward Model Q Value

October 2, 2024

From Reward Shaping to Q-Shaping: Achieving Unbiased Learning with LLM-Guided Knowledge
Xiefeng Wu
Large Language Model Reinforcement Learning Reward Shaping Unbiased Learning Q Value

September 14, 2024

Enhancing Decision-Making for LLM Agents via Step-Level Q-Value Models
Yuanzhao Zhai, Tingkai Yang, Kele Xu, Feng Dawei, Cheng Yang, Bo Ding, Huaimin Wang
Decision Making LLM Agent Q Function Q Value Direct Policy

July 16, 2024

Why long model-based rollouts are no reason for bad Q-value estimates
Philipp Wissmann, Daniel Hein, Steffen Udluft, Volker Tresp
Reinforcement Learning Reason Giving Model Based Offline Reinforcement Learning Q Value Model Rollouts

July 14, 2024

Towards Adapting Reinforcement Learning Agents to New Tasks: Insights from Q-Values
Ashwin Ramaswamy, Ransalu Senanayake
Reinforcement Learning Policy Gradient DCU Insight AQ New Task Q Value

February 25, 2024

Optimizing Portfolio Management and Risk Assessment in Digital Assets Using Deep Learning for Predictive Analysis
Qishuo Cheng, Le Yang, Jiajian Zheng, Miao Tian, Duan Xin
Deep Learning Predictive Model DQN Agent Risk Assessment Portfolio Management Q Value Digital Asset

December 12, 2023

A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning
Yinmin Zhang, Jie Liu, Chuming Li, Yazhe Niu, Yaodong Yang, Yu Liu, Wanli Ouyang
Reinforcement Learning Visual Perspective Q Value Offline RL Algorithm Offline to Online Reinforcement Learning

October 6, 2023

Understanding, Predicting and Better Resolving Q-Value Divergence in Offline-RL
Yang Yue, Rui Lu, Bingyi Kang, Shiji Song, Gao Huang
Human Prediction Human Understanding Good Better Q Value Offline RL Algorithm Function Estimation

July 20, 2023

Beyond Black-Box Advice: Learning-Augmented Algorithms for MDPs with Q-Value Predictions
Tongxin Li, Yiheng Lin, Shaolei Ren, Adam Wierman
Markov Decision Process Learning Augmented Algorithm Q Value Machine Learned Advice

July 4, 2023

Multi-Task Learning to Enhance Generalizability of Neural Network Equalizers in Coherent Optical Systems
Sasipim Srivallapanondh, Pedro J. Freire, Ashraful Alam, Nelson Costa, Bernhard Spinnler, Antonio Napoli, Egor Sedov, Sergei K. Turitsyn, Jaroslaw E. Prilepsky
Multi Task Learning Multi Task Stronger Generalizability Parametric Equalizer Q Value Neural Network Based Equalizer Coherent Optical

June 29, 2023

Eigensubspace of Temporal-Difference Dynamics and How It Improves Value Approximation in Reinforcement Learning
Qiang He, Tianyi Zhou, Meng Fang, Setareh Maghsudi
Reinforcement Learning Deep Reinforcement Learning Value Function Temporal Difference Q Value Critic Regularization

January 6, 2023

A Data-Driven Gaussian Process Filter for Electrocardiogram Denoising
Mircea Dumitru, Qiao Li, Erick Andres Perez Alday, Ali Bahrami Rad, Gari D. Clifford, Reza Sameni
Gaussian Process New Filtering Q Value

October 24, 2022

MEET: A Monte Carlo Exploration-Exploitation Trade-off for Buffer Sampling
Julius Ott, Lorenzo Servadei, Jose Arjona-Medina, Enrico Rinaldi, Gianfranco Mauro, Daniela Sánchez Lopera, Michael Stephan, Thomas Stadelmayer, Avik Santra, Robert Wille
Reinforcement Learning Reinforcement Learning Agent Data Sampling Exploration Exploitation Trade Bentonite Buffer Q Value

September 8, 2022

Tuning arrays with rays: Physics-informed tuning of quantum dot charge states
Joshua Ziegler, Florian Luthi, Mick Ramsey, Felix Borjans, Guoji Zheng, Justyna P. Zwolak
Quantum Computer Multidimensional Array Storage Strategy Q Value Camera Ray Charge Tuning Physic Informed Tuning

June 11, 2022

Feature Selection using e-values
Subhabrata Majumdar, Snigdhansu Chatterjee
Feature Selection Model Based Parametric Model Dimensional Feature Q Value Parametric Learning

April 28, 2022

Bilinear value networks
Zhang-Wei Hong, Ge Yang, Pulkit Agrawal
Q Function Q Value Bilinear Matrix Inequality Multi Goal Reinforcement Learning Value Network

April 7, 2022

Optimizing the Long-Term Behaviour of Deep Reinforcement Learning for Pushing and Grasping
Rodrigo Chau
Deep Reinforcement Learning Long Term BEHAVIOR Explanation Action Sequence Q Value UNO Push

March 29, 2022

Topological Experience Replay
Zhang-Wei Hong, Tao Chen, Yen-Chen Lin, Joni Pajarinen, Pulkit Agrawal
Deep Q Learning Experience Replay Replay Buffer Q Value