Value Function

Value functions, central to reinforcement learning and optimal control, estimate the expected cumulative reward from a given state or state-action pair, guiding agents towards optimal behavior. Current research focuses on improving value function approximation accuracy and stability, particularly using neural networks (including shallow ReLU networks and transformers), and developing algorithms that address challenges like offline learning, multi-task optimization, and robustness to noise and uncertainty. These advancements are crucial for enhancing the efficiency and reliability of reinforcement learning agents in diverse applications, from robotics and autonomous systems to personalized recommendations and safe AI.

Papers

November 7, 2024

Vision Language Models are In-Context Value Learners
Yecheng Jason Ma, Joey Hejna, Ayzaan Wahid, Chuyuan Fu, Dhruv Shah, Jacky Liang, Zhuo Xu, Sean Kirmani, Peng Xu, Danny Driess, Ted Xiao, Jonathan Tompson, Osbert Bastani, Dinesh Jayaraman, Wenhao Yu, Tingnan Zhang, Dorsa Sadigh, Fei Xia
Vision Language Model Context Learning Value Function Visuomotor Policy Pixel Trajectory Temporal Evolution

October 23, 2024

Generalizable Motion Planning via Operator Learning
Sharath Matada, Luke Bhan, Yuanyuan Shi, Nikolay Atanasov
Motion Planning Value Function Operator Learning Neural Planner

October 14, 2024

Improved Regret Bound for Safe Reinforcement Learning via Tighter Cost Pessimism and Reward Optimism
Kihyun Yu, Duksang Lee, William Overman, Dabeen Lee
Markov Decision Process Safe Reinforcement Learning Regret Bound Value Function Stochastic Reward

October 11, 2024

October 7, 2024

October 2, 2024

Stable Offline Value Function Learning with Bisimulation-based Representations
Brahma S. Pavse, Yudong Chen, Qiaomin Xie, Josiah P. Hanna
Reinforcement Learning Value Function State Action Pair Bisimulation Metric Continual Offline Reinforcement Learning

September 26, 2024

Inference-Time Language Model Alignment via Integrated Value Guidance
Zhixuan Liu, Zhanhui Zhou, Yuanfu Wang, Chao Yang, Yu Qiao
Large Language Model Language Model Value Function Language Model Alignment Multi Concept Guidance Sentiment Expression

September 18, 2024

July 31, 2024

Tractable and Provably Efficient Distributional Reinforcement Learning with General Value Function Approximation
Taehyun Cho, Seungyub Han, Kyungjae Lee, Seokhun Ju, Dohyeong Kim, Jungwoo Lee
Value Function Distributional Reinforcement Learning Regret Analysis Episodic Markov Decision Process Approximate Dynamic Programming

July 29, 2024

A Method for Fast Autonomy Transfer in Reinforcement Learning
Dinuka Sahabandu, Bhaskar Ramasubramanian, Michail Alexiou, J. Sukarno Mertoguno, Linda Bushnell, Radha Poovendran
Reinforcement Learning Practical Method Value Function Actor Critic Algorithm Novel Reinforcement Learning Autonomy Transfer

July 8, 2024

$\mathrm{E^{2}CFD}$: Towards Effective and Efficient Cost Function Design for Safe Reinforcement Learning via Large Language Model
Zepeng Wang, Chao Ma, Linjiang Zhou, Libing Wu, Lei Yang, Xiaochuan Shi, Guojun Peng
Large Language Model Safe Reinforcement Learning Value Function Computational Fluid Dynamic Safety Critical Scenario

June 28, 2024

Safety through feedback in Constrained RL
Shashank Reddy Chirra, Pradeep Varakantham, Praveen Paruchuri
Human Feedback Human SAFETY Value Function Feedback System Trajectory Preference

June 26, 2024

Boosting Soft Q-Learning by Bounding
Jacob Adamczyk, Volodymyr Makarenko, Stas Tiomkin, Rahul V. Kulkarni
Zero Shot Value Function General Bound Optimal Value Function Soft Q Learning

June 10, 2024

Is Value Functions Estimation with Classification Plug-and-play for Offline Reinforcement Learning?
Denis Tarasov, Kirill Brilliantov, Dmitrii Kharlapenko
Deep Neural Network Deep Reinforcement Learning Offline Reinforcement Learning Reinforcement Learning Algorithm Value Function Cross Entropy Plug and Play

June 3, 2024

Learning the Target Network in Function Space
Kavosh Asadi, Yao Liu, Shoham Sabach, Ming Yin, Rasool Fakoor
Reinforcement Learning Deep Reinforcement Learning Value Function Function Space Adaptive Network Model Equivalence Goal Node

May 29, 2024

State Space Models are Comparable to Transformers in Estimating Functions with Dynamic Smoothness
Naoki Nishikawa, Taiji Suzuki
Deep Neural Network Transformer Megatron Decepticons State Space Model Value Function Input Sequence Generalized Smoothness

May 13, 2024

Adaptive Exploration for Data-Efficient General Value Function Evaluations
Arushi Jain, Josiah P. Hanna, Doina Precup
Reinforcement Learning Value Function Adaptive Exploration Variance Estimation

Value Function

Papers

Vision Language Models are In-Context Value Learners

Generalizable Motion Planning via Operator Learning

Improved Regret Bound for Safe Reinforcement Learning via Tighter Cost Pessimism and Reward Optimism

Hierarchical Universal Value Function Approximators

Zero-Shot Offline Imitation Learning via Optimal Transport

Function Gradient Approximation with Random Shallow ReLU Networks with Control Applications

Goal-Conditioned Terminal Value Estimation for Real-time and Multi-task Model Predictive Control

Stable Offline Value Function Learning with Bisimulation-based Representations

Inference-Time Language Model Alignment via Integrated Value Guidance

Making Large Language Models into World Models with Precondition and Effect Knowledge

Almost Sure Convergence of Linear Temporal Difference Learning with Arbitrary Features

Tractable and Provably Efficient Distributional Reinforcement Learning with General Value Function Approximation

A Method for Fast Autonomy Transfer in Reinforcement Learning

$\mathrm{E^{2}CFD}$: Towards Effective and Efficient Cost Function Design for Safe Reinforcement Learning via Large Language Model

Safety through feedback in Constrained RL

Boosting Soft Q-Learning by Bounding

Is Value Functions Estimation with Classification Plug-and-play for Offline Reinforcement Learning?

Learning the Target Network in Function Space

State Space Models are Comparable to Transformers in Estimating Functions with Dynamic Smoothness

Adaptive Exploration for Data-Efficient General Value Function Evaluations