Online Reinforcement Learning

Online reinforcement learning (RL) focuses on training agents to make optimal decisions in dynamic environments through continuous interaction and feedback. Current research emphasizes improving sample efficiency, particularly through offline data pre-training and techniques like prioritized experience replay and ensemble methods, as well as exploring novel model architectures such as Kolmogorov-Arnold Networks. These advancements aim to address challenges like reward sparsity, distribution shifts between offline and online data, and the need for safe and reliable learning in high-stakes applications such as robotics and healthcare, ultimately leading to more robust and efficient RL agents.

Papers

May 11, 2024

Fairness in Reinforcement Learning: A Survey
Anka Reuel, Devin Ma
Reinforcement Learning Timely Survey Multi Agent Reinforcement Learning Procedural Fairness Online Reinforcement Learning

April 23, 2024

The Power of Resets in Online Reinforcement Learning
Zakaria Mhammedi, Dylan J. Foster, Alexander Rakhlin
Reinforcement Learning Real Power Online Reinforcement Learning Episodic Markov Decision Process Optimal Value Function Global Reset Feature

March 23, 2024

Safe Reinforcement Learning for Constrained Markov Decision Processes with Stochastic Stopping Time
Abhijit Mazumdar, Rafal Wisniewski, Manuela L. Bujorianu
Optimal Policy Safe Reinforcement Learning Efficient Exploration Safety Constraint Online Reinforcement Learning Constrained Markov Decision Process Stopping Time

March 12, 2024

Symmetric Q-learning: Reducing Skewness of Bellman Error in Online Reinforcement Learning
Motoki Omura, Takayuki Osa, Yusuke Mukuta, Tatsuya Harada
Deep Reinforcement Learning Online Reinforcement Learning State of the Art Reinforcement Error Distribution Skewed Non IID Bellman Error

March 7, 2024

Why Online Reinforcement Learning is Causal
Oliver Schulte, Pascal Poupart
Reinforcement Learning Causal Model Causal Relationship Online Reinforcement Learning Causal Learning Causal Event

February 28, 2024

Implementing Online Reinforcement Learning with Clustering Neural Networks
James E. Smith
Reinforcement Learning Online Reinforcement Learning Reward Signal Neural Clustering

February 26, 2024

Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Iris Yan, Finale Doshi-Velez, Susan A. Murphy
Reinforcement Learning Online Reinforcement Learning Clinical Trial Fidelity Metric

February 22, 2024

Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs
Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, Sara Hooker
Reinforcement Learning LeArning Abstract Medical LLM Human Feedback Proximal Policy Optimization Style Representation Online Reinforcement Learning AI Alignment Basic Concept

February 20, 2024

Reflect-RL: Two-Player Online RL Fine-Tuning for LMs
Runlong Zhou, Simon S. Du, Beibin Li
Language Model Reinforcement Learning Fine Tuning Online Reinforcement Learning

February 19, 2024

The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning
Anya Sims, Cong Lu, Jakob Foerster, Yee Whye Teh
Offline Reinforcement Learning Model Based Reinforcement Learning Online Reinforcement Learning Action Free Offline Learned Dynamic Value Learning

February 14, 2024

LL-GABR: Energy Efficient Live Video Streaming Using Reinforcement Learning
Adithya Raman, Bekir Turkkan, Tevfik Kosar
Energy Efficiency Online Reinforcement Learning Video Quality Assessment Quality of Experience Adaptive Bitrate Live Video

February 11, 2024

Online Iterative Reinforcement Learning from Human Feedback with General Preference Model
Chenlu Ye, Wei Xiong, Yuheng Zhang, Hanze Dong, Nan Jiang, Tong Zhang
Reinforcement Learning Human Feedback Online Reinforcement Learning Preference Model

February 7, 2024

A Bayesian Approach to Online Learning for Contextual Restless Bandits with Applications to Public Health
Biyonka Liang, Lily Xu, Aparna Taneja, Milind Tambe, Lucas Janson
Financial Application Online Learning Online Reinforcement Learning Bayesian Approach Bayesian Learning Restless Multi Armed Bandit Restless Bandit Guideline Adherence Adherence Prediction

January 18, 2024

Harnessing Density Ratios for Online Reinforcement Learning
Philip Amortila, Dylan J. Foster, Nan Jiang, Ayush Sekhari, Tengyang Xie
Online Reinforcement Learning Density Ratio

January 10, 2024

Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces
Yaqi Duan, Martin J. Wainwright
Reinforcement Learning Offline Reinforcement Learning Core Stability Online Reinforcement Learning State Action Space Tame Function Continuous Time Markov Decision Process

December 19, 2023

Sample Efficient Reinforcement Learning with Partial Dynamics Knowledge
Meshal Alharbi, Mardavij Roozbehani, Munther Dahleh
Sample Complexity Q Learning Online Reinforcement Learning Sample Efficient Reinforcement Learning

December 10, 2023

The Generalization Gap in Offline Reinforcement Learning
Ishita Mediratta, Qingfei You, Minqi Jiang, Roberta Raileanu
Strong Generalization Offline Reinforcement Learning Online Reinforcement Learning Generalization Gap Offline Learning

November 6, 2023

Uni-O4: Unifying Online and Offline Deep Reinforcement Learning with Multi-Step On-Policy Optimization
Kun Lei, Zhengmao He, Chenhao Lu, Kaizhe Hu, Yang Gao, Huazhe Xu
Reinforcement Learning Online Service Online Reinforcement Learning Offline Policy Policy Improvement Online Fine Tuning Offline Deep Reinforcement Learning

October 17, 2023

Towards Optimal Regret in Adversarial Linear MDPs with Bandit Feedback
Haolin Liu, Chen-Yu Wei, Julian Zimmert
Optimal Regret Bandit Feedback Online Reinforcement Learning Adversarial Loss Adversarial Markov Decision Process

October 12, 2023

Offline Retraining for Online RL: Decoupled Policy Learning to Mitigate Exploration Bias
Max Sobol Mark, Archit Sharma, Fahim Tajwar, Rafael Rafailov, Sergey Levine, Chelsea Finn
Reinforcement Learning Online Reinforcement Learning Action Free Offline Online Fine Tuning Exploration Bias