Real World Reinforcement Learning

Real-world reinforcement learning (RL) focuses on training agents to make optimal decisions directly within real-world environments, overcoming the limitations of simulation-based approaches. Current research emphasizes efficient exploration strategies, robust handling of uncertainty and non-stationarity, and techniques to reduce reliance on meticulously designed reward functions, often incorporating human feedback or pre-trained models to improve sample efficiency. This field is crucial for advancing robotics, control systems, and other applications requiring autonomous agents to learn and adapt in complex, dynamic settings, leading to improved performance and safety in real-world scenarios.

Papers

October 26, 2024

Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL
Andrew Wagenmaker, Kevin Huang, Liyiming Ke, Byron Boots, Kevin Jamieson, Abhishek Gupta
Reinforcement Learning Simulation Study Efficient Exploration Human Like RL Real World Reinforcement Learning

September 30, 2024

Continuously Improving Mobile Manipulation with Autonomous Real-World RL
Russell Mendonca, Emmanuel Panov, Bernadette Bucher, Jiuguang Wang, Deepak Pathak
Autonomous System Mobile Manipulation Real World Reinforcement Learning Efficient Policy Learning

May 25, 2024

Pausing Policy Learning in Non-stationary Reinforcement Learning
Hyunin Lee, Ming Jin, Javad Lavaei, Somayeh Sojoudi
Reinforcement Learning Real Time Inference Real World Reinforcement Learning Reinforcement Learning Framework Non Stationary Reinforcement Learning

May 4, 2024

UDUC: An Uncertainty-driven Approach for Learning-based Robust Control
Yuan Zhang, Jasper Hoffmann, Joschka Boedecker
Reinforcement Learning High Dimensional Predictive Control Learning Based Robust Optimization Uncertainty Aware Real World Reinforcement Learning

October 31, 2023

Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedback
Max Balsells, Marcel Torne, Zihan Wang, Samedh Desai, Pulkit Agrawal, Abhishek Gupta
Continual Learning Real World Reinforcement Learning Robot Experience Goal Oriented Reinforcement Learning Autonomous Reinforcement Learning

October 26, 2023

Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotion
Laura Smith, Yunhao Cao, Sergey Levine
Deep Reinforcement Learning Continuum Limit Quadrupedal Robot Efficient Exploration Legged Locomotion Robot Locomotion Real World Reinforcement Learning Policy Regularization Continuous Improvement

October 23, 2023

Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning
Jingyun Yang, Max Sobol Mark, Brandon Vu, Archit Sharma, Jeannette Bohg, Chelsea Finn
Reinforcement Learning Fine Tuning Reward Report Prior Policy Real World Reinforcement Learning Online Fine Tuning Robot Policy Learning

October 12, 2023

A Lightweight Calibrated Simulation Enabling Efficient Offline Learning for Optimal Control of Real Buildings
Judah Goldfeder, John Sipple
Reinforcement Learning Optimal Control Real World Reinforcement Learning Simulation Based Approach

October 6, 2023

Improving Reinforcement Learning Efficiency with Auxiliary Tasks in Non-Visual Environments: A Comparison
Moritz Lange, Noah Krystiniak, Raphael C. Engelhardt, Wolfgang Konen, Laurenz Wiskott
High Dimensional Robotics Domain Consistent Comparison Robotic Task Auxiliary Task Real World Reinforcement Learning

June 2, 2023

Efficient Reinforcement Learning with Impaired Observability: Learning to Act with Delayed and Missing State Observations
Minshuo Chen, Jie Meng, Yu Bai, Yinyu Ye, H. Vincent Poor, Mengdi Wang
LeArning Abstract Optimal Regret Efficient Reinforcement Learning Full Observability Real World Reinforcement Learning Significant Delay Change

May 10, 2023

An Option-Dependent Analysis of Regret Minimization Algorithms in Finite-Horizon Semi-Markov Decision Processes
Gianluca Drappo, Alberto Maria Metelli, Marcello Restelli
Hierarchical Reinforcement Learning Finite Horizon Real World Reinforcement Learning Planning Horizon Semi Markov Decision Process Path Dependent Option

January 8, 2023

Learning Symbolic Representations for Reinforcement Learning of Non-Markovian Behavior
Phillip J. K. Christoffersen, Andrew C. Li, Rodrigo Toro Icarte, Sheila A. McIlraith
Reinforcement Learning Non Markovian Real World Reinforcement Learning Non Markovian Reward

December 19, 2022

September 23, 2022

SAFER: Safe Collision Avoidance using Focused and Efficient Trajectory Search with Reinforcement Learning
Mario Srouji, Hugues Thomas, Hubert Tsai, Ali Farhadi, Jian Zhang
Reinforcement Learning Collision Avoidance Collision Free Trajectory Planning Real World Reinforcement Learning SAfER Instruct Trajectory Modulation

July 19, 2022

Human-to-Robot Imitation in the Wild
Shikhar Bahl, Abhinav Gupta, Deepak Pathak
Reinforcement Learning Wild Challenge Feature Imitation Shot Imitation Real World Reinforcement Learning

July 11, 2022

Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning
Homer Walke, Jonathan Yang, Albert Yu, Aviral Kumar, Jedrzej Orbik, Avi Singh, Sergey Levine
Reinforcement Learning Skill Learning Scratch Project Real World Reinforcement Learning Independent Learning Robot Policy Learning

July 5, 2022

Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularization
Yuan Zhang, Jianhong Wang, Joschka Boedecker
Reinforcement Learning Regularization Model Robust Policy Robust Reinforcement Learning Continuous Control Task Real World Reinforcement Learning

June 4, 2022

Real World Reinforcement Learning

Papers

Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL

Continuously Improving Mobile Manipulation with Autonomous Real-World RL

Pausing Policy Learning in Non-stationary Reinforcement Learning

UDUC: An Uncertainty-driven Approach for Learning-based Robust Control

Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedback

Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotion

Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning

A Lightweight Calibrated Simulation Enabling Efficient Offline Learning for Optimal Control of Real Buildings

Improving Reinforcement Learning Efficiency with Auxiliary Tasks in Non-Visual Environments: A Comparison

Efficient Reinforcement Learning with Impaired Observability: Learning to Act with Delayed and Missing State Observations

An Option-Dependent Analysis of Regret Minimization Algorithms in Finite-Horizon Semi-Markov Decision Processes

Learning Symbolic Representations for Reinforcement Learning of Non-Markovian Behavior

Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance

Near-optimal Policy Identification in Active Reinforcement Learning

SAFER: Safe Collision Avoidance using Focused and Efficient Trajectory Search with Reinforcement Learning

Human-to-Robot Imitation in the Wild

Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning

Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularization

Deciding What to Model: Value-Equivalent Sampling for Reinforcement Learning

Between Rate-Distortion Theory & Value Equivalence in Model-Based Reinforcement Learning