Human in the Loop Reinforcement

Human-in-the-loop reinforcement learning (HiRL) aims to improve the efficiency and effectiveness of reinforcement learning (RL) agents by incorporating human expertise and feedback during the training process. Current research focuses on developing methods that reduce human workload, such as preference-based learning and techniques leveraging sub-optimal data or pre-training, often employing algorithms like Proximal Policy Optimization (PPO) and incorporating human input through reward shaping, action injection, or interactive learning. HiRL is significant because it addresses challenges in traditional RL, like reward function design and data scarcity, enabling the development of more robust and adaptable agents for complex real-world applications in diverse fields, including autonomous vehicles, robotics, and particle physics.

Papers

September 18, 2024

HARP: Human-Assisted Regrouping with Permutation Invariant Critic for Multi-Agent Reinforcement Learning
Huawen Hu, Enze Shi, Chenxi Yue, Shuocun Yang, Zihao Wu, Yiwei Li, Tianyang Zhong, Tuo Zhang, Tianming Liu, Shu Zhang
Multi Agent Reinforcement Learning Single Agent Human in the Loop Reinforcement Agent Learning

August 22, 2024

Human-In-The-Loop Machine Learning for Safe and Ethical Autonomous Vehicles: Principles, Challenges, and Opportunities
Yousef Emami, Luis Almeida, Kai Li, Wei Ni, Zhu Han
Autonomous Vehicle Technical Challenge Active Learning Human in the Loop General Principle Legal Autonomy Human in the Loop Reinforcement

May 24, 2024

Human-in-the-loop Reinforcement Learning for Data Quality Monitoring in Particle Physics Experiments
Olivia Jullian Parra, Julián García Pardiñas, Lorenzo Del Pianta Pérez, Maximilian Janisch, Suzanne Klaver, Thomas Lehéricy, Nicola Serra
Proximal Policy Optimization Data Quality High Energy Physic Human in the Loop Reinforcement Particle Physic

May 23, 2024

Tell me why: Training preferences-based RL with human preferences and step-level explanations
Jakob Karalus
Training Data Human Preference Confidence Oracle External Feedback Human in the Loop Reinforcement Preference Understanding

April 30, 2024

Leveraging Sub-Optimal Data for Human-in-the-Loop Reinforcement Learning
Calarina Muslimani, Matthew E. Taylor
Reward Function Preference Based Reinforcement Learning Reward Engineering Human in the Loop Reinforcement Optimal Data

March 19, 2024

Digital Twin-Driven Reinforcement Learning for Obstacle Avoidance in Robot Manipulators: A Self-Improving Online Training Framework
Yuzhu Sun, Mien Van, Stephen McIlvanna, Nguyen Minh Nhat, Kabirat Olayemi, Jack Close, Seán McLoone
Non Humanoid Robot Digital Twin Obstacle Avoidance Self Training Robotic Manipulator Collaborative Robot Human in the Loop Reinforcement Virtual Robot

January 11, 2024

Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents
Quentin Delfosse, Sebastian Sztwiertnia, Mark Rothermel, Wolfgang Stammer, Kristian Kersting
Deep Reinforcement Learning Sparse Reward Concept Bottleneck Model Human in the Loop Reinforcement Agent Alignment

January 6, 2024

HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving
Zilin Huang, Zihao Sheng, Chengyuan Ma, Sikai Chen
Deep Reinforcement Learning Autonomous Vehicle Traffic Management Human in the Loop Reinforcement Efficient Autonomous Driving

January 4, 2024

Human-in-the-Loop Policy Optimization for Preference-Based Multi-Objective Reinforcement Learning
Ke Li, Han Guo
Policy OpTimization Multi Objective Reinforcement Learning Human in the Loop Reinforcement

July 14, 2023

Reinforcement Learning for Photonic Component Design
Donald Witt, Jeff Young, Lukas Chrostowski
Reinforcement Learning Photonic Device Human in the Loop Reinforcement Nanophotonic Device Photonic Crystal

April 7, 2023

UAV Obstacle Avoidance by Human-in-the-Loop Reinforcement in Arbitrary 3D Environment
Xuyang Li, Jianwu Fang, Kai Du, Kuizhi Mei, Jianru Xue
Deep Reinforcement Learning Unmanned Aerial Vehicle Collision Avoidance Aerial Vehicle Navigation Complex 3D Scene Human in the Loop Reinforcement

December 6, 2022

Few-Shot Preference Learning for Human-in-the-Loop RL
Joey Hejna, Dorsa Sadigh
Reinforcement Learning Multi Task Learning Reward Function Preference Learning Human in the Loop Reinforcement Informative Reward

November 20, 2022

Efficient Meta Reinforcement Learning for Preference-based Fast Adaptation
Zhizhou Ren, Anji Liu, Yitao Liang, Jian Peng, Jianzhu Ma
Meta Reinforcement Learning Fast Adaptation Reinforcement Learning Benchmark Policy Adaptation Human in the Loop Reinforcement Meta Rl Algorithm

June 27, 2022

Humans are not Boltzmann Distributions: Challenges and Opportunities for Modelling Human Feedback and Interaction in Reinforcement Learning
David Lindner, Mennatallah El-Assady
Reinforcement Learning Human Feedback Real Human Reward Function Interaction Generation Human Model Boltzmann Distribution Human in the Loop Reinforcement

May 23, 2022

Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximation
Xiaoyu Chen, Han Zhong, Zhuoran Yang, Zhaoran Wang, Liwei Wang
Optimal Policy Human in the Loop Preference Based Reinforcement Learning Human in the Loop Reinforcement Trajectory Preference

February 5, 2022

ASHA: Assistive Teleoperation via Human-in-the-Loop Reinforcement Learning
Sean Chen, Jensen Gao, Siddharth Reddy, Glen Berseth, Anca D. Dragan, Sergey Levine
Robotic Manipulation Sparse Reward Human in the Loop Human in the Loop Reinforcement User Feedback Assistive Teleoperation

Human in the Loop Reinforcement

Papers

HARP: Human-Assisted Regrouping with Permutation Invariant Critic for Multi-Agent Reinforcement Learning

Human-In-The-Loop Machine Learning for Safe and Ethical Autonomous Vehicles: Principles, Challenges, and Opportunities

Human-in-the-loop Reinforcement Learning for Data Quality Monitoring in Particle Physics Experiments

Tell me why: Training preferences-based RL with human preferences and step-level explanations

Leveraging Sub-Optimal Data for Human-in-the-Loop Reinforcement Learning

Digital Twin-Driven Reinforcement Learning for Obstacle Avoidance in Robot Manipulators: A Self-Improving Online Training Framework

Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents

HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving

Human-in-the-Loop Policy Optimization for Preference-Based Multi-Objective Reinforcement Learning

Reinforcement Learning for Photonic Component Design

UAV Obstacle Avoidance by Human-in-the-Loop Reinforcement in Arbitrary 3D Environment

Few-Shot Preference Learning for Human-in-the-Loop RL

Efficient Meta Reinforcement Learning for Preference-based Fast Adaptation

Humans are not Boltzmann Distributions: Challenges and Opportunities for Modelling Human Feedback and Interaction in Reinforcement Learning

Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximation

ASHA: Assistive Teleoperation via Human-in-the-Loop Reinforcement Learning