Interactive Reinforcement Learning

Interactive Reinforcement Learning (IRL) aims to accelerate the training of reinforcement learning agents by incorporating human feedback, addressing the limitations of traditional RL's sample inefficiency and reward specification challenges. Current research focuses on improving feedback mechanisms (e.g., using scalar scores, large language model evaluations, or multi-trainer aggregation), developing more efficient algorithms (like actor-critic methods and adaptive learning schemes), and applying IRL to diverse domains such as robotics, UAV control, and recommendation systems. This approach holds significant promise for enhancing the performance and applicability of RL in complex real-world scenarios where direct reward design is difficult or impossible.

Papers

October 31, 2024

Prosody as a Teaching Signal for Agent Learning: Exploratory Studies and Algorithmic Implications
Matilda Knierim, Sahil Jain, Murat Han Aydoğan, Kenneth Mitra, Kush Desai, Akanksha Saran, Kim Baraka
Prosodic Feature Exploratory Study Algorithmic Decision Social Cue Interactive Reinforcement Learning Agent Learning Human Agent Interaction Learning Signal

July 16, 2024

A Comparative Analysis of Interactive Reinforcement Learning Algorithms in Warehouse Robot Grid Based Environment
Arunabh Bora
Comparative Study Q Learning Training Robot Interactive Reinforcement Learning Warehouse Robot Unmanned Warehouse

May 29, 2024

Advancing Household Robotics: Deep Interactive Reinforcement Learning for Efficient Training and Enhanced Performance
Arpita Soni, Sujatha Alla, Suresh Dodda, Hemanth Volikatla
Reinforcement Learning Deep Reinforcement Learning Efficient Training Performance Improvement Home Robot Interactive Reinforcement Learning

May 6, 2024

Robot Air Hockey: A Manipulation Testbed for Robot Learning with Reinforcement Learning
Caleb Chuck, Carl Qi, Michael J. Munje, Shuozhe Li, Max Rudolph, Chang Shi, Siddhant Agarwal, Harshit Sikchi, Abhinav Peri, Sarthak Dayal, Evan Kuo, Kavan Mehta, Anthony Wang, Peter Stone, Amy Zhang, Scott Niekum
Reinforcement Learning Robot Learning RL Agent Interactive Reinforcement Learning Air Hockey Manipulation Domain

February 21, 2024

Leveraging Demonstrator-perceived Precision for Safe Interactive Imitation Learning of Clearance-limited Tasks
Hanbit Oh, Takamitsu Matsubara
Multidimensional Local Precision Rate Interactive Reinforcement Learning Real Robot Experiment Interactive Imitation Learning

January 2, 2024

Reinforcement Learning for SAR View Angle Inversion with Differentiable SAR Renderer
Yanni Wang, Hecheng Jia, Shilei Fu, Huiping Lin, Feng Xu
Reinforcement Learning Synthetic Aperture Radar Synthetic Aperture Radar Data Interactive Reinforcement Learning

November 17, 2023

From "Thumbs Up" to "10 out of 10": Reconsidering Scalar Feedback in Interactive Reinforcement Learning
Hang Yu, Reuben M. Aronson, Katherine H. Allen, Elaine Schaertl Short
Interactive Reinforcement Learning Binary Feedback Scalar Feedback

November 4, 2023

Accelerating Reinforcement Learning of Robotic Manipulations via Feedback from Large Language Models
Kun Chu, Xufeng Zhao, Cornelius Weber, Mengdi Li, Stefan Wermter
Language Model Reinforcement Learning Human Feedback Manipulation Task Robot Manipulation Interactive Reinforcement Learning

July 11, 2023

Boosting Feedback Efficiency of Interactive Reinforcement Learning by Adaptive Learning from Scores
Shukai Liu, Chenming Wu, Ying Li, Liangjun Zhang
Adaptive Learning Performance Score Sparse Reward Environment Pairwise Preference Interactive Reinforcement Learning User Feedback

June 28, 2023

Diversity is Strength: Mastering Football Full Game with Interactive Reinforcement Learning of Multiple AIs
Chenglu Sun, Shuo Shen, Sijia Xu, Weidong Zhang
Deep Reinforcement Learning Diversity Awareness Estimated Team Strength Advanced AI Google Research Football AI Competition Interactive Reinforcement Learning Artificial Intelligence Training

December 6, 2022

Reinforcement Learning for UAV control with Policy and Reward Shaping
Cristian Millán-Arias, Ruben Contreras, Francisco Cruz, Bruno Fernandes
Reinforcement Learning Unmanned Aerial Vehicle Autonomous Agent Reinforcement Learning Agent Reward Shaping Health Policy Recommendation Drone Control Interactive Reinforcement Learning

October 14, 2022

Multi-trainer Interactive Reinforcement Learning System
Zhaori Guo, Timothy J. Norman, Enrico H. Gerding
Sparse Reward Environment Interactive Reinforcement Learning

October 11, 2022

Broad-persistent Advice for Interactive Reinforcement Learning Scenarios
Francisco Cruz, Adam Bignold, Hung Son Nguyen, Richard Dazeley, Peter Vamplew
Autonomous Agent Interactive Reinforcement Learning Broad Persistent Advice

July 20, 2022

Quantifying the Effect of Feedback Frequency in Interactive Reinforcement Learning for Robotic Tasks
Daniel Harnack, Julie Pivin-Bachler, Nicolás Navarro-Guerrero
Reinforcement Learning Mixed Effect Action Space Robot Control Robotic Task Interactive Reinforcement Learning

July 5, 2022

Ask-AC: An Initiative Advisor-in-the-Loop Actor-Critic Framework
Shunyu Liu, Kaixuan Chen, Na Yu, Jie Song, Zunlei Feng, Mingli Song
Actor Critic Interactive Reinforcement Learning Actor Critic Framework User in the Loop

January 19, 2022

Online POI Recommendation: Learning Dynamic Geo-Human Interactions in Streams
Dongjie Wang, Kunpeng Liu, Hui Xiong, Yanjie Fu
Link Stream Temporal Interaction Reinforcement Learning Environment Interactive Reinforcement Learning POI Recommendation

December 2, 2021

Towards Interactive Reinforcement Learning with Intrinsic Feedback
Benjamin Poole, Minwoo Lee
Reinforcement Learning Neural Activity Brain Signal Feedback Mechanism Interactive Reinforcement Learning

Interactive Reinforcement Learning

Papers

Prosody as a Teaching Signal for Agent Learning: Exploratory Studies and Algorithmic Implications

A Comparative Analysis of Interactive Reinforcement Learning Algorithms in Warehouse Robot Grid Based Environment

Advancing Household Robotics: Deep Interactive Reinforcement Learning for Efficient Training and Enhanced Performance

Robot Air Hockey: A Manipulation Testbed for Robot Learning with Reinforcement Learning

Leveraging Demonstrator-perceived Precision for Safe Interactive Imitation Learning of Clearance-limited Tasks

Reinforcement Learning for SAR View Angle Inversion with Differentiable SAR Renderer

From "Thumbs Up" to "10 out of 10": Reconsidering Scalar Feedback in Interactive Reinforcement Learning

Accelerating Reinforcement Learning of Robotic Manipulations via Feedback from Large Language Models

Boosting Feedback Efficiency of Interactive Reinforcement Learning by Adaptive Learning from Scores

Diversity is Strength: Mastering Football Full Game with Interactive Reinforcement Learning of Multiple AIs

Reinforcement Learning for UAV control with Policy and Reward Shaping

Multi-trainer Interactive Reinforcement Learning System

Broad-persistent Advice for Interactive Reinforcement Learning Scenarios

Quantifying the Effect of Feedback Frequency in Interactive Reinforcement Learning for Robotic Tasks

Ask-AC: An Initiative Advisor-in-the-Loop Actor-Critic Framework

Online POI Recommendation: Learning Dynamic Geo-Human Interactions in Streams

Towards Interactive Reinforcement Learning with Intrinsic Feedback