Transfer Reinforcement Learning

Transfer reinforcement learning (TRL) aims to accelerate reinforcement learning (RL) by leveraging knowledge from previously learned tasks to improve performance on new, related tasks. Current research focuses on improving sample efficiency and generalization across tasks with varying reward functions, transition dynamics, and even action spaces, employing techniques like successor features, goal-conditioned policies, and various model-based and model-free approaches including deep Q-networks and actor-critic methods. These advancements are significant because they enable faster and more robust RL in complex environments, with applications ranging from robotics and multi-agent systems to resource management and air traffic control. The development of provably efficient algorithms and better metrics for evaluating transferability are also key areas of ongoing investigation.

Papers

November 6, 2024

Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency from Shifted-Dynamics Data
Chengrui Qu, Laixi Shi, Kishan Panaganti, Pengcheng You, Adam Wierman
Reinforcement Learning Online Reinforcement Learning Sample Efficiency Transfer Reinforcement Learning Dynamic Shift Data Shift

October 28, 2024

The Limits of Transfer Reinforcement Learning with Latent Low-rank Structure
Tyler Sam, Yudong Chen, Christina Lee Yu
Reinforcement Learning Continuum Limit Inner Structure Transferable Representation Intermediate Latent LOw Rank Transfer Reinforcement Learning

October 18, 2024

Transfer Reinforcement Learning in Heterogeneous Action Spaces using Subgoal Mapping
Kavinayan P. Sivakumar, Yan Zhang, Zachary Bell, Scott Nivison, Michael M. Zavlanos
Transfer Learning Action Space Transfer Reinforcement Learning Subgoal Search

June 3, 2024

Multi-Agent Transfer Learning via Temporal Contrastive Learning
Weihao Zeng, Joseph Campbell, Simon Stepputtis, Katia Sycara
Contrastive Learning Goal Conditioned Policy Transfer Reinforcement Learning Goal Conditioned Agent

May 24, 2024

SF-DQN: Provable Knowledge Transfer using Successor Feature for Deep Reinforcement Learning
Shuai Zhang, Heshan Devaka Fernando, Miao Liu, Keerthiram Murugesan, Songtao Lu, Pin-Yu Chen, Tianyi Chen, Meng Wang
Deep Reinforcement Learning Q Learning Knowledge Transfer Transfer Reinforcement Learning Successor Feature Generalized Policy Improvement

March 10, 2024

Distributional Successor Features Enable Zero-Shot Policy Optimization
Chuning Zhu, Xinqi Wang, Tyler Han, Simon S. Du, Abhishek Gupta
Reinforcement Learning Transfer Reinforcement Learning Model Transferability Occupancy Model

February 13, 2024

Enabling Multi-Agent Transfer Reinforcement Learning via Scenario Independent Representation
Ayesha Siddika Nipu, Siming Liu, Anthony Harris
Multi Agent Reinforcement Learning Multi Agent Knowledge Transfer Multi Agent Learning Multi Agent Challenge Task Agnostic Representation Transfer Reinforcement Learning

January 23, 2024

Multi-Agent Based Transfer Learning for Data-Driven Air Traffic Applications
Chuhao Deng, Hong-Cheol Choi, Hyunsang Park, Inseok Hwang
Bidirectional Encoder Representation Pre Trained BERT Air Traffic Transfer Reinforcement Learning Air Traffic Controller

December 19, 2023

Value Explicit Pretraining for Learning Transferable Representations
Kiran Lekkala, Henghui Bao, Sumedh Sontakke, Laurent Itti
Value Function Transferable Representation Generalizable Representation Transfer Reinforcement Learning

November 12, 2023

An advantage based policy transfer algorithm for reinforcement learning with metrics of transferability
Md Ferdous Alam, Parinaz Naghizadeh, David Hoelzle
Reinforcement Learning Reinforcement Learning Algorithm Task Transferability Metric Library Advantage Feedback Transfer Reinforcement Learning Policy Transfer

September 22, 2023

H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps
Haoyi Niu, Tianying Ji, Bingqi Liu, Haocheng Zhao, Xiangyu Zhu, Jianying Zheng, Pengfei Huang, Guyue Zhou, Jianming Hu, Xianyuan Zhan
Reinforcement Learning Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Efficient Framework Transfer Reinforcement Learning Real Robot Experiment Offline to Online Reinforcement Learning Dynamic Mismatch

August 14, 2023

IOB: Integrating Optimization Transfer and Behavior Transfer for Multi-Policy Reuse
Siyuan Li, Hao Li, Jin Zhang, Zhen Wang, Peng Liu, Chongjie Zhang
Reinforcement Learning Transfer Reinforcement Learning Transfer Optimization Dexterous Policy Target Policy

June 2, 2023

Efficient Multi-Task and Transfer Reinforcement Learning with Parameter-Compositional Framework
Lingfeng Sun, Haichao Zhang, Wei Xu, Masayoshi Tomizuka
Multi Task Multi Task Reinforcement Learning Multi Task Training Compositional Model Transfer Reinforcement Learning

February 18, 2023

Reinforcement Learning in the Wild with Maximum Likelihood-based Model Transfer
Hannes Eriksson, Debabrota Basu, Tommy Tram, Mina Alibeigi, Christos Dimitrakakis
Reinforcement Learning Markov Decision Process Wild Challenge Transfer Reinforcement Learning Maximum Mean Caution Aware Transfer

November 4, 2022

Robotic Assembly Control Reconfiguration Based on Transfer Reinforcement Learning for Objects with Different Geometric Features
Yuhang Gai, Bing Wang, Jiwen Zhang, Dan Wu, Ken Chen
Reinforcement Learning Arbitrary Object Assembly Task Geometric Feature Transfer Reinforcement Learning Reconfigurable Robot Self Reconfiguration Compliance Controller

October 13, 2022

Transfer Deep Reinforcement Learning-based Large-scale V2G Continuous Charging Coordination with Renewable Energy Sources
Yubao Zhang, Xin Chen, Yuchen Zhang
Electric Vehicle Transfer Reinforcement Learning Renewable Energy Source Vehicle to Grid

September 26, 2022

DEFT: Diverse Ensembles for Fast Transfer in Reinforcement Learning
Simeon Adebola, Satvik Sharma, Kaushik Shivakumar
Reinforcement Learning Policy Gradient Ensemble Learning Diverse Ensemble Deep Ensemble Actor Critic Transfer Reinforcement Learning Efficient Transfer

August 4, 2022

Transferable Multi-Agent Reinforcement Learning with Dynamic Participating Agents
Xuting Tang, Jia Xu, Shusen Wang
Multi Agent Reinforcement Learning Shot Learning Model Adaptation Centralized Training Transfer Reinforcement Learning Collaborative Agent

May 27, 2022

Provably Sample-Efficient RL with Side Information about Latent Dynamics
Yao Liu, Dipendra Misra, Miro Dudík, Robert E. Schapire
Reinforcement Learning Reinforcement Learning Algorithm Robust Policy Latent Dynamic Sample Efficient Latent State Side Information Transfer Reinforcement Learning

February 9, 2022

Transferred Q-learning
Elynn Y. Chen, Michael I. Jordan, Sai Li
Transfer Learning Q$ Learning Transfer Reinforcement Learning Function Estimation Q$ Learning Algorithm Target Policy

Transfer Reinforcement Learning

Papers

Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency from Shifted-Dynamics Data

The Limits of Transfer Reinforcement Learning with Latent Low-rank Structure

Transfer Reinforcement Learning in Heterogeneous Action Spaces using Subgoal Mapping

Multi-Agent Transfer Learning via Temporal Contrastive Learning

SF-DQN: Provable Knowledge Transfer using Successor Feature for Deep Reinforcement Learning

Distributional Successor Features Enable Zero-Shot Policy Optimization

Enabling Multi-Agent Transfer Reinforcement Learning via Scenario Independent Representation

Multi-Agent Based Transfer Learning for Data-Driven Air Traffic Applications

Value Explicit Pretraining for Learning Transferable Representations

An advantage based policy transfer algorithm for reinforcement learning with metrics of transferability

H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps

IOB: Integrating Optimization Transfer and Behavior Transfer for Multi-Policy Reuse

Efficient Multi-Task and Transfer Reinforcement Learning with Parameter-Compositional Framework

Reinforcement Learning in the Wild with Maximum Likelihood-based Model Transfer

Robotic Assembly Control Reconfiguration Based on Transfer Reinforcement Learning for Objects with Different Geometric Features

Transfer Deep Reinforcement Learning-based Large-scale V2G Continuous Charging Coordination with Renewable Energy Sources

DEFT: Diverse Ensembles for Fast Transfer in Reinforcement Learning

Transferable Multi-Agent Reinforcement Learning with Dynamic Participating Agents

Provably Sample-Efficient RL with Side Information about Latent Dynamics

Transferred Q-learning