Federated Offline Reinforcement Learning

Federated offline reinforcement learning (FORL) addresses the challenge of training reinforcement learning agents collaboratively across multiple decentralized datasets without direct environment interaction, respecting data privacy. Current research focuses on developing algorithms that efficiently aggregate locally-trained policies, often employing techniques like dual regularization, momentum-based methods, and ensemble learning, to mitigate distributional shifts and improve sample efficiency. This approach is particularly significant for applications like personalized medicine and robotics where data is distributed, privacy is paramount, and online data collection is impractical or expensive, enabling the development of robust and efficient AI systems in these constrained environments.

Papers

May 24, 2024

February 8, 2024

Federated Offline Reinforcement Learning: Collaborative Single-Policy Coverage Suffices
Jiin Woo, Laixi Shi, Gauri Joshi, Yuejie Chi
Offline Reinforcement Learning Single Agent Tabular Markov Decision Process Federated Offline Reinforcement Learning

January 7, 2024

Decentralized Federated Policy Gradient with Byzantine Fault-Tolerance and Provably Fast Convergence
Philip Jordan, Florian Grötschla, Flint Xiaofeng Fan, Roger Wattenhofer
Federated Reinforcement Learning Byzantine Robust Byzantine Robust Federated Learning Exponential Convergence Rate Byzantine Fault Tolerance Federated Offline Reinforcement Learning

May 22, 2023

FEDORA: Flying Event Dataset fOr Reactive behAvior
Amogh Joshi, Adarsh Kosta, Wachirawit Ponghiran, Manish Nagaraj, Kaushik Roy
Optical Flow Event Based Camera Event Data Lateral Velocity Reactive Behavior Vision Based Autonomous System Federated Offline Reinforcement Learning

May 21, 2023

Federated Offline Policy Learning with Heterogeneous Observational Data
Aldo Gael Carranza, Susan Athey
Policy Learning Regret Analysis Federated Offline Reinforcement Learning

May 4, 2023

Federated Ensemble-Directed Offline Reinforcement Learning
Desik Rengarajan, Nitin Ragothaman, Dileep Kalathil, Srinivas Shakkottai
Offline Reinforcement Learning Federated Prompt Cooperation Federated Offline Reinforcement Learning Federated Learning Platform

June 11, 2022

Federated Offline Reinforcement Learning
Doudou Zhou, Yufeng Zhang, Aaron Sonabend-W, Zhaoran Wang, Junwei Lu, Tianxi Cai
Markov Decision Process Offline Reinforcement Learning Policy Optimization Offline Reinforcement Learning Algorithm Dynamic Treatment Regime Federated Offline Reinforcement Learning