Federated Reinforcement Learning

Federated Reinforcement Learning (FRL) aims to collaboratively train reinforcement learning agents across multiple decentralized devices without directly sharing their private data, addressing privacy concerns while leveraging distributed computational resources. Current research focuses on overcoming challenges posed by data heterogeneity across devices, employing algorithms like Federated Q-learning, policy gradient methods (including natural policy gradient and actor-critic variants), and addressing issues of convergence and communication efficiency through techniques such as momentum and ADMM. FRL's significance lies in its potential to enable large-scale, privacy-preserving applications in diverse fields, including recommendation systems, medical imaging, and resource allocation in networked systems like smart grids and V2X networks.

Papers

October 3, 2024

Joint Channel Selection using FedDRL in V2X
Lorenzo Mancini, Safwan Labbi, Karim Abed Meraim, Fouzi Boukhalfa, Alain Durmus, Paul Mangold, Eric Moulines
Channel Estimation Federated Reinforcement Learning Cellular V2X Vehicle to Everything V2X Technology

September 27, 2024

Enhancing Spectrum Efficiency in 6G Satellite Networks: A GAIL-Powered Policy Learning via Asynchronous Federated Inverse Reinforcement Learning
Sheikh Salman Hassan, Yu Min Park, Yan Kyaw Tun, Walid Saad, Zhu Han, Choong Seon Hong
Gene Level GNN Policy Learning Federated Reinforcement Learning Traditional Reinforcement Learning Spectrum Efficiency Wireless Network Optimization Generative Adversarial Imitation

September 23, 2024

FedSlate:A Federated Deep Reinforcement Learning Recommender System
Yongxin Deng, Xiaoyu Tan, Xihe Qiu, Yaochu Jin
Recommendation System Federated Reinforcement Learning Learning Based Recommender System Recommendation Benchmark

July 8, 2024

FedMRL: Data Heterogeneity Aware Federated Multi-agent Deep Reinforcement Learning for Medical Imaging
Pranab Sahoo, Ashutosh Tripathi, Sriparna Saha, Samrat Mondal
Multi Agent Reinforcement Learning Medical Imaging Data Heterogeneity Multi Agent Deep Reinforcement Learning Medical Datasets Federated Reinforcement Learning

May 29, 2024

May 24, 2024

May 6, 2024

Federated Reinforcement Learning with Constraint Heterogeneity
Hao Jin, Liangyu Zhang, Zhihua Zhang
Policy Gradient Proximal Policy Optimization Primal Dual Federated Reinforcement Learning Natural Policy Gradient Heterogeneity Challenge

March 29, 2024

CAESAR: Enhancing Federated RL in Heterogeneous MDPs through Convergence-Aware Sampling with Screening
Hei Yi Mak, Flint Xiaofeng Fan, Luca A. Lanzendörfer, Cheston Tan, Wei Tsang Ooi, Roger Wattenhofer
Federated Reinforcement Learning Convergence AwarE Sampling Mass Convergence Event

March 27, 2024

FRESCO: Federated Reinforcement Energy System for Cooperative Optimization
Nicolas Mauricio Cuadrado, Roberto Alejandro Gutierrez, Martin Takáč
Federated Reinforcement Learning Hierarchical Control Energy Market Selfish Agent Ancient FreSCo

March 20, 2024

Federated reinforcement learning for robot motion planning with zero-shot generalization
Zhenyuan Yuan, Siyuan Xu, Minghui Zhu
Zero Shot Federated Reinforcement Learning Robot Motion Planning Policy Adaptation Zero Shot Generalization

March 15, 2024

Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries
Swetha Ganesh, Jiayu Chen, Gugan Thoppe, Vaneet Aggarwal
Policy Gradient Federated Reinforcement Learning Adversary Agent Adversarial Agent Global Convergence Guarantee

February 8, 2024

Reinforcement Learning as a Catalyst for Robust and Fair Federated Learning: Deciphering the Dynamics of Client Contributions
Jialuo He, Wei Chen, Xiaojin Zhang
Reinforcement Learning Adversarial Attack Deep Reinforcement Learning Native Robustness Robust Version Client Contribution Federated Reinforcement Learning FAIR Federated Learning

January 27, 2024

Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learning
Chenyu Zhang, Han Wang, Aritra Mitra, James Anderson
Multi Agent Collaboration Federated Reinforcement Learning Finite Time State Action Space Reinforcement Learning Task

January 7, 2024

Decentralized Federated Policy Gradient with Byzantine Fault-Tolerance and Provably Fast Convergence
Philip Jordan, Florian Grötschla, Flint Xiaofeng Fan, Roger Wattenhofer
Federated Reinforcement Learning Byzantine Robust Byzantine Robust Federated Learning Exponential Convergence Rate Byzantine Fault Tolerance Federated Offline Reinforcement Learning

November 21, 2023

Resilient Control of Networked Microgrids using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations
Sayak Mukherjee, Ramij R. Hossain, Sheik M. Mohiuddin, Yuan Liu, Wei Du, Veronica Adetola, Rohit A. Jinsiwale, Qiuhua Huang, Tianzhixi Yin, Ankit Singhal
Federated Reinforcement Learning DC Microgrid Application Diverse Design Resilient Control

November 3, 2023

Epidemic Decision-making System Based Federated Reinforcement Learning
Yangxi Zhou, Junping Du, Zhe Xue, Zhenhui Pan, Weikang Chen
Federated Reinforcement Learning Epidemic Decision Epidemic Data

November 1, 2023

Federated Natural Policy Gradient and Actor Critic Methods for Multi-task Reinforcement Learning
Tong Yang, Shicong Cen, Yuting Wei, Yuxin Chen, Yuejie Chi
Multi Task Policy OpTimization Federated Reinforcement Learning Multi Task Reinforcement Learning Tabular Markov Decision Process

October 15, 2023

Federated Reinforcement Learning for Resource Allocation in V2X Networks
Kaidi Xu, Shenglong Zhou, Geoffrey Ye Li
Reinforcement Learning Resource Allocation Federated Reinforcement Learning Cellular V2X V2X Communication Vehicle to Everything

Federated Reinforcement Learning

Papers

Joint Channel Selection using FedDRL in V2X

Enhancing Spectrum Efficiency in 6G Satellite Networks: A GAIL-Powered Policy Learning via Asynchronous Federated Inverse Reinforcement Learning

FedSlate:A Federated Deep Reinforcement Learning Recommender System

FedMRL: Data Heterogeneity Aware Federated Multi-agent Deep Reinforcement Learning for Medical Imaging

Momentum for the Win: Collaborative Federated Reinforcement Learning across Heterogeneous Environments

Federated Q-Learning with Reference-Advantage Decomposition: Almost Optimal Regret and Logarithmic Communication Cost

Federated Offline Policy Optimization with Dual Regularization

Momentum-Based Federated Reinforcement Learning with Interaction and Communication Efficiency

Federated Reinforcement Learning with Constraint Heterogeneity

CAESAR: Enhancing Federated RL in Heterogeneous MDPs through Convergence-Aware Sampling with Screening

FRESCO: Federated Reinforcement Energy System for Cooperative Optimization

Federated reinforcement learning for robot motion planning with zero-shot generalization

Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries

Reinforcement Learning as a Catalyst for Robust and Fair Federated Learning: Deciphering the Dynamics of Client Contributions

Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learning

Decentralized Federated Policy Gradient with Byzantine Fault-Tolerance and Provably Fast Convergence

Resilient Control of Networked Microgrids using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations

Epidemic Decision-making System Based Federated Reinforcement Learning

Federated Natural Policy Gradient and Actor Critic Methods for Multi-task Reinforcement Learning

Federated Reinforcement Learning for Resource Allocation in V2X Networks