Federated Automatic Differentiation

Federated automatic differentiation (FAD) extends automatic differentiation techniques to federated learning, enabling the computation of gradients across decentralized data and communication boundaries. Current research focuses on developing efficient FAD algorithms for various model architectures, including those based on stochastic approximation, expectation-maximization, and reinforcement learning, while addressing challenges posed by data heterogeneity and communication costs. This work is significant because it facilitates the development of more sophisticated and efficient federated learning algorithms, improving privacy-preserving model training across diverse datasets and accelerating convergence in applications ranging from intrusion detection to reinforcement learning.

Papers

November 2, 2024

Optimizing Federated Learning by Entropy-Based Client Selection
Andreas Lutz, Gabriele Steidl, Karsten Müller, Wojciech Samek
Deep Learning Federated Learning Label Distribution Client Selection Federated Automatic Differentiation

August 11, 2024

On the Convergence of a Federated Expectation-Maximization Algorithm
Zhixu Tao, Rajita Chandak, Sanjeev Kulkarni
Early Stage Convergence Convergence Rate Expectation Maximization Federated Learning Algorithm Federated Automatic Differentiation

July 17, 2024

Private and Federated Stochastic Convex Optimization: Efficient Strategies for Centralized Systems
Roie Reshef, Kfir Y. Levy
Differential Privacy Stochastic Optimization Stochastic Convex Optimization Simple Strategy Gradient Obfuscation Federated Automatic Differentiation

February 6, 2024

SCAFFLSA: Taming Heterogeneity in Federated Linear Stochastic Approximation and TD Learning
Paul Mangold, Sergey Samsonov, Safwan Labbi, Ilya Levin, Reda Alami, Alexey Naumov, Eric Moulines
Sample Complexity Stochastic Approximation Linear Function Approximation Communication Complexity Heterogeneous Agent TD Learning Federated Automatic Differentiation

June 2, 2023

Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation
Davoud Ataee Tarzanagh, Mingchen Li, Pranay Sharma, Samet Oymak
Convergence Guarantee Stochastic Approximation Optimal Rate Approximation Theory Compositional Optimization Federated Automatic Differentiation Hypergradient Estimation

May 27, 2023

Federated Empirical Risk Minimization via Second-Order Method
Song Bian, Zhao Song, Junze Yin
Empirical Risk Minimization Second Order Kernel Regression P$ Norm Interior Point Federated Automatic Differentiation

January 18, 2023

Federated Automatic Differentiation
Keith Rush, Zachary Charles, Zachary Garrett
Loss Function Automatic Differentiation FedAvg Algorithm Federated Automatic Differentiation

June 21, 2022

Federated Stochastic Approximation under Markov Noise and Heterogeneity: Applications in Reinforcement Learning
Sajad Khodadadian, Pranay Sharma, Gauri Joshi, Siva Theja Maguluri
Q Learning Federated Reinforcement Learning Linear Speedup Slow Convergence Markovian Sampling Federated Automatic Differentiation

May 23, 2022

FedSA: Accelerating Intrusion Detection in Collaborative Environments with Federated Simulated Annealing
Helio N. Cunha Neto, Ivana Dusparic, Diogo M. F. Mattos, Natalia C. Fernandes
Intrusion Detection Intrusion Detection System Attack Detection Collaborative Environment Federated Automatic Differentiation

April 9, 2022

Divergence-aware Federated Self-Supervised Learning
Weiming Zhuang, Yonggang Wen, Shuai Zhang
Self Supervised Siamese Network Federated Self Supervised Learning Federated Automatic Differentiation

November 3, 2021

Federated Expectation Maximization with heterogeneity mitigation and variance reduction
Aymeric Dieuleveut, Gersende Fort, Eric Moulines, Geneviève Robin
Expectation Maximization Latent Variable Model Variance Reduction Expectation Maximization Algorithm Federated Automatic Differentiation

Federated Automatic Differentiation

Papers

Optimizing Federated Learning by Entropy-Based Client Selection

On the Convergence of a Federated Expectation-Maximization Algorithm

Private and Federated Stochastic Convex Optimization: Efficient Strategies for Centralized Systems

SCAFFLSA: Taming Heterogeneity in Federated Linear Stochastic Approximation and TD Learning

Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation

Federated Empirical Risk Minimization via Second-Order Method

Federated Automatic Differentiation

Federated Stochastic Approximation under Markov Noise and Heterogeneity: Applications in Reinforcement Learning

FedSA: Accelerating Intrusion Detection in Collaborative Environments with Federated Simulated Annealing

Divergence-aware Federated Self-Supervised Learning

Federated Expectation Maximization with heterogeneity mitigation and variance reduction