Hybrid FedGraph: An efficient hybrid federated learning algorithm using graph convolutional neural network
Authors: Jaeyeon Jang, Diego Klabjan, Veena Mendiratta, Fanfei Meng
Organizations: Department of Data Science, The Catholic University of Korea · Department of Industrial Engineering and Management Sciences, Northwestern University · Department of Electrical and Computer Engineering, Northwestern University
Federated learning is an emerging paradigm for decentralized training of machine learning models on distributed clients, without revealing the data to the central server. Most existing works have focused on horizontal or vertical data distributions, where each client possesses different samples with shared features, or each client fully shares only sample indices, respectively. However, the hybrid scheme is much less studied, even though it is much more common in the real world. Therefore, in this paper, we propose a generalized algorithm, FedGraph, that introduces a graph convolutional neural network to capture feature-sharing information while learning features from a subset of clients. We also develop a simple but effective clustering algorithm that aggregates features produced by the deep neural networks of each client while preserving data privacy.
Figures & tables
Figure 1: The data distribution patterns of (a) HFL, (b) VFL, and (c) HBFL.
Figure 2: Medical diagnosis: A use case example of HBFL
Figure 3: An overview of the training procedure for the proposed model includes: (a) training each client’s model, (b) obtaining the optimal feature extractor from each client, and (c) training the server’s GCN using embeddings produced by the clients’ feature extractors.
Figure 4: An illustrative example of graph construction for a GCN based on a three-client scenario.
Hyperparameter
Range
Number of hidden nodes
24∼28
Number of hidden layers
3∼10
Learning rate
10−4∼10−1
Table 1: Summary on hyperparameter optimization.
Figure 5: Validation results for Fashion MNIST regarding: (a) the number of hidden layers and (b) the cluster size δ . In (b), we assess the accuracy of collaborative predictions made by the trained GCN.
Baseline
Random clustering
K-means
CRC
Accuracy
0.766
0.153
0.669
0.844
Table 2: Comparison of clustering algorithm in terms of accuracy for use in our federated learning algorithm.